Let me add to the correct response that has already been given. It began with the division of the Marxist left into two groups: the “Mackerel left,” which refers to those who were poor and became hardcore Marxists (of course, the fish refers to what they eat, as mackerel is very cheap here), and the “Caviar left,” which refers to those who came from a bourgeoisie background, never had any need, but nonetheless became hardcore Marxists. “Those who were fed mackerel and those who were fed caviar” is mocked in the food reference.
The mackerel allusion faded away over time, but the caviar reference remained. As a result, it effectively became an adjective in and of itself. In today’s slang, “caviar” refers to what was previously stated: From an affluent background, an extreme socialist.
A Caviar is a non-poor individual who believes in socialist politics.
This is the sort of person I believe Stalin, Mao, and Polpot despised the most.