Today, the majority of Caviar is made from farm-raised sturgeon and, to a lesser degree, salmon eggs (though some claim that salmon eggs are not Caviar). The fish are slaughtered and the eggs extracted, then the carcass is transported for food processing. So, if you don’t think that fish farming is ethical, the answer is no. Yes, if you’re comfortable with it.
Caviar, as far as I know, has no mind, can’t act on its own, and bears no responsibility. As a result, it can’t be both ethical and immoral. Returning to your original query.