If you don't eat something for a couple of years, could your taste buds change and start disliking that food?

I use to like fish, but my brother is allergic to it so we don't eat it. Now about 5 years later I don't like it. Is it a mental thing or have my taste buds changed?