It's more of the brain being able to isolate specific sounds from the noisy environment
This is actually a benefit listening to classical music especially music with a degree of polyphony and where every individual part of the piece are equal and important not just the surface instruments.
If anything this actually increases the appreciation of music. If you can clearly hear every part of the piece of music equally instead of isolating the vocals or lead instruments above all other instruments. You can more appreciate the complete craftmanship of the music better.
It doesn't lend well to "lyrics" focus music though.