The Verge did a blind test:
The results were very, very surprising to me. It was generally random across the board, though Spotify fared slightly worse than Apple Music and Tidal overall. In roughly 29 percent of the tests, subjects couldn’t tell any notable difference at all. Tidal — which wants you to pay more for lossless quality — most definitely didn’t take the crown, and in several cases, subjects actually identified it as the worst-sounding of the three.
Granted, it’s extremely hard to do a proper blind test of audio quality,1 and it’s arguable whether the “pick the best” method or an ABX test is more relevant. And I can quibble over some details of the test: the sample group was very small, they only tested three songs, and I find the Sony MDR-7506 headphones to be good for under $100, but not very good overall, and very poorly suited to this role.2
But even if you disagree with my nitpicks or have plenty of your own,3 it says a lot that in a fairly decent test, three different encodings (including lossless) weren’t reliably distinguishable.
If you need to try really hard to identify whether there’s any difference at all, it probably doesn’t matter.
Invest your energy and money in what matters so clearly and obviously that nobody needs to strain to hear the difference: great headphones or speakers fed by great recordings of great music.
The hardest problems: making sure each source is playing the same recording/mastering of the songs, and making sure the volume level is properly matched between them. If there’s even a slight, nearly imperceptible difference in volume levels — some say the difference can be as small as 0.1 dB — people will subconsciously favor the louder one. ↩︎
Most decent modern headphones resolve far more detail than the 7506, and its harsh midrange and rolled-off bass and treble can mask a lot, good and bad. Many of the specific complaints about the sound in The Verge’s video — “crunchy” cymbals, etc. — were identifying common flaws of the 7506, not the music’s encoding. This 24-year-old headphone is well-suited to monitoring cameras and sound boards cheaply, comfortably, and durably, not critical listening or enjoying music. ↩︎
Audiophiles will find much more about this test to complain about, like using an iPhone’s built-in amp and DAC and not using overpriced placebo-quackery cables, but none of these matter for this test. (Or ever, for the most part.) ↩︎