From time to time, various HydrogenAudio members have conducted listening tests, both private and public, to evaluate the merit of lossy encoding technologies at a point in time. These test results are provided for the explicit purpose of documenting the performance of various codecs in relation to one another, in turn providing useful data in gauging the best, most efficient encoder over certain bitrates at the time the test was conducted.

It should be noted that these tests may become obsolete over time as development of the tested codecs progress. With that in mind, please ensure that you consult the most recent test results before drawing conclusions.

When you read the test results, you may come across unfamiliar terminology, especially concerning artifacts. ff123's Artifact Training Page is the place to go to understand what these 'artifacts' are all about.

Note: There is no such thing as listening tests for Lossless compression, as recordings encoded using Lossless codecs are, by nature, bit-identical in their output to their respective original uncompressed recordings. When testing Lossless codecs, only performance characteristics are taken into account (e.g. compression ration, encode/decode speed, etc). If you are interested in these aspects of lossless encoding, take a look at the Lossless comparison page, instead.

Format-specific Tests

All tests sorted by date, with older ones at top. All dates are in ISO8601 format.

AAC

Date	Title Samples	Performed by	Type	Codecs	Links
2004-12-09	Ahead vs Apple 128-kbps Classical	Guruboolez	Private	iTunes 4.1.052, 4.7.042; Nero 2.5.9.7, 2.9.998 CBR, 2.9.998 VBR	HA Thread
2005-09-05	48- & 64-kbps HE-AAC Classical	Guruboolez	Private	CT aacPlus v.2 48- & 64-kbps Helix 11.0.0.1897, Nero 3.2.0.20 VBR	HA Thread
2006-02-05	48-kbps HE-AAC Classical	Ivan Dimkovic	Public	Nero HE-AAC v1 & v2	HA Thread
2006-02-20	48-kbps HE-AAC General	Gabriel	Public	3gpp (v1), CT (v1 & v2), Nero (v1 & v2)	HA Thread Complete Result

MP3

Date	Title Samples	Performed by	Type	Codecs	Links
2004-01-18	Dithering Test (1) Special	Guruboolez	Private	MP3 Decoders: MAD, LAME, Fraunhofer, foobar2000	HA Thread
2004-11-15	130-kbps LAME rjamorim	Guruboolez	Private	LAME 3.90.3 ABR, 3.96.1 ABR, 3.96.1 VBR	HA Thread
2005-01-11	128-kbps LAME ff123	Guruboolez	Private	LAME 3.90.3, 3.96.1, 3.97a5	HA Thread
2005-10-17	LAME 3.98 -V5 Classical	Guruboolez	Private	LAME 3.98: -V5, -V5 --vbr-new, -V5 --vbr-new --athaa-sensitivity 1	HA Thread

Notes: (1) Performed under extreme listening conditions not reproducible on daily listening.

MPC

Date	Title Samples	Performed by	Type	Codecs	Links
2005-06-18	--standard Classical	Guruboolez	Private	1.01j, 1.15u	HA Thread

Vorbis

Date	Title Samples	Performed by	Type	Codecs	Links
2003-06-18	128-kbps ff123/rjamorim	Guruboolez	Private	GT2 & PCVS	HA Thread
2004-06-17	160-kbps Special	Guruboolez	Private	CVS, GT3b2, aoTuVb2, aoTuVb2+QK	HA Thread
2004-02-05	-q4 Transients	Guruboolez	Private	RC3 2002-03, RC3 2002-04, 1.01, 1.01 QK2	HA Thread

WMA

Date	Title Samples	Performed by	Type	Codecs	Links
2003-06-18	128-kbps ff123/rjamorim	Guruboolez	Private	WMA9 Standard & Pro	HA Thread

Multiformat Tests

Date	Title Samples	Performed by	Type	Codecs	Links
2003-10-11	130-kbps Classical (Baroque)	Guruboolez	Private	AAC (FAAC, PsyTel, Nero, QuickTime) MP3 LAME 3.90.3, MPC 1.14, Vorbis GT3b1, WMA9 Pro	HA Thread
2003-12-13	128-kbps Classical	Guruboolez	Private	AAC (iTunes, Nero, FAAC), Vorbis 1.01, WMA9, MP3 LAME 3.93.1	HA Thread
2004-07-12	175-kbps Classical	Guruboolez	Private	Vorbis Megamix -q5,50, -q5,99 -q6,00 MP3 LAME 3.97a3 -V3, -V2, MPC -q5	HA Thread
2004-12-09	130-kbps Classical	Guruboolez	Private	AAC Nero 2.9.9.998 Vorbis aoTuVb3	HA Thread
2005-03-05	140-kbps VBR Classical	Guruboolez	Private	AAC Nero 3.0.0.4, Vorbis 1.1, QK28	HA Thread
2005-03-17	256-kbps Transcode from MP3 subset of ff123	Guruboolez	Private	AAC iTunes 4.7, MP3 LAME 3.97a8 MPC 1.15u, Vorbis aoTuVb3, WavPack 4.2b3 Lossy	HA Thread
2005-07-11	80-kbps Various	Guruboolez	Private	LAME MP3, AAC-HE and -LC, Vorbis aoTuV, WMA Std	HA Thread
2005-08-09	96-kbps Classical & Various	Guruboolez	Private	LAME MP3, AAC,Vorbis aoTuV (Lancer), MPC, WMA	Hardware.fr thread (in French)
2005-08-22	180-kbps Classical	Guruboolez	Private	AAC, MP3, MPC, Vorbis aoTuVb4	HA Thread
2006-01-13	128-kbps General	Sebastian Mares	Public	iTunes & Nero AAC, LAME MP3, Vorbis aoTuV, WMA Pro	HA Thread Complete Result
2006-11-22	48-kbps General	Sebastian Mares	Public	Vorbis aoTuV, Nero AAC, WMA Pro, WMA Standard	HA Thread

External Tests

Other people also held listening tests outside of Hydrogenaudio (although the organizer of the tests are usually HA members :)

So, check out the following sites:

ff123's site on audio testing and listening tests (most recent test 2002-07-27)
Roberto's Public Listening Tests (most recent test 2004-07-11)
Sebastian Mares' Listening Tests (most recent test updated 2006-11-22)
SoundExpert Listening Tests (continually updated)