Hydrogenaudio Listening Tests
From time to time, various HydrogenAudio members have conducted listening tests, both private and public, to evaluate the merit of lossy encoding technologies at a point in time. These test results are provided for the explicit purpose of documenting the performance of various codecs in relation to one another, in turn providing useful data in gauging the best, most efficient encoder over certain bitrates at the time the test was conducted.
It should be noted that these tests may become obsolete over time as development of the tested codecs progress. With that in mind, please ensure that you consult the most recent test results before drawing conclusions.
When you read the test results, you may come across unfamiliar terminology, especially concerning artifacts. ff123's Artifact Training Page is the place to go to understand what these 'artifacts' are all about.
Note: There is no such thing as listening tests for Lossless compression, as recordings encoded using Lossless codecs are, by nature, bit-identical in their output to their respective original uncompressed recordings. When testing Lossless codecs, only performance characteristics are taken into account (e.g. compression ration, encode/decode speed, etc). If you are interested in these aspects of lossless encoding, take a look at the Lossless comparison page, instead.
Format-specific Tests
All tests sorted by date, with older ones at top. All dates are in ISO8601 format.
AAC
| Date | Title Samples |
Performed by | Type | Codecs | Links |
|---|---|---|---|---|---|
| 2004-12-09 | Ahead vs Apple 128-kbps Classical |
Guruboolez | Private | iTunes 4.1.052, 4.7.042; Nero 2.5.9.7, 2.9.998 CBR, 2.9.998 VBR |
HA Thread |
| 2005-09-05 | 48- & 64-kbps HE-AAC Classical |
Guruboolez | Private | CT aacPlus v.2 48- & 64-kbps Helix 11.0.0.1897, Nero 3.2.0.20 VBR |
HA Thread |
| 2006-02-05 | 48-kbps HE-AAC Classical |
Ivan Dimkovic | Public | Nero HE-AAC v1 & v2 | HA Thread |
| 2006-02-20 | 48-kbps HE-AAC General |
Gabriel | Public | 3gpp (v1), CT (v1 & v2), Nero (v1 & v2) | HA Thread Complete Result |
MP3
| Date | Title Samples |
Performed by | Type | Codecs | Links |
|---|---|---|---|---|---|
| 2004-01-18 | Dithering Test (1) Special |
Guruboolez | Private | MP3 Decoders: MAD, LAME, Fraunhofer, foobar2000 |
HA Thread |
| 2004-11-15 | 130-kbps LAME rjamorim |
Guruboolez | Private | LAME 3.90.3 ABR, 3.96.1 ABR, 3.96.1 VBR | HA Thread |
| 2005-01-11 | 128-kbps LAME ff123 |
Guruboolez | Private | LAME 3.90.3, 3.96.1, 3.97a5 | HA Thread |
| 2005-10-17 | LAME 3.98 -V5 Classical |
Guruboolez | Private | LAME 3.98: -V5, -V5 --vbr-new, -V5 --vbr-new --athaa-sensitivity 1 |
HA Thread |
| 2008-10-17 | 128-kbps Various |
Sebastian Mares | Public | LAME 3.97, LAME 3.98.2, iTunes, FhG, Helix | HA Thread Complete Result |
Notes: (1) Performed under extreme listening conditions not reproducible on daily listening.
MPC
| Date | Title Samples |
Performed by | Type | Codecs | Links |
|---|---|---|---|---|---|
| 2005-06-18 | --standard Classical |
Guruboolez | Private | 1.01j, 1.15u | HA Thread |
Vorbis
| Date | Title Samples |
Performed by | Type | Codecs | Links |
|---|---|---|---|---|---|
| 2003-06-18 | 128-kbps ff123/rjamorim |
Guruboolez | Private | GT2 & PCVS | HA Thread |
| 2004-06-17 | 160-kbps Special |
Guruboolez | Private | CVS, GT3b2, aoTuVb2, aoTuVb2+QK | HA Thread |
| 2004-02-05 | -q4 Transients |
Guruboolez | Private | RC3 2002-03, RC3 2002-04, 1.01, 1.01 QK2 | HA Thread |
WMA
| Date | Title Samples |
Performed by | Type | Codecs | Links |
|---|---|---|---|---|---|
| 2003-06-18 | 128-kbps ff123/rjamorim |
Guruboolez | Private | WMA9 Standard & Pro | HA Thread |
Multiformat Tests
| Date | Title Samples |
Performed by | Type | Codecs | Links |
|---|---|---|---|---|---|
| 2003-10-11 | 130-kbps Classical (Baroque) |
Guruboolez | Private | AAC (FAAC, PsyTel, Nero, QuickTime) MP3 LAME 3.90.3, MPC 1.14, Vorbis GT3b1, WMA9 Pro |
HA Thread |
| 2003-12-13 | 128-kbps Classical |
Guruboolez | Private | AAC (iTunes, Nero, FAAC), Vorbis 1.01, WMA9, MP3 LAME 3.93.1 |
HA Thread |
| 2004-07-12 | 175-kbps Classical |
Guruboolez | Private | Vorbis Megamix -q5,50, -q5,99 -q6,00 MP3 LAME 3.97a3 -V3, -V2, MPC -q5 |
|
| 2004-12-09 | 130-kbps Classical |
Guruboolez | Private | AAC Nero 2.9.9.998 Vorbis aoTuVb3 |
|
| 2005-03-05 | 140-kbps VBR Classical |
Guruboolez | Private | AAC Nero 3.0.0.4, Vorbis 1.1, QK28 | |
| 2005-03-17 | 256-kbps Transcode from MP3 subset of ff123 |
Guruboolez | Private | AAC iTunes 4.7, MP3 LAME 3.97a8 MPC 1.15u, Vorbis aoTuVb3, WavPack 4.2b3 Lossy |
|
| 2005-07-11 | 80-kbps Various |
Guruboolez | Private | LAME MP3, AAC-HE and -LC, Vorbis aoTuV, WMA Std |
HA Thread |
| 2005-08-09 | 96-kbps Classical & Various |
Guruboolez | Private | LAME MP3, AAC,Vorbis aoTuV (Lancer), MPC, WMA | Hardware.fr thread (in French) |
| 2005-08-22 | 180-kbps Classical |
Guruboolez | Private | AAC, MP3, MPC, Vorbis aoTuVb4 | HA Thread |
| 2006-01-13 | 128-kbps General |
Sebastian Mares | Public | iTunes & Nero AAC, LAME MP3, Vorbis aoTuV, WMA Pro |
HA Thread Complete Result |
| 2006-11-22 | 48-kbps General |
Sebastian Mares | Public | Vorbis aoTuV, Nero AAC, WMA Pro, WMA Standard |
HA Thread Complete Result |
| 2007-08-16 | 64-kbps General |
Sebastian Mares | Public | Vorbis aoTuV, Nero AAC, WMA Pro |
HA Thread Complete Result |
External Tests
Other people have held listening tests in venues outside of Hydrogenaudio (although the organizers of the tests tend to be HA members :). The results of these tests are found below:
- ff123's site on audio testing and listening tests (most recent test 2002-07-27)
- Roberto's Public Listening Tests (most recent test 2004-07-11)
- Sebastian Mares' Listening Tests (most recent test updated 2008-11-22)
- SoundExpert Listening Tests (continually updated)