AccurateRip: Difference between revisions
m (→Checksum calculation: 1776 is still the largest offset) |
m (→Checksum calculation: "CRC" not accurate terminology for AR checksum. I was careful everywhere else to use the word checksum, but forgot here b/c Spoon called it a CRC in the post I linked to.) |
||
| Line 23: | Line 23: | ||
===Checksum calculation=== | ===Checksum calculation=== | ||
Technical details on | Technical details on checksum calculation can be found here: [http://forum.dbpoweramp.com/showthread.php?20641-AccurateRip-CRC-Calculation AccurateRip-CRC-Calculation] | ||
Each AccurateRip checksum is based on a complete track rip, from the beginning of the track to the end, as determined by the track's entry in the disc's table of contents. This means that any silence or "gap" at the beginning or end of the track, except before the first track, must be included in the rip. If your DAE software is configured to trim silence or to do anything with gaps other than put them at the end of the preceding track (as normally happens when gaps aren't taken into account), then it's unlikely the checksums of your rips will be submitted to or compared against those in the AccurateRip database. | Each AccurateRip checksum is based on a complete track rip, from the beginning of the track to the end, as determined by the track's entry in the disc's table of contents. This means that any silence or "gap" at the beginning or end of the track, except before the first track, must be included in the rip. If your DAE software is configured to trim silence or to do anything with gaps other than put them at the end of the preceding track (as normally happens when gaps aren't taken into account), then it's unlikely the checksums of your rips will be submitted to or compared against those in the AccurateRip database. | ||
Revision as of 16:59, 10 September 2012
AccurateRip is a online ripping accuracy database. Using submissions from people all around the world, a bit-perfect verification of an audio track rip can be assured with a positive match. AccurateRip also allows the offsets of drives to be determined.
The database is maintained by Illustrate Ltd., the company run by "Spoon", primary developer of dBpoweramp. All of the data is submitted by the Windows-based ripping programs dBpoweramp and Exact Audio Copy via a DLL licensed from Illustrate.[1] Other rippers and tools, such as CUETools and Mac OS-based rippers, can receive info from the database in order to look up drive offsets or check rip quality, but they cannot submit their own rip results to it.
Drive read offsets
Very few CD drives actually start reading data from audio CDs exactly at the sector requested by DAE software. There are drives that are off by over 1 sector (1/75th of a second), but most are off by much less (1/250 to 1/350 second). Most modern CD drives have "Accurate Stream" technology, so there's no "jitter", meaning in this case that the variance is consistent from read to read, and will tend to be the same for all drives of a certain make & model.
The AccurateRip database allows one to find out the read offset, which is normally constant for given make & model of CD drive. This number can then be used by DAE software to ensure that each track is ripped from its exact start to its exact finish.
The offset is given in samples. One "sample" on an audio CD is 4 bytes, consisting of a 2-byte left-channel value and a 2-byte right-channel value. There are 2352 bytes, or 588 samples, in each sector of an audio CD, corresponding to 1/75th of a second of sound. Therefore, an AccurateRip offset of +134 means the drive consistently delivers data from 536 bytes behind (earlier than) where it was asked to read from, so the DAE software needs to look that far ahead (hence the positive offset) in order to get the right data.
When offsets are taken into account, the DAE software might have to ask the drive to "overread" into the lead-in or lead-out portions of the disc, where there's no audio data. Some drives can't be asked to do it, some drives will try to do it and fail, and some will just return null samples (a stream of "0" bytes, a.k.a. digital silence). If the drive can't overread, then there will be samples missing from the extracted track. The DAE software can correct for this by padding the track with digital silence so it's the correct length.
Offset accuracy
Determining the actual read offset of a drive is difficult. The reference measurements Andre Wiethoff made and which were adopted by Spoon for the AccurateRip database were challenged in late 2006; strong evidence was presented to support a claim that the reference is actually off by 30 samples. The offsets are 30 samples too low, or the correction values are 30 samples too high, however you want to look at it. Wiethoff feels that it's too late to change to a different reference now that the database is populated. However, since all submissions are calibrated to the same reference, and the apparent error so small, the reliability of AccurateRip data is not compromised in any way whatsoever.
Ripped track checksums
Once all the samples for a track have been extracted and put into a file such as a WAV, a checksum can be generated to summarize the sample data. Identical data will produce identical checksums. If the data is the slightest bit different, the checksums will usually be very different. The checksums derived from the same tracks from the same pressings of the same CDs, so long as drive offsets have been accounted for, can be compared in order to determine whether the extraction was error-free. That is, if you rip a track and find that your checksum matches what everyone else got, then you can be confident there are no missing or incorrect samples (or that you've all got exactly the same damage, which is nearly impossible). See the secure ripping article for more on this subject.
The AccurateRip database contains over 2 million unique discs. DAE software can use this info to decide whether to try re-reading a track that produced a different checksum than was expected.
Checksum calculation
Technical details on checksum calculation can be found here: AccurateRip-CRC-Calculation
Each AccurateRip checksum is based on a complete track rip, from the beginning of the track to the end, as determined by the track's entry in the disc's table of contents. This means that any silence or "gap" at the beginning or end of the track, except before the first track, must be included in the rip. If your DAE software is configured to trim silence or to do anything with gaps other than put them at the end of the preceding track (as normally happens when gaps aren't taken into account), then it's unlikely the checksums of your rips will be submitted to or compared against those in the AccurateRip database.
The checksum algorithm ignores the first 2939 samples[2] (just under 5 frames, a little over 0.013s of audio) at the beginning of the first track, and ignores 2940 samples (exactly 5 frames) at the end of the last track. The largest drive offset in the database, as of late 2011, is only 1776 samples, so when an offset-corrected drive "overreads" beyond the boundaries of the audio data, it normally won't affect the checksum. If the overread were to exceed 2940 samples, though, the algorithm assumes the data is padded with digital silence (nothing but zeroes). If your drive isn't capable of overreading and your DAE software doesn't zero-pad the rip to simulate an overread, then it's unlikely the checksums of your rips will be submitted to or compared against those in the AccurateRip database.
An optimization oversight in the original AccurateRip checksum algorithm results in an unintended loss of accuracy: about 3% of the audio data is not counted in the checksum at all. The left channel's samples are fully included, but in a 65,536-sample cycle, half of the right-channel samples are treated as if they're missing anywhere from 1 bit to all 16 bits. Proposals for improving the algorithm, the database, and the database's API were made and an improved algorithm was implemented for AccurateRip v2 checksums, which are now the default calculated by the rippers which support AccurateRip. The v2 checksums are treated as separate pressings from the old ones, so there's no risk to the integrity of existing data.
Pressings
AccurateRip v2 was introduced to lesson the impact of pressings. Pressings, for AccurateRip's purposes, are where a batch of identical audio CDs is created by the pressing plant, then another batch is created, usually at a later date, with the same data but a different offset. The offset is due to variations in the way the master discs are replicated for the pressing machinery. Apart from this offset, discs across pressings are identical; the audio data on them is exactly the same. Previously, AccurateRip could only verify results for a specific pressing (same data, same offset), but an AccurateRip v2 compatible program is able to verify a pressing not currently in the database by using results from any of the other pressings that have the same data with a different offset.
AccurateRip data analysis
It is possible to run statistical analysis on the hundreds of millions of submissions to AccurateRip to determine how accurate a given drive is, with the idea that with enough spread of drive submissions, anomalous results from damaged CDs will be averaged out. The last such calculation can be found here: CD-DVD-Drive-Accuracy-List-2011
There is also a report of drive offsets here: CD Drive Offsets
Submitting offsets and checksums
The database is designed to be anonymously accessed by DAE software, via HTTP. Submitting data should only be done through such software.
Software that uses AccurateRip
History
AccurateRip was introduced as an add-on for dBpowerAMP Music Converter (dMC) in January 2003.
Notes and references
- ↑ As per a Jan. 2010 post by Spoon, "Only two programs submit to AR, EAC and dBpoweramp, these two submit through the standard AccurateRip access dll, which ensures that offsets are correct, we are not talking submissions to freedb where one record does not effect the submission of future records if keyed with a wrong offset. You might not like it, but it is the only way to ensure that the quality of the database is not impaired."
- ↑ Not 2940 samples; see http://jonls.dk/2009/10/calculating-accuraterip-checksums/