| Site last updated: | 09/04/05 |
| This page last updated: | 09/25/04 |
Next: The tables for 32-bit Up: A faster way to Previous: Introduction
The tables and potential overflow
In this section we will look at the tables and show how they were chosen in order to avoid overflow. We will first examine the tables that were used for 32-bit platforms and then the corresponding tables for 16-bit platforms. In order to estimate the potential overflow we will first consider the case of 8-bit sample values. In this case the JPEG Standard states in Annex F.1.1.4: ``The quantized DCT coefficient values are signed, two’s complement integers with 11-bit precision for 8-bit input precision...''. This simply means that the DCT coefficients after dequantization have a range of 10000000000...11111111111, 00000000000, 00000000001, ...01111111111 or in decimals: -1024 ...-1, 0, 1, ...1023. In the following we will use 1024 for worst case scenarios as the maximum input value and will simply disregard the sign of the constants that need to be absorbed into the dequantization table values, while keeping in mind that the dequantization table values have a range of 0 ...255.
Subsections
