| Site last updated: | 09/04/05 |
| This page last updated: | 09/25/04 |
Next: The tables for 12-bit Up: The tables and potential Previous: The tables for 32-bit
The tables for 16-bit platforms
For 16-bit platforms we have to make sure that any operation's result is less than 215 if the int data type is used and 231 if the INT32 data type is used for storing the result. If USE_INACCURATE_IDCT is not defined on a 16-bit platform, the intermediate results of the multiplications are kept in INT32 variables, but are scaled to int values after the first pass. Additionally, the following fixed-point constants are used to implement the table of constants that are absorbed into the dequantization table:
128, 116, 64, 41, 64, 27, 64, 23, 116, 105, 58, 37, 58, 25, 58, 21, 64, 58, 32, 20, 32, 14, 32, 12, 41, 37, 20, 13, 20, 9, 20, 7, 64, 58, 32, 20, 32, 14, 32, 12, 27, 25, 14, 9, 14, 6, 14, 5, 64, 58, 32, 20, 32, 14, 32, 12, 23, 21, 12, 7, 12, 5, 12, 4
The problem here is that the quantization tables are implemented as an int array, therefore the maximum value of this table (128) times the maximum value in a dequantization table (255) may not exceed 215 - 1. This can for maximum accuracy only be done with this table, since 128×255 = 32640 < 215 = 32768. Therefore this table must be chosen which is simply the table with the real values presented before, but this time scaled with 32 = 25. If as in this case INT32 variables are used for intermediate results in the first pass, no overflow will happen, since (128 + 116 + 41 + 27 + 23)×1024 = 343040 < 231. But in order to store this result in a (16-bit) int variable that can be added to itself 5 times during the second pass, this value needs to be scaled by a right shift over 6 digits (= division by 26 = 64). This leads to a maximum value of 343040/64 = 5360. In the second pass, if this value is added 5 times to itself, this yields a maximum of 5×5360 = 26800 < 215. Finally, the second pass scales the final result by a right shift over 4 digits to obtain the pixel values in the spatial domain.
If USE_INACCURATE_IDCT is defined on a 16-bit platform, the following fixed-point constants are used to implement the table of constants that are absorbed into the dequantization table:
8, 7, 4, 3, 4, 2, 4, 1, 7, 7, 4, 2, 4, 2, 4, 1, 4, 4, 2, 1, 2, 1, 2, 1, 3, 2, 1, 1, 1, 1, 1, 0, 4, 4, 2, 1, 2, 1, 2, 1, 2, 2, 1, 1, 1, 0, 1, 0, 4, 4, 2, 1, 2, 1, 2, 1, 1, 1, 1, 0, 1, 0, 1, 0
Again, this table is simply the table with the real values presented before, but this time scaled with 2 = 21. This table allows all intermediate variables in both passes to be (16-bit) int variables: In the first pass, (8 + 7 + 3 + 2 + 1)×1024 = 21504 does not exceed 215 - 1. In order for the second pass not to lead to an overflow this value needs to be scaled by a right shift over 2 digits to be added to itself 5 times in the second pass: 21504/4 = 5376 and 5×5376 = 26880 < 231. Finally, the second pass must end with a right shift over 4 digits to obtain the pixel values in the spatial domain.
