optimize forward 16x16 DCT for accuracy
This commit added pre/post scaling for first half of fDCT16x16 to reduce error, by simulation of 100,000 blocks for random inputs, the average sse reduced from 2.1/block to 0.0498/block. also enabled tests for 16x16 fDCT and iDCT Change-Id: Id2a95f0464c6dd4118797d456237ae90274c0f02
Showing
Please register or sign in to comment