Commit 499fe05d authored by Yaowu Xu's avatar Yaowu Xu
Browse files

optimize forward 16x16 DCT for accuracy

This commit added pre/post scaling for first half of fDCT16x16 to
reduce error, by simulation of 100,000 blocks for random inputs,
the average sse reduced from 2.1/block to 0.0498/block.

also enabled tests for 16x16 fDCT and iDCT

Change-Id: Id2a95f0464c6dd4118797d456237ae90274c0f02
parent 0c9e2e9a
No related merge requests found
Showing with 5 additions and 440 deletions
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment