Commit 78ecbc98 authored by Deb Mukherjee's avatar Deb Mukherjee
Browse files

Bug fix in probability update savings computation

Found this bug while tracking down some anomalies in my experiments.
Since vp8_cost_one and vp8_cost_zero return unsigned int, the
bit shift by 8 will be incorrect if the value is negative.
I am cautiously optimistic that this fix will make the prob
updates more correct and somewhat improve results across the board.
But the update probabilities will need to be retuned I think.

Patch 2: Adding more of the same fixes using a macro.

Change-Id: I1a168f040e74e8c67e7225103b1c2af9a611da49
Showing with 6 additions and 8 deletions
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment