This commit enables a recursive partition type search for non-RD
mode decisions. It allows the encoder to choose variable block
sizes in a 64x64 block based on rate-distortion modeling.

It improves speed -6 coding efficiency for rtc set by 2.4%. Most
of the gains come from 32-40dB range, where many sequences gain
about 5% to 20%. Local tests suggest there is no speed change.

Change-Id: I06300016e500a21652812b7b3b081db39a783d66

This commit reformats non-RD coding flow layout to allow mode
decision with fixed and variable block sizes.

Change-Id: I2cdd3bb9f26c499ee4a9849004fd925cdd195d09

Change-Id: Id76a628495c822e23825b66a7589b4a3279680e2

Valgrind was complaining uninitialzed values are being used.

Change-Id: Ia9eb619f4e67d1fb7af1057d50304f15adc02aab

The flag x->skip_recode interacts badly with
the cpi->sf.use_nonrd_pick_mode and
cpi->sf.skip_encode_sb speed settings.

Restricting the use of the skip_decode flag when
these other speed choices are in use helps quality
for speeds 3 and 4 by a large amount with only a
small impact on speed.

Average improvmentes for 2 pass speed 4:
Derf +8.8%
Yt + 10.53%
Std-Hd +6.95%
yt-hd + 22.95%

Change-Id: I8010876d8012042a11077c92e69d813c3dfa58eb

In addition to a few cleanups.

Change-Id: Ice5938ef494513921a47e7c64ba9928f2202e24e

Change-Id: Id1fa36c92cb007b73a450cc8552e810cedad38b9

This is an initial attempt to allow variable block size partition
in non-RD coding flow. It tests 8x8, 16x16 and 32x32 block size per
64x64 block, all using non-RD mode decision and the associated rate
distortion costs from modeling, then selects the best block size to
encode the entire 64x64 block. Such operations are triggered every
other 3 frames. The blocks of intermediate frames will reuse the
collocated block's partition type.

It improves the compression performance by 13.2%. Note that the gains
are not evenly distributed. For many hard clips, the compression
performance is improved by 20% to 28%. Local speed test shows that
it will also increase runtime by 50%, as compared to speed -7. It is
now enabled in speed -6 setting.

Change-Id: Ib4fb8659d21621c9075b3c369ddaa9ecb0a4b204

Change-Id: I512482853bdf2695fbdf1c705a2ada354ccf76cf

The use of uninitialized skip flag will trigger inconsistency in
coding statistics, when alternate RD and non-RD coding modes are
enabled. This commit fixes this issue and removes unnecessary if
statements from update_state_rt.

Change-Id: I7d549dcb0e3ef48b999e5bbc78174ba84502cfcf

Activated using aq_mode=3.

Change-Id: Ied628b9e7bd0e88b0c75790276bca75b19eb5c07

Change-Id: I0d39bdbb8166ee44561f1008d1a2e76e70b36f30

This is for the purpose to test variable partition choices for the
same 64x64 block.

Change-Id: I0fab7351a62e7d3dfccd667e908bccb996ee677e

Change-Id: Iec885af58c95e65814fcb6ab528ce4e0874eb573

The block coding skip flags are assigned in the normal RD mode
decision loop. They are then used in the final encoding stage.
In the non-RD mode decision, the forward transform and quantization
stages are replaced by modeling based on SSE and variance of
prediction residues. This commit applies reset to this array in
the non-RD coding mode.

Change-Id: I66584669b035e9c8ac23e95047849ff277472742

This commit moves the position where rdmult is saved to make sure it
is the correct value. Prior, an uninitialized value may be saved and
restored.

This addresses issue:
https://code.google.com/p/webm/issues/detail?id=733

Change-Id: I436407f289169bc63da3c5a6bf609bed16cb71b5

This causes a build failure in VS 2008.

Change-Id: I03b0c0c006589b56d92a238d7b794711eb62575e

This function was subsumed by nonrd_use_partition, hence removed.

Change-Id: Id36757bc600ce6cf6ca03ad5df80268c4786d386

Change-Id: I12ce22c036b12073a45b762d4dd38f6ec4960e56

Combine the common variables and functions used by different
speed features.

Change-Id: Ifd1fa86200edbcc9e50b3fc3b1ba8a275bc3c17f

This commit unifies the non-RD partition use cases for both fixed
and variable block sizes. Deprecate and remove the separate function
for fixed partition type only.

Change-Id: I2b6cb945e90c1566f985adcebc4d0757480a8004

Brings back most of Jim's previous patch for choosing
partitioning based on variance while making it compatible
with the current state of the code. Also adds a
nonrd_use_partition() function to recursively encode  for any
arbitrary sb_type decisions within a 64x64 block; and
includes some refactoring.

Currently, when the VAR_BASED_PARTITIONING mode is turned on
for speed 7, there is a 10+% speed-up observed.

Experiments/improvements with this new partitioning method
will be conducted subsequently.

Change-Id: Ie6f43bfbde30583e941f450bf07c3b48828c9571

Clean-ups include
a. redundant code in rt -5 speed feature settings
b. code that guarantees square block availability in
rd_auto_partition_range()

Change-Id: Ic7b04d45b6dc15c461e0edbbb4e78aec20348291

These were discovered by ActiveMapTest.

Change-Id: Ifcd305ae2d954b6ec7edcaed0f80baf18c769e01

Change-Id: I894977e6655630823145c60c8258551bd3c8c76b

Change-Id: I6e2fec2814fd366a1ce1321208d3b0e8aa0d98b0

Change-Id: Ied12b39c55667b26fd3bf90eb331e601c53a10f6

Change-Id: I47b3c38aadfd8f3ea08515a18a5948aa1375c650

Change-Id: Ib105db0f3deeedf6a8402a292db3df1d77a3d7c1

Changing aq_mode type from int to AQ_MODE enum.

Change-Id: Ib7b5f0b70d02ded58a31dfade9c05a347f73beca

Change-Id: I446fca8aa11a4d4fc2b23d4b32348b74d74d0202

This commit replaces SAD cost with modeled rate-distortion cost
for non-RD mode decision. It translates the prediction residual
SSE into estimate rate and reconstruction distorion costs, hence
capturing the quantization setting effect. The compression
performance of speed -7 for rtc set is improved by 14.79%.

Change-Id: Ifda014eb0501d13109fe7f92680bf1410b463632

Properly set intra mode information entries.

Change-Id: Ie3a8992e415dcdd9087a55993bbca06087a0107a

The frame level transform size selection is done inside
encode_frame_internal().

Change-Id: Id7e5e417a3a3f88f875bf12b21bb8029bc6d0d82

Set speed features before running frame encoding. This avoids
redundant RD threshold calculation in key frame coding.

Change-Id: If8e3cf2c02976baa59b310c1c23af9eea0c46e36

Change-Id: I916944950deb22f4c2301d83a803b732bf3ecd77

Change-Id: I6d47618e8aee6e6c2fe1cf0b8bb4da0b2a0bab66

In addition, a local variable was renamed to avoid confusion.

Change-Id: Id4c497f9cfa219e8a414aa9fee9a85af5f147249

Adds a speed 8 to VP9 where only the nearestmv (0 mv) is searched.
This seems to be about the same speed as vp8 speed 5.

Adds a new speed feature to disable inter modes based on a mask for
each blocksize.

Adds code for having lower complexity motion search methods
in nonrd pick mode function, even though speed 7 still uses DIAMOND
search for now.

Also uses HEX search for speed 6 rather than FAST_HEX which improves
psnr by 0.56% without any noticeable speed drop (tested on gipsmotion).

Change-Id: Ic13176572dbd3aed5884a26786940a4b1bbd8a75

For blocks at frame boundary, the selected block size sometimes needs
to be smaller than that was first given. This commit forces such block
size change only between square blocks, so as to avoid the potential
use case containing 32x16 + 16x8 + 16x8, for 1080p sequences.

Local test suggested no visible coding speed difference. Borg test
reveals no difference in terms of compression performance.

Change-Id: Ie8de87f3c6febc3acf11b4cbfdf2077f9f6def52