This commit allows the non-RD mode decision process to return the
rate and distortion costs associated with the selected mode.

Change-Id: Ibe0f67d323f65839fd9cb0a726c1219bf7b55da9

Brings back most of Jim's previous patch for choosing
partitioning based on variance while making it compatible
with the current state of the code. Also adds a
nonrd_use_partition() function to recursively encode  for any
arbitrary sb_type decisions within a 64x64 block; and
includes some refactoring.

Currently, when the VAR_BASED_PARTITIONING mode is turned on
for speed 7, there is a 10+% speed-up observed.

Experiments/improvements with this new partitioning method
will be conducted subsequently.

Change-Id: Ie6f43bfbde30583e941f450bf07c3b48828c9571

This commit adjusts the rate-distortion modeling for non-RD mode
decision. It puts more weights on energy from AC coefficients when
estimating the cost. The coding performance for rtc testset is
improved by 0.72%.

Change-Id: Ifa6ff11311a513ec2b10586589e82a9a21f6c61c

Assign interp_kernel value in MACROBLOCKD. This will be used to
select prediction filter coefficient sets and generate motion
compensated prediction.

Change-Id: I28c8dfb2dae6566f6939bb328aca5875c94bee65

Change-Id: I47b3c38aadfd8f3ea08515a18a5948aa1375c650

Adds a fast diamond search which is about 5% faster than FAST_HEX
with only a 0.1% drop in psnr when turned on for both speeds 5 and 7.
This search is turned on for speed 7.

Change-Id: I497630aa88a5148926086bb3038e7975e5f4eb98

This commit allows the non-RD mode decision to skip mode RD modelling
check, if the motion vector associated with the current mode is
same as that of NEARESTMV mode. This makes speed -7 about 2% faster.
Previous change that converts cost metric from SAD to model based RD
value makes the codec 6% slower at speed -7.

Change-Id: I30cfec5452f606a671b8432a2f7f0c94fbb49fc8

The rate-distortion model in non-RD coding mode is only applied to
luma component. This commit removed a few redundant addition steps.

Change-Id: Id8edc0a47c2dbef8deba43debe2c95db39454de3

This commit replaces SAD cost with modeled rate-distortion cost
for non-RD mode decision. It translates the prediction residual
SSE into estimate rate and reconstruction distorion costs, hence
capturing the quantization setting effect. The compression
performance of speed -7 for rtc set is improved by 14.79%.

Change-Id: Ifda014eb0501d13109fe7f92680bf1410b463632

Removes some unused variables and assignments

Change-Id: I228f43d658ce50237d65ce8dc8ab2ccf0a0b21ca

The core motion estimation fucntions all return sad now consistently.
The only exception is vp9_full_pixel_diamond(), however the core diamond
and refining search routines called from vp9_full_pixel_diamond() also
return SAD. If variance of pred error + mv cost is desired it must be
calculated explicitly outside these functions. For very fast encoding,
hopefully this will eliminate some redundant computations.

Also suggests reimplementing FAST_HEX with the vp9_pattern_search
framework. It is not exactly the same as the existing FAST_HEX, but
performance is slightly better and speed is very similar. Enables
removing a lot of duplicate code.

Change-Id: I152736393438c25bdf7e96b37cbb8ce330f4f94a

Change-Id: I916944950deb22f4c2301d83a803b732bf3ecd77

Change-Id: I16dd4d4aaae8ce6a482da3c9d142f41fe9155e82

Clear warning caused by implied conversion from int32 to int64.

Change-Id: I473b37a54984cbfb22702eb3f712465881ff86e8

Adds a speed 8 to VP9 where only the nearestmv (0 mv) is searched.
This seems to be about the same speed as vp8 speed 5.

Adds a new speed feature to disable inter modes based on a mask for
each blocksize.

Adds code for having lower complexity motion search methods
in nonrd pick mode function, even though speed 7 still uses DIAMOND
search for now.

Also uses HEX search for speed 6 rather than FAST_HEX which improves
psnr by 0.56% without any noticeable speed drop (tested on gipsmotion).

Change-Id: Ic13176572dbd3aed5884a26786940a4b1bbd8a75

Change-Id: Ib68d4a2de838254a66272324b909292048cbe60d

Change-Id: Icc057b819d80c608a6003fd7ef58aefed7a15824

This commit checks if the motion vector associated with the current
mode has been computed in previous mode tests. If possible, skip the
redundant reference block generation and SAD calculation in the
non-RD mode decision process.

For test sequence pedestrian_area 1080p, the runtime goes from
24261 ms to 23770 ms. This does not change compression performance.
The speed-up is mostly around places with consistent motion.

Change-Id: I97be63c6a2d07c57be26b3c600fbda3803adddda

Added fast HEX search while doing non-rd partition picking to
speed up the encoder.

Borg test(speed 7) on rtc set showed 1.8% overall PSNR loss.
Encoder speedup was 5% - 15% for different rtc clips.

Change-Id: I9c83026eabc70b69fcc747c90369ec60bfa3ca24

In non-RD mode decision, assign motion vector INVALID_MV when intra
prediction mode is selected.

Change-Id: I8f6ec39b71e755758f0f698074a2c17d934622ea

As Yunqing suggested, this commit makes non-RD mode decision always run
sub-pixel motion search in NEWMV mode. The compression performance
gains becomes fairly significant after we enabled sub-pixel accuracy
motion compensated prediction to calculate SAD cost.

For test sequences pedestrian_area at 1080p and vidyo1 at 720p, the
runtime goes slower by 5%. For rtc test set, the compression performance
is improved by 21.20%.

Change-Id: I38cbfdd5c53d79423e1fafb3154f8ddeed63bbf0

This commit builds the actual prediction block in sub-pixel accuracy
and uses which to calculate SAD for non-RD mode decision. In the trail
run on pedestrian_area at 1080p, rtc speed -7 runtime goes from
23495 ms -> 25107 ms (7% slower). The compression performance is
improved by 20.57% for rtc test set.

Change-Id: I438589cd103fe99f1b50c2d1939ac6ca43fa0157

Use a set of dedicated variables to buffer the current best mode
in non-RD mode decision. This allows to use mode_info for more
complicated test in the non-RD process.

Change-Id: I6024c9feb0662afd3eb29f7017f7b5a5446f303f

Change-Id: I5b3918441f63ceb7889dcf546e9a2d781be4a3cb

This commit makes a refactoring of the rtc_use_partition. It allows
the encoder to take a preferred block size for non-RD mode decision.
The boundary blocks are handled such that smaller block sizes that
fit in the boundary size will be used instread.

In rtc mode, the coding performance of speed -6 for pedestrian_1080p
goes from
158980 b/f, 38.934 dB, 22721 ms to
159008 b/f, 40.064 dB, 23721 ms.

For rtc set, the speed -6 compression performance is improved by
26%. Still about 2dB behind speed -5 at this point.

Change-Id: If0944f0880eaf1ad340bc325d97cea8d0f9dd53f

This commit enables the use of DC, vertical, and horizontal intra
prediction mode in rtc non-RD mode decision. When the best cost value
of inter modes is above a given threshold, the encoder runs the
above three intra modes and selects the one that has minimum
prediction residual in terms of SAD.

This together with recent changes on non-RD mode decision and coding
control improves compression performance of speed -6 by
derf  91%
yt    61%
hd    46%
stdhd 52%

In terms of encoding speed, it is about 3 times faster than speed -5.

Change-Id: I6b483bfd0307e6482bb22a6676ae4e25a52b1310

In the first coding run of a 64x64 block, check the coding mode
for each 8x8 block. Will need a second annealing stage to decide
the partition size to be encoded.

Change-Id: Ida9417805ff3358979b0c0429d4099c023c88866

Run sub-pixel motion search when NEWMV gives lower rate-distortion
cost. This improves coding performance of derf set by 8%, std-hd by
2.2%.

Change-Id: Ife50f7fda8463927784fe59a41cc439c833e941a

Change-Id: Ibe4276fa9ffeb2ef30871d6db8ac75c4648f1e58

Change-Id: If33a5a12c4025d9b5ec863dfccea7ee70f800665

Change-Id: I716ea2967c6771d7adea571406f315a3197eab93

Change-Id: I036e815ca63d00cba71202ae09ba0f6ef745dcb8

Cosmetic change to use pre-defined macros.

Change-Id: I93e9fa90113d0242599048940b39694660385a6f

This commit moves unnecessary mode_info update steps out of inter
mode test loop.

Change-Id: I3705450c44cda060021f4fd76c34f1b20d11658b

Based on frame level settings.

Change-Id: I922c09c195ab055585eaa4f544c51aceb1203a7a

Left shift of negative values caused IOC warnings.

Change-Id: I6f41b020ca0ff51f7861719d41393e9460b75d4e

This patch only works if the video is a width and height that are both
a multiple of 32..   It sets every partition to 16x16, and does INTRADC
only on the first frame and ZEROMV on every other frame.   It always does
does the largest possible transform, and loop filter level is set to 4.

Was ~20% faster than speed -5 of vp8

Now 20% slower but adds motion search ( every block ), nearest, near
and zeromv

The SVC test was changed because - while this realtime mode produces
bad quality albeit quickly, it isn't obeying all the rules it should
about which frames are available.

Change-Id: I235c0b22573957986d41497dfb84568ec1dec8c7

Change-Id: Ic9472f361f8749a165aa464c2fa859681b66381b

Change-Id: Ia63e2d05fa5dca3fb67788b0f1f28e9802ef27dd

This commit setups a test framework for real-time coding. It enables
a light motion search for non-RD mode decision purpose.

Change-Id: I8bec656331539e963c2b685a70e43e0ae32a6e9d