This commit fixes a mismatched use case of block size in non-RD
intra prediction check. The residual SSE and variance should be
calculated per transform block size, instead of operating block
size, which caused chrome valgrind warning on conditional jump
based on uninitialized value (webm issue 823). This commit
resolves this issue.

Change-Id: I595c06599c7e0fd0e4a08736519ba68fc14bc79a

Change-Id: I3d9130e726a1299fd258f6dfe93315e2d12f76da

Changed to use defined inline functions consistently through
the code.

Change-Id: I7644d24fa7a837378564a6e0790416d3725dd200

Use a weaker filter for second level arf frames.
Average gain across all sets and metrics ~0.3%

Remove code for arnr_type which is no longer
supported in VP9 which always uses a centered blur.

Re-factor and some cleanup.

Change-Id: Ieb4b8940e99e4e02b3fcc9fca6f2d4109e6ed639

Currently, vp9_diamond_search_sadx4() is only called when sse3 is
enabled, which is improper since sse2 optimization of sdx4df
functions are available. Changed to always use
vp9_diamond_search_sadx4().

Change-Id: I4b95d6b7a3c6c645783c373f0ba8d645ece24717

There are sse2 optimization of sdx4df functions. Instead of calling
vp9_refining_search_sadx4 only when sse3 is enabled, call it always.

Change-Id: I24f93818f7d4209d1425039e0eb099ff9ff08fe9

Change-Id: I3a3ceeeed489f8b1ccd7199ff97f3fb991bbf5a4

Deleted vp9_find_best_sub_pixel_comp_tree(), and combined it in
vp9_find_best_sub_pixel_tree().

Change-Id: Ifb25763c8b19822df5537cc1daa76ce88dc3b056

Use FAST_HEX in speed 5 and 6, which covers more points than
FAST_DIAMOND and improves motion search quality.

At speed 6, RTC set borg tests showed slight quality gain (psnr
gain: 0.143%, ssim gain: 0.226%). No noticeable encoding speed
change.

Change-Id: Ifa62875d9a52ee382ec494f271382bb77d8c67bf

This commit combined the full pel and sub pel motion search into a
single function to avoid code duplication. The commit does not change
encoder outputs.

Change-Id: Ibe18342c4f64073bef20f9cf6c6ca0a20d01bf0d

This commit enables a new quantization process for 32x32 2D-DCT
transform coefficient blocks. It improves the compression
performance of speed 5 by 1.4%. The overall compression gains of
speed 5 due to the new quantization scheme is 4.7%. It also includes
the SSSE3 implementation of the 32x32 quantization process.

Change-Id: I0855b124fd6462418683f783f5bcb44255c9993b

In the previous version, only certain buffers in the macroblockd were saved and
the restored. In this version, all of the buffers are saved and restored. The
code was then rolled into a loop for readability.

Also contains a tiny fix for when the -DOUTPUT_YUV_DENOISED flag is used.

Change-Id: Id925ef8b3fa122ae88acfa1d9a1e4df45df83518

This commit further simplifies the SSSE3 implementation of the fast
path quantization process.

Change-Id: I5be3286ec0f1bd81d1cf5be3168fece6384fb9ca

Change-Id: Id6eedc502c86433df1456dd994aee6bc9a1359a2

* Replace max_step_search_steps with constant MAX_MVSEARCH_STEPS
* Fold (reduce_first_step_size + speed > 5) into reduce_first_step_size
  replacing uses of reduce_first_step_size that don't add the speed
  check with zero.

Change-Id: Iae46395dbf3eaca138bf4d18b838a9e364b5a198

Change-Id: I39a37335ba5b3a969d328afb1f425ddb2cf7ddda

vp9_rdopt is for making rd optimal mode decisions. vp9_rd is for all
other rd related routines. Anything used outside of making an rd optimal
decision belongs in rd.

Change-Id: I772a3073f7588bdf139f551fb9810b6864d8e64b

Change-Id: I262ecac02d376de83097bb40f744f5584e987603

Moved the threshold adjustment before reference flag checking,
which could set the threshold to INT_MAX for disabled reference
frame, and cause overflow if the adjustment is done after that.

Change-Id: I85e94f8726d5e3ae93f65965aa978721dddc9957

Renamed updating_running_avg() to filter(). Extended function with the rest of
the filter procedure. Made all of the empirically-determined constants used in
VP8 into functions so they can be tweaked more easily.

Change-Id: I41730c8c92370c76885950a43742347477ca4e7e

As in VP8.

Currently, this parameter is set with the VP8E_SET_NOISE_SENSITIVITY flag.
The flag was not renamed so that we don't break the interface for webrtc. This
should probably be changed at some point in the future.

Change-Id: Ic73fcb0dde9d1d019e9d042050b617333ac65472

Change-Id: Icbe05657f0e92c3838e6a5a975f4f82d21328a2e

Add test code to turn multi-arf on and off depending
on group length and zero motion.

Changes to active max group length for mult-arf.

Fund second arf only from normal frame bits.

Change-Id: I920287fac1c886428c15a39f731a25d07c2b796c

This commit added a speed feature to control the step_param used in
full pixel motion search. The intention is to reduced the search
steps for high speed real time coding.

Change-Id: I21d2f0105c2b647783a6688615da7fcf2b6d670b

Adapt the use of segmentation in AQ mode 2 based on
the ambient kf/arf/gf Q.

Disable segmentation where the rate per SB is very
low and overheads are likely to outweigh the benefits.

This patch reduces the -ve average metrics impact
of AQ mode 2 while allowing stronger 3 segment AQ
in some cases. Average improvement ~0.5-1.0%.

Change-Id: I5892dfcc7507c5cc6444531cc7fe17554cf8d0c7

This commit re-designs the quantization process for transform
coefficient blocks of size 4x4 to 16x16. It improves compression
performance for speed 7 by 3.85%. The SSSE3 version for the
new quantization process is included.

The average runtime of the 8x8 block quantization is reduced
from 285 cycles -> 255 cycles, i.e., over 10% faster.

Change-Id: I61278aa02efc70599b962d3314671db5b0446a50

Change-Id: I2fc4ecfc2dd3ff1dd241a68c9ed4c280291b41f2

Add a conditional compile flag for this feature. Also add a
switch to enable the encoder to use these statistics in the
second pass. Currently, the switch is turned off.

Change-Id: Ia1c858c35ec90e36f19f5cffe156b97ddaa04922

The current threshold is knid of low, and in many cases NEWMV
mode is checked but not picked as the best mode. This patch
added a speed feature to increase NEWMV threshold, so that
less partition mode checking goes to check NEWMV. This feature
is enabled for speed 6 and 7.

Rtc set borg tests showed:
1. Speed 6, overall psnr: -0.088%, ssim: -1.339%;
   Average speedup on rtc set is 11.1%.
2. Speed 7, overall psnr: -0.505%, ssim: -2.320%
   Average speedup on rtc set is 12.9%.

Change-Id: I953b849eeb6e0d5a1f13eacba30c14204472c5be

Change-Id: Id52a7869fd1f31bb060de170e3295da7435adb9e

Grayscale is conditionally compiled.

Change-Id: I482ab237560d0bae8d397fd9999e78d38104f2a1

Fixed the signed/unsigned mismatch.

Change-Id: Id83d603b8f1745b71f4cf695a0751e55518b1316

Encoder still uses SWITCHABLE as default via DEFAULT_INTERP_FILTER,
but does not override the default if it is not SWITCHABLE.

Change-Id: I3c0f6653bd228381a623a026c66599b0a87d01d5

Moved the encode_breakout_test out of vp9_pick_inter_mode().

Change-Id: I6966d0293ae5210a5a28b0e8debacb24d1c0d2d4

When the frame is intra coded only, the encoder takes the RD
coding flow. Hence the function set_mode_info is not practically
in use. This commit removes it and the associated conditional
branches.

Change-Id: I1e42659ceb55b771ba712d1cdecacb446aa6460d

For real time speed 7, once encode breakout is on(i.e. encoding
setting --static-thresh=1), a proper encode breakout threshold
is set to speed up the encoder.

Set --static-thresh=1, RTC set borg test showed a slight overall
psnr loss of 0.162%, but ssim gain of 0.287%. The average speedup
on RTC set is 6%, and for some clips, the speedup can be 10+%.

Change-Id: Id522d9ce779ff7c699936d13d0c47083de4afb85

Before encoding a frame, calculate and store each 16x16 block's
variance of source difference between last and current frame.
Find partitioning threshold T for the frame from its variance
histogram, and then use T to make partition decisions.

Comparing with fixed 16x16 partitioning, rtc set test showed an
overall psnr gain of 3.242%, and ssim gain of 3.751%. The best
psnr gain is 8.653%.

The overall encoding speed didn't change much. It got faster for
some clips(for example, 12% speedup for vidyo1), and a little
slower for others.

Also, a minor modification was made in datarate unit test.

Change-Id: Ie290743aa3814e83607b93831b667a2a49d0932c

Change-Id: I7f989d197444d166133ad91eb23ac1033109f58d

As a way to speed-up rtc encoding at speed 7.

Change-Id: Ie36a010392cf7b741dc130df21a4e733622a75b7

To experiment with different speed/quality compromises.

Change-Id: Ia9d4b85243554d620498a327da37c356e752b07f