• Ronald S. Bultje's avatar
    Redo banding for all transforms. · 3120dbdd
    Ronald S. Bultje authored
    Now that the first AC coefficient in both directions use the same DC
    as their context, there no longer is a purpose in letting both have
    their own band. Merging these two bands allows us to split bands for
    some of the very high-frequency AC bands.
    
    In addition, I'm redoing the banding for the 1D-ADST col/row scans. I
    don't think the old banding made any sense at all (it merged the last
    coefficient of the first row/col in the same band as the first two of
    the second row/col), which was clearly an oversight from the band being
    applied in scan-order (rather than in their actual position). Now,
    coefficients at the same position will be in the same band, regardless
    what scan order is used. I think this makes most sense for the purpose
    of banding, which is basically "predict energy for this coefficient
    depending on the energy of context coefficients" (i.e. pt).
    
    After full re-training, together with previous patch, derf gains about
    1.2-1.3%, and hd/stdhd gain about 0.9-1.0%.
    
    Change-Id: I7a0cc12ba724e88b278034113cb4adaaebf87e0c
    3120dbdd