• Ronald S. Bultje's avatar
    vp10: skip unreachable cat6 token extrabits. · 3461e8ce
    Ronald S. Bultje authored
    We have historically added new bits to cat6 whenever we added a new
    transform size (or bitdepth, for that matter). However, we have
    always coded these new bits regardless of the actual transform size,
    which means that for smaller transforms, we code bits that cannot
    possibly be set. The coding (quality) impact of this is negligible,
    but the bigger issue is that this allows creating bitstreams with
    coefficient values that are nonsensible and can cause int overflows,
    which then de facto become part of the bitstream spec. By not coding
    these bits, we remove this possibility.
    
    See issue 1065.
    
    Change-Id: Ib3186eca2df6a7a15ddc60c8b55af182aadd964d
    3461e8ce