• Yunqing Wang's avatar
    Fix an integer overflow in vp9_mcomp.c · f22b828d
    Yunqing Wang authored
    The MV unit test revealed an integer overflow issue in vp9_mcomp.c.
    This was caused if the MV was very large. In mv_err_cost(), when
    mv->row = 8184, mv->col = 8184 and ref_mv is 0, mv_cost = 34363
    and error_per_bit = 132412, causing the overflow.
    
    BUG=webm:1406
    
    Change-Id: I35f8299f22f9bee39cd9153d7b00d0993838845e
    f22b828d