• Martin Storsjö's avatar
    mpegvideo: Do REBASE_PICTURE with byte pointers · a65f965c
    Martin Storsjö authored
    REBASE_PICTURE (more specifically, this half of it) takes a Picture
    pointer that points into one larger struct, finds the offset of
    that Picture within the struct and finds the corresponding field
    within another instance of a similar struct.
    
    The pointer difference "pic - (Picture*)old_ctx" is a value given
    in sizeof(Picture) units, and when applied back on
    (Picture*)new_ctx gets multiplied back with sizeof(Picture). Many
    compilers seem to optimize out this division/multiplication, but
    not all do.
    
    GCC 4.2 on OS X doesn't seem to remove the division/multiplication,
    therefore the new pointer didn't turn out to point to exactly
    the right place in the new struct since it only had sizeof(Picture)
    granularity (and the Picture is not aligned on a sizeof(Picture)
    boundary within the encompassing struct). This bug has been present
    before 47318953
    
     as well - with H264, pointers to h->ref_list[0][0]
    pointed to 88 bytes before h->ref_list[0][0] after the rebase. After
    shrinking Picture, the difference ended up even larger, making
    writes via such a Picture pointer overwrite other fields at random
    in H264Context, ending up in crashes later.
    
    This fixes H264 multithreaded decoding on OS X with GCC 4.2.
    
    Signed-off-by: default avatarMartin Storsjö <martin@martin.st>
    a65f965c