Commit 65bf4c9c authored by Martin Storsjö's avatar Martin Storsjö Committed by Reinhard Tartler

mpegvideo: Do REBASE_PICTURE with byte pointers

REBASE_PICTURE (more specifically, this half of it) takes a Picture
pointer that points into one larger struct, finds the offset of
that Picture within the struct and finds the corresponding field
within another instance of a similar struct.

The pointer difference "pic - (Picture*)old_ctx" is a value given
in sizeof(Picture) units, and when applied back on
(Picture*)new_ctx gets multiplied back with sizeof(Picture). Many
compilers seem to optimize out this division/multiplication, but
not all do.

GCC 4.2 on OS X doesn't seem to remove the division/multiplication,
therefore the new pointer didn't turn out to point to exactly
the right place in the new struct since it only had sizeof(Picture)
granularity (and the Picture is not aligned on a sizeof(Picture)
boundary within the encompassing struct). This bug has been present
before 47318953 as well - with H264, pointers to h->ref_list[0][0]
pointed to 88 bytes before h->ref_list[0][0] after the rebase. After
shrinking Picture, the difference ended up even larger, making
writes via such a Picture pointer overwrite other fields at random
in H264Context, ending up in crashes later.

This fixes H264 multithreaded decoding on OS X with GCC 4.2.

Fixes Bug: #439
Signed-off-by: 's avatarMartin Storsjö <martin@martin.st>
(cherry picked from commit a65f965c)
Signed-off-by: 's avatarReinhard Tartler <siretart@tauware.de>
parent d61c6ebc
...@@ -719,7 +719,7 @@ typedef struct MpegEncContext { ...@@ -719,7 +719,7 @@ typedef struct MpegEncContext {
#define REBASE_PICTURE(pic, new_ctx, old_ctx) (pic ? \ #define REBASE_PICTURE(pic, new_ctx, old_ctx) (pic ? \
(pic >= old_ctx->picture && pic < old_ctx->picture+old_ctx->picture_count ?\ (pic >= old_ctx->picture && pic < old_ctx->picture+old_ctx->picture_count ?\
&new_ctx->picture[pic - old_ctx->picture] : pic - (Picture*)old_ctx + (Picture*)new_ctx)\ &new_ctx->picture[pic - old_ctx->picture] : (Picture*) ((uint8_t*)pic - (uint8_t*)old_ctx + (uint8_t*)new_ctx))\
: NULL) : NULL)
/* mpegvideo_enc common options */ /* mpegvideo_enc common options */
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment