And make sure the nb_side_data field is reset as well.

Based on an initial patch from wm4 <nfxjfg@googlemail.com>.

CC: libav-stable@libav.org