The old code would start in a mixed state, where all the reference
frames were pointing to frame buffer 0, but the reference counts
were 0. This is why we needed special code for the first frame.

Change-Id: I734961012917654ff8c0c8b317aac00ab75ded1a