If a reference frame is inter, the only valid modes would
be inter modes.  This check is unnecessary.

Change-Id: Ib8433ab5a3418f94149ee4e3062d48d7740d225a