This allows for padding/trimming at the start of stream. By default, no
assumption is made about the first frame's expected pts, so no padding or
trimming is done.