swscale.txt 4.58 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
    The official guide to swscale for confused developers.
   ========================================================

Current (simplified) Architecture:
---------------------------------
                        Input
                          v
                   _______OR_________
                 /                   \
               /                       \
       special converter     [Input to YUV converter]
              |                         |
              |          (8bit YUV 4:4:4 / 4:2:2 / 4:2:0 / 4:0:0 )
              |                         |
              |                         v
              |                  Horizontal scaler
              |                         |
              |      (15bit YUV 4:4:4 / 4:2:2 / 4:2:0 / 4:1:1 / 4:0:0 )
              |                         |
              |                         v
              |          Vertical scaler and output converter
              |                         |
              v                         v
                         output


Ramiro Polla's avatar
Ramiro Polla committed
27 28
Swscale has 2 scaler paths. Each side must be capable of handling
slices, that is, consecutive non-overlapping rectangles of dimension
29 30 31
(0,slice_top) - (picture_width, slice_bottom)

special converter
Ramiro Polla's avatar
Ramiro Polla committed
32
    These generally are unscaled converters of common
33 34 35 36
    formats, like YUV 4:2:0/4:2:2 -> RGB15/16/24/32. Though it could also
    in principle contain scalers optimized for specific common cases.

Main path
Ramiro Polla's avatar
Ramiro Polla committed
37 38
    The main path is used when no special converter can be used. The code
    is designed as a destination line pull architecture. That is, for each
39 40 41 42 43 44 45 46
    output line the vertical scaler pulls lines from a ring buffer. When
    the ring buffer does not contain the wanted line then it is pulled from
    the input slice through the input converter and horizontal scaler, and
    the result is also stored in the ring buffer to serve future vertical
    scaler requests.
    When no more output can be generated because lines from a future slice
    would be needed, then all remaining lines in the current slice are
    converted, horizontally scaled and put in the ring buffer.
47 48 49 50 51
    [this is done for luma and chroma, each with possibly different numbers
     of lines per picture]

Input to YUV Converter
    When the input to the main path is not planar 8bit per component yuv or
Ramiro Polla's avatar
Ramiro Polla committed
52 53
    8bit gray then it is converted to planar 8bit YUV. 2 sets of converters
    exist for this currently, one performing horizontal downscaling by 2
Diego Biurrun's avatar
Diego Biurrun committed
54
    before the conversion and the other leaving the full chroma resolution
55
    but being slightly slower. The scaler will try to preserve full chroma
Ramiro Polla's avatar
Ramiro Polla committed
56 57
    here when the output uses it. It is possible to force full chroma with
    SWS_FULL_CHR_H_INP though even for cases where the scaler thinks it is
58 59 60
    useless.

Horizontal scaler
Ramiro Polla's avatar
Ramiro Polla committed
61
    There are several horizontal scalers. A special case worth mentioning is
Diego Biurrun's avatar
Diego Biurrun committed
62
    the fast bilinear scaler that is made of runtime generated MMX2 code
63
    using specially tuned pshufw instructions.
Ramiro Polla's avatar
Ramiro Polla committed
64 65
    The remaining scalers are specially tuned for various filter lengths.
    They scale 8bit unsigned planar data to 16bit signed planar data.
66
    Future >8bit per component inputs will need to add a new scaler here
Diego Biurrun's avatar
Diego Biurrun committed
67
    that preserves the input precision.
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90

Vertical scaler and output converter
    There is a large number of combined vertical scalers+output converters
    Some are:
    * unscaled output converters
    * unscaled output converters that average 2 chroma lines
    * bilinear converters                (C, MMX and accurate MMX)
    * arbitrary filter length converters (C, MMX and accurate MMX)
    And
    * Plain C  8bit 4:2:2 YUV -> RGB converters using LUTs
    * Plain C 17bit 4:4:4 YUV -> RGB converters using multiplies
    * MMX     11bit 4:2:2 YUV -> RGB converters
    * Plain C 16bit Y -> 16bit gray
      ...

    RGB with less than 8bit per component uses dither to improve the
    subjective quality and low frequency accuracy.


Filter coefficients:
--------------------
There are several different scalers (bilinear, bicubic, lanczos, area, sinc, ...)
Their coefficients are calculated in initFilter().
Diego Biurrun's avatar
Diego Biurrun committed
91 92
Horizontal filter coeffs have a 1.0 point at 1<<14, vertical ones at 1<<12.
The 1.0 points have been chosen to maximize precision while leaving a
93 94 95 96
little headroom for convolutional filters like sharpening filters and
minimizing SIMD instructions needed to apply them.
It would be trivial to use a different 1.0 point if some specific scaler
would benefit from it.
97
Also as already hinted at, initFilter() accepts an optional convolutional
98 99 100
filter as input that can be used for contrast, saturation, blur, sharpening
shift, chroma vs. luma shift, ...