• Ganesh Ajjanagadde's avatar
    avutil/mathematics: correct documentation for av_gcd · 20a30077
    Ganesh Ajjanagadde authored
    av_gcd is now always defined regardless of input. This documents this
    change in the "documented API". Two benefits (closely related):
    1. The function is robust, and there is no need to worry about INT64_MIN, etc.
    2. Clients of av_gcd, like av_reduce, can now be made fully correct. Currently,
    av_reduce can trigger undefined behavior if e.g num is INT64_MIN due to
    integer overflow in the FFABS. Furthermore, this undefined behavior is
    completely undocumented, and could be a fuzzer's paradise. The FFABS was needed in the past as
    av_gcd was undefined for negative inputs. In order to make av_reduce
    robust, it is essential to guarantee that av_gcd works for all int64_t.
    Reviewed-by: 's avatarMichael Niedermayer <michael@niedermayer.cc>
    Signed-off-by: 's avatarGanesh Ajjanagadde <gajjanagadde@gmail.com>
mathematics.h 5.1 KB