avcodec/apedec: Use FFABSU() in do_apply_filter()

Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 29053/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-4814432697974784

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer 2021-01-21 00:07:46 +01:00
parent 5dd9567080
commit 59a6becf8e

View File

@ -1333,7 +1333,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
/* Version 3.98 and later files */
/* Update the adaption coefficients */
absres = FFABS(res);
absres = FFABSU(res);
if (absres)
*f->adaptcoeffs = APESIGN(res) *
(8 << ((absres > f->avg * 3) + (absres > f->avg * 4 / 3)));