MrBesen/ffmpeg
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

swscale: fix overflow in gray16 vertical scaling.

Browse Source 
This fixes the same overflow as in the RGB48/16-bit YUV scaling;
some filters can overflow both negatively and positively (e.g.
spline/lanczos), so we bias a signed integer so it's "half signed"
and "half unsigned", and can cover overflows in both directions
while maintaining full 31-bit depth.

Signed-off-by: Mans Rullgard <mans@mansr.com>
This commit is contained in:
Ronald S. Bultje 2011-12-17 19:56:40 +00:00 committed by Mans Rullgard
parent 86a10281c7
commit 72dafea0fc
1 changed files with 6 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
libswscale/swscale.c
View File

@ -386,8 +386,8 @@ yuv2gray16_X_c_template(SwsContext *c, const int16_t *lumFilter,
for (i = 0; i < (dstW >> 1); i++) {
int j;
int Y1 = 1 << 14;
int Y2 = 1 << 14;
int Y1 = (1 << 14) - 0x40000000;
int Y2 = (1 << 14) - 0x40000000;
for (j = 0; j < lumFilterSize; j++) {
Y1 += lumSrc[j][i * 2] * lumFilter[j];
@ -395,12 +395,10 @@ yuv2gray16_X_c_template(SwsContext *c, const int16_t *lumFilter,
}
Y1 >>= 15;
Y2 >>= 15;
if ((Y1 | Y2) & 0x10000) {
Y1 = av_clip_uint16(Y1);
Y2 = av_clip_uint16(Y2);
}
output_pixel(&dest[i * 2 + 0], Y1);
output_pixel(&dest[i * 2 + 1], Y2);
Y1 = av_clip_int16(Y1);
Y2 = av_clip_int16(Y2);
output_pixel(&dest[i * 2 + 0], 0x8000 + Y1);
output_pixel(&dest[i * 2 + 1], 0x8000 + Y2);
}
}