Fixes use of uninitialized memory
Fixes: a96874b9466b6edc660a519c7ad47977_signal_sigsegv_7ffff713351a_744_nc_sample.avi with memlimit 2147483648
Found-by: Samuel Groß, Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes use of uninitialized memory
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_heap-oob_35ca682_1474_cov_3230122439_aletrek_tga_16bit.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'f84a1b597c29dc035b8d5529ef88c2d7ff057820':
swscale: support AV_PIX_FMT_YA16 as input
Conflicts:
libswscale/swscale_unscaled.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
The rationale is that you have a packed format in form
<greyscale sample> <alpha sample> <greyscale sample> <alpha sample>
and shortening greyscale to 'G' might make one thing about Greenscale instead.
An alias pixel format and color space name are provided for compatibility.
Prevent a division by zero down the codepath.
Sample-Id: 00001721-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
This way all context fields get initialized and can be used without needing to reinit the context
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '92b099daf4b8ef93513e38b43899cb8458a2fde3':
swscale: support converting YVYU422 pixel format
Conflicts:
libswscale/input.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Both of these dithering methods are from http://pippin.gimp.org/a_dither/ for
GIF they can be considered better than bayer (provides more gray-levels), and
spatial stability - often more than twice as good compression and less visual
flicker than error diffusion methods (the methods also avoids error-shadow
artifacts of diffusion dithers).
These methods are similar to blue/green noise type dither masks; but are
simple enough to generate their mask on the fly. They are still research work
in progress; though more expensive to generate masks (which can be used in a
LUT) like 'void and cluster' and similar methods will yield superior results
Improves rgb -> gray16 conversion
Fixes Ticket3422
The pam and png output files look visually similar, in both cases the
dynamics increase to 0x0 -> 0xfffb.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '6b3ff6f91a535d6383f41ca7bdf760165dcb6015':
swscale: provide a default scaler if none is set
Conflicts:
libswscale/utils.c
The default is left at bicubic until someone has compared the scalers
properly speed and quality wise.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Lanczos for general case, sinc for upscaling, Gaussian for
downscaling. According to current literature these scalers
should be the best quality-wise algorithms for each case.
Inspired from a patch by wm4 <nfxjfg@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Previously this would have lead to a memcpy(a,a) which violates the
requirement of non overlapping src and dst.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '79aec43ce813a3e270743ca64fa3f31fa43df80b':
x86: Add and use more convenience macros to check CPU extension availability
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '1909f6b1b6da2db371c179bed5a19aaf979b7557':
swscale: cosmetics: Drop silly camelCase from swScale function pointer name
Conflicts:
libswscale/swscale_unscaled.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'f61bece684d9685b07895508e6c1c733b5564ccf':
ppc: Add and use convenience macro to check for AltiVec availability
Conflicts:
libavcodec/ppc/dsputil_ppc.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This allows specifying more dither algorithms without using up flags and
without ambiguities.
Also initialize the new field based on the flags and use it.
Note, improving the logic of the checks is left to subsequent
commits, this here only switches from flags to enum.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes scaling in MSVC/ICL, where inline asm is not present, but the
YASM scaling code is.
Reviewed-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'c1eb3e7fecdc270e03a700d61ef941600a6af491':
swscale: add support for endianness only conversion
Conflicts:
libswscale/utils.c
libswscale/version.h
Merged-by: Michael Niedermayer <michaelni@gmx.at>
They are currently always constant and thus theres no point to
store them in the context.
This makes the context 16kb smaller
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
For the default table we keep using the exact previous
coefficients, this way individual functions can be updated seperately
without breaking fate
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This reverts parts of d6d5ef5534, that didn't work right. (The
tests that were added failed on big endian, and the output looked
garbled on little endian as well.)
This is due to the fact that the intermediate scaling values (from
e.g. hScale8To19_c or hScale16To19_c) are stored as int32_t and
thus requires a separate output function, while yuv2gbrp_full_X_c
only interprets it as int16_t.
Signed-off-by: Martin Storsjö <martin@martin.st>
This code path is not implemented and makes not much sense to implement
either.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* commit '05b0998f511ffa699407465d48c7d5805f746ad2':
dsputil: Fix error by not using redzone and register name
swscale: GBRP output support
Conflicts:
libswscale/output.c
libswscale/swscale.c
libswscale/swscale_internal.h
libswscale/utils.c
tests/ref/lavfi/pixdesc
tests/ref/lavfi/pixfmts_copy
tests/ref/lavfi/pixfmts_null
tests/ref/lavfi/pixfmts_scale
tests/ref/lavfi/pixfmts_vflip
Merged-by: Michael Niedermayer <michaelni@gmx.at>
sws_getCachedContext() and sws_getContext() expect sws_alloc_context()
to return NULL when out of memory, as follows.
if (!(context = sws_alloc_context()))
return NULL;
This patch fixes sws_alloc_context() to return NULL in that case.
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
* commit 'fa8fcab1e0d31074c0644c4ac5194474c6c26415':
x86: h264_chromamc_10bit: drop pointless PAVG %define
x86: mmx2 ---> mmxext in function names
swscale: do not forget to swap data in formats with different endianness
Conflicts:
libavcodec/x86/dsputil_mmx.c
libavfilter/x86/gradfun.c
libswscale/input.c
libswscale/utils.c
libswscale/x86/swscale.c
tests/ref/lavfi/pixfmts_scale
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
pixfmt: support more yuva formats
swscale: support gray to 9bit and 10bit formats
configure: rewrite print_config() function using awk
FATE: fix (AD)PCM test dependencies broken in e519990
Use ptrdiff_t instead of int for intra pred "stride" function parameter.
x86: use PRED4x4/8x8/8x8L/16x16 macros to declare intrapred prototypes.
Conflicts:
libavcodec/h264pred.c
libavcodec/h264pred_template.c
libavutil/pixfmt.h
libswscale/swscale_unscaled.c
tests/ref/lavfi/pixdesc
tests/ref/lavfi/pixfmts_copy
tests/ref/lavfi/pixfmts_null
tests/ref/lavfi/pixfmts_scale
tests/ref/lavfi/pixfmts_vflip
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
swscale: try to use mmap only if available
configure: check for mprotect
wmapro: use planar sample format
wmalossless: output in planar sample format
wmadec: use float planar sample format output
shorten: use planar sample format
lavc: update documentation for AVFrame.extended_data
Conflicts:
libavcodec/shorten.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Some systems, e.g. Minix, have sys/mman.h defining MAP_ANONYMOUS without
providing (working) mmap and friends. The mmx filter generation code
checks only for MAP_ANONYMOUS, not for availability of mmap itself which
leads to build errors on aforementioned systems.
This changes the conditional compilation to use mmap only if all the
required functions are available.
Signed-off-by: Mans Rullgard <mans@mansr.com>
* qatar/master:
swscale: Provide the right alignment for external mmx asm
x86: Replace checks for CPU extensions and flags by convenience macros
configure: msvc: fix/simplify setting of flags for hostcc
x86: mlpdsp: mlp_filter_channel_x86 requires inline asm
Conflicts:
libavcodec/x86/fft_init.c
libavcodec/x86/h264_intrapred_init.c
libavcodec/x86/h264dsp_init.c
libavcodec/x86/mpegaudiodec.c
libavcodec/x86/proresdsp_init.c
libavutil/x86/float_dsp_init.c
libswscale/utils.c
libswscale/x86/swscale.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This reverts parts of e0c6cce447. There is external mmx asm that
requires this alignment.
This fixes crashes when using swscale in builds with external mmx,
without inline assembly.
Signed-off-by: Martin Storsjö <martin@martin.st>
* qatar/master:
MSS1 and MSS2: set final pixel format after common stuff has been initialised
MSS2 decoder
configure: handle --disable-asm before check_deps
x86: Split inline and external assembly #ifdefs
configure: x86: Separate inline from standalone assembler capabilities
pktdumper: Use a custom define instead of PATH_MAX for buffers
pktdumper: Use av_strlcpy instead of strncpy
pktdumper: Use sizeof(variable) instead of the direct buffer length
Conflicts:
Changelog
configure
libavcodec/allcodecs.c
libavcodec/avcodec.h
libavcodec/codec_desc.c
libavcodec/dct-test.c
libavcodec/imgconvert.c
libavcodec/mss12.c
libavcodec/version.h
libavfilter/x86/gradfun.c
libswscale/x86/yuv2rgb.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
mpegvideo: reduce excessive inlining of mpeg_motion()
mpegvideo: convert mpegvideo_common.h to a .c file
build: factor out mpegvideo.o dependencies to CONFIG_MPEGVIDEO
Move MASK_ABS macro to libavcodec/mathops.h
x86: move MANGLE() and related macros to libavutil/x86/asm.h
x86: rename libavutil/x86_cpu.h to libavutil/x86/asm.h
aacdec: Don't fall back to the old output configuration when no old configuration is present.
rtmp: Add message tracking
rtsp: Support mpegts in raw udp packets
rtsp: Support receiving plain data over UDP without any RTP encapsulation
rtpdec: Remove an unused include
rtpenc: Remove an av_abort() that depends on user-supplied data
vsrc_movie: discourage its use with avconv.
avconv: allow no input files.
avconv: prevent invalid reads in transcode_init()
avconv: rename OutputStream.is_past_recording_time to finished.
Conflicts:
configure
doc/filters.texi
ffmpeg.c
ffmpeg.h
libavcodec/Makefile
libavcodec/aacdec.c
libavcodec/mpegvideo.c
libavformat/version.h
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
lavr: fix handling of custom mix matrices
fate: force pix_fmt in lagarith-rgb32 test
fate: add tests for lagarith lossless video codec.
ARMv6: vp8: fix stack allocation with Apple's assembler
ARM: vp56: allow inline asm to build with clang
fft: 3dnow: fix register name typo in DECL_IMDCT macro
x86: dct32: port to cpuflags
x86: build: replace mmx2 by mmxext
Revert "wmapro: prevent division by zero when sample rate is unspecified"
wmapro: prevent division by zero when sample rate is unspecified
lagarith: fix color plane inversion for YUY2 output.
lagarith: pad RGB buffer by 1 byte.
dsputil: make add_hfyu_left_prediction_sse4() support unaligned src.
Conflicts:
doc/APIchanges
libavcodec/lagarith.c
libavfilter/x86/gradfun.c
libavutil/cpu.h
libavutil/version.h
libswscale/utils.c
libswscale/version.h
libswscale/x86/yuv2rgb.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Refactoring mmx2/mmxext YASM code with cpuflags will force renames.
So switching to a consistent naming scheme beforehand is sensible.
The name "mmxext" is more official and widespread and also the name
of the CPU flag, as reported e.g. by the Linux kernel.
* qatar/master:
v410dec: Implement explode mode support
zerocodec: fix direct rendering.
wav: init st to NULL to avoid a false-positive warning.
wavpack: set bits_per_raw_sample for S32 samples to properly identify 24-bit
h264: refactor NAL decode loop
RTMPTE protocol support
RTMPE protocol support
rtmp: Add ff_rtmp_calc_digest_pos()
rtmp: Rename rtmp_calc_digest to ff_rtmp_calc_digest and make it global
swscale: add missing HAVE_INLINE_ASM check.
lavfi: place x86 inline assembly under HAVE_INLINE_ASM.
vc1: Add a test for interlaced field pictures
swscale: Mark all init functions as av_cold
swscale: x86: Drop pointless _mmx suffix from filenames
lavf: use conditional notation for default codec in muxer declarations.
swscale: place inline assembly bilinear scaler under HAVE_INLINE_ASM.
dsputil: ppc: cosmetics: pretty-print
dsputil: x86: add SHUFFLE_MASK_W macro
configure: respect CC_O setting in check_cc
Conflicts:
Changelog
configure
libavcodec/v410dec.c
libavcodec/zerocodec.c
libavformat/asfenc.c
libavformat/version.h
libswscale/utils.c
libswscale/x86/swscale.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
swscale: K&R formatting cosmetics (part II)
tiffdec: Add a malloc check and refactor another.
faxcompr: Check malloc results and unify return path
configure: escape colons in values written to config.fate
ac3dsp: call femms/emms at the end of float_to_fixed24() for 3DNow and SSE
matroska: Fix leaking memory allocated for laces.
pthread: Fix crash due to fctx->delaying not being cleared.
vp3: Assert on invalid filter_limit values.
h264: fix 10bit biweight functions after recent x86inc.asm fixes.
ffv1: Fix size mismatch in encode_line.
movenc: Remove a dead initialization
git-howto: Explain how to avoid Windows line endings in git checkouts.
build: Move all arch OBJS declarations into arch subdirectory Makefiles.
Conflicts:
configure
libavcodec/vp3.c
libavformat/matroskadec.c
libavutil/Makefile
libswscale/Makefile
libswscale/swscale.c
libswscale/swscale_internal.h
libswscale/utils.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (58 commits)
amrnbdec: check frame size before decoding.
cscd: use negative error values to indicate decode_init() failures.
h264: prevent overreads in intra PCM decoding.
FATE: do not decode audio in the nuv test.
dxa: set audio stream time base using the sample rate
psx-str: do not allow seeking by bytes
asfdec: Do not set AVCodecContext.frame_size
vqf: set packet parameters after av_new_packet()
mpegaudiodec: use DSPUtil.butterflies_float().
FATE: add mp3 test for sample that exhibited false overreads
fate: add cdxl test for bit line plane arrangement
vmnc: return error on decode_init() failure.
libvorbis: add/update error messages
libvorbis: use AVFifoBuffer for output packet buffer
libvorbis: remove unneeded e_o_s check
libvorbis: check return values for functions that can return errors
libvorbis: use float input instead of s16
libvorbis: do not flush libvorbis analysis if dsp state was not initialized
libvorbis: use VBR by default, with default quality of 3
libvorbis: fix use of minrate/maxrate AVOptions
...
Conflicts:
Changelog
doc/APIchanges
libavcodec/avcodec.h
libavcodec/dpxenc.c
libavcodec/libvorbis.c
libavcodec/vmnc.c
libavformat/asfdec.c
libavformat/id3v2enc.c
libavformat/internal.h
libavformat/mp3enc.c
libavformat/utils.c
libavformat/version.h
libswscale/utils.c
tests/fate/video.mak
tests/ref/fate/nuv
tests/ref/fate/prores-alpha
tests/ref/lavf/ffm
tests/ref/vsynth1/prores
tests/ref/vsynth2/prores
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (40 commits)
swf: check return values for av_get/new_packet().
wavpack: Don't shift minclip/maxclip
rtpenc: Expose the max packet size via an avoption
rtpenc: Move max_packet_size to a context variable
rtpenc: Add an option for not sending RTCP packets
lavc: drop encode() support for video.
snowenc: switch to encode2().
snowenc: don't abuse input picture for storing information.
a64multienc: switch to encode2().
a64multienc: don't write into output buffer when there's no output.
libxvid: switch to encode2().
tiffenc: switch to encode2().
tiffenc: properly forward error codes in encode_frame().
lavc: drop libdirac encoder.
gifenc: switch to encode2().
libvpxenc: switch to encode2().
flashsvenc: switch to encode2().
Remove libpostproc.
lcl: don't overwrite input memory.
swscale: take first/lastline over/underflows into account for MMX.
...
Conflicts:
.gitignore
Makefile
cmdutils.c
configure
doc/APIchanges
libavcodec/Makefile
libavcodec/allcodecs.c
libavcodec/libdiracenc.c
libavcodec/libxvidff.c
libavcodec/qtrleenc.c
libavcodec/tiffenc.c
libavcodec/utils.c
libavformat/mov.c
libavformat/movenc.c
libpostproc/Makefile
libpostproc/postprocess.c
libpostproc/postprocess.h
libpostproc/postprocess_altivec_template.c
libpostproc/postprocess_internal.h
libpostproc/postprocess_template.c
libswscale/swscale.c
libswscale/utils.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
FATE: update reference for seek-alac_mp4
sunrast: Return AVERROR values instead of -1.
sunrast: Add support for gray8 decoding.
swscale: enforce a minimum filtersize.
alacenc: use AVCodec.encode2()
alacenc: cosmetics: indentation
alacenc: consolidate bitstream writing into a single function.
alacenc: only encode frame size in header for a final smaller frame
alacenc: store current frame size in AlacEncodeContext.
alacenc: return AVERROR codes in alac_encode_frame()
alacenc: calculate a new max frame size for the final small frame
alacenc: pretty-printing and other cosmetics
alacenc: fix error handling and potential memleaks in alac_encode_init()
alacenc: do not set coded_frame->key_frame
alacenc: do not set bits_per_coded_sample
alacenc: remove unneeded frame_size check in alac_encode_frame()
tta: error out if samplerate is zero.
ttadec: fix invalid free when an error occurs while decoding 24-bit tta
wavpack: add needed braces for 2 statements inside an if block
Conflicts:
tests/ref/acodec/alac
Merged-by: Michael Niedermayer <michaelni@gmx.at>
At very small dimensions, this calculation could lead to zero-sized
filters, which leads to uninitialized output, zero-sized allocations,
loop overflows in SIMD that uses do{..}while(i++<filtersize); instead
of for(i=0;i<filtersize;i++){..} and several other similar failures.
Therefore, require a minimum filtersize of 1.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
* qatar/master:
aacenc: Fix LONG_START windowing.
aacenc: Fix a bug where deinterleaved samples were stored in the wrong place.
avplay: use the correct array size for stride.
lavc: extend doxy for avcodec_alloc_context3().
APIchanges: mention avcodec_alloc_context()/2/3
avcodec_align_dimensions2: set only 4 linesizes, not AV_NUM_DATA_POINTERS.
aacsbr: ARM NEON optimised sbrdsp functions
aacsbr: align some arrays
aacsbr: move some simdable loops to function pointers
cosmetics: Remove extra newlines at EOF
Conflicts:
libavcodec/utils.c
libavfilter/formats.c
libavutil/mem.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
