libav: invalid memory access in ff_put_pixels8_mmx (fpel_mmx.c)

Description:
Libav is an open source set of tools for audio and video processing.

A fuzzing with an mp3 file as input discovered an invalid memory access in ff_put_pixels8_mmx.

The complete ASan output:

# avconv -i $FILE -f null -
avconv version 11.7, Copyright (c) 2000-2016 the Libav developers
  built on Aug 16 2016 15:34:42 with clang version 3.8.1 (tags/RELEASE_381/final)
[h263 @ 0x61a00001f280] Format detected only with low score of 25, misdetection possible!
[h263 @ 0x619000000580] warning: first frame is no keyframe
[h263 @ 0x619000000580] cbpc damaged at 2 0
[h263 @ 0x619000000580] Error at MB: 2
[h263 @ 0x619000000580] illegal ac vlc code at 3x40
[h263 @ 0x619000000580] Error at MB: 3563
[h263 @ 0x619000000580] concealing 6336 DC, 6336 AC, 6336 MV errors
[h263 @ 0x61a00001f280] Estimating duration from bitrate, this may be inaccurate
Input #0, h263, from '82.crashes':
  Duration: N/A, bitrate: N/A
    Stream #0.0: Video: h263, yuv420p, 1408x1152 [PAR 12:11 DAR 4:3], 25 fps, 25 tbn, 29.97 tbc
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf56.1.0
    Stream #0.0: Video: rawvideo, yuv420p, 1408x1152 [PAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.1.0 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (h263 (native) -> rawvideo (native))
Press ctrl-c to stop encoding
[h263 @ 0x61900001e580] warning: first frame is no keyframe
[h263 @ 0x61900001e580] cbpc damaged at 2 0
[h263 @ 0x61900001e580] Error at MB: 2
[h263 @ 0x61900001e580] illegal ac vlc code at 3x40
[h263 @ 0x61900001e580] Error at MB: 3563
[h263 @ 0x61900001e580] concealing 6336 DC, 6336 AC, 6336 MV errors
[h263 @ 0x61900001e580] Error at MB: 0
[h263 @ 0x61900001e580] concealing 6336 DC, 6336 AC, 6336 MV errors
[h263 @ 0x61900001e580] cbpc damaged at 0 0
[h263 @ 0x61900001e580] Error at MB: 0
[h263 @ 0x61900001e580] concealing 6336 DC, 6336 AC, 6336 MV errors
[h263 @ 0x61900001e580] illegal ac vlc code at 0x0
[h263 @ 0x61900001e580] Error at MB: 0
[h263 @ 0x61900001e580] concealing 99 DC, 99 AC, 99 MV errors
Input stream #0:0 frame changed from size:1408x1152 fmt:yuv420p to size:176x144 fmt:yuv420p
[h263 @ 0x61900001e580] Bad marker
[h263 @ 0x61900001e580] header damaged
Error while decoding stream #0:0
ASAN:DEADLYSIGNAL
=================================================================
==29430==ERROR: AddressSanitizer: SEGV on unknown address 0x62bfffffffd0 (pc 0x7f3a2a642150 bp 0x7ffcbfdb8af0 sp 0x7ffcbfdb8a80 T0)
    #0 0x7f3a2a64214f in ff_put_pixels8_mmx /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/x86/fpel_mmx.c:81:5
    #1 0x7f3a297c0de0 in hpel_motion /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/mpegvideo_motion.c:224:5
    #2 0x7f3a297c0de0 in apply_8x8 /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/mpegvideo_motion.c:798
    #3 0x7f3a297c0de0 in mpv_motion_internal /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/mpegvideo_motion.c:877
    #4 0x7f3a297c0de0 in ff_mpv_motion /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/mpegvideo_motion.c:981
    #5 0x7f3a296ed59b in mpv_decode_mb_internal /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/mpegvideo.c:2223:21
    #6 0x7f3a296ed59b in ff_mpv_decode_mb /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/mpegvideo.c:2358
    #7 0x7f3a285ffc95 in decode_slice /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/h263dec.c:273:13
    #8 0x7f3a285fb2cd in ff_h263_decode_frame /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/h263dec.c:575:11
    #9 0x7f3a29f86906 in avcodec_decode_video2 /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/utils.c:1600:19
    #10 0x5647eb in decode_video /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/avconv.c:1259:11
    #11 0x5647eb in process_input_packet /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/avconv.c:1398
    #12 0x550e63 in process_input /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/avconv.c:2440:11
    #13 0x550e63 in transcode /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/avconv.c:2488
    #14 0x550e63 in main /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/avconv.c:2647
    #15 0x7f3a2631b61f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
    #16 0x41d098 in _init (/usr/bin/avconv+0x41d098)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /var/tmp/portage/media-video/libav-11.7/work/libav-11.7/libavcodec/x86/fpel_mmx.c:81:5 in ff_put_pixels8_mmx
==29430==ABORTING

Affected version:
11.7

Fixed version:
N/A

Commit fix:
N/A

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
N/A

Timeline:
2016-08-15: bug discovered
2016-08-16: bug reported to upstream
2016-10-04: blog post about the issue

Note:
This bug was found with American Fuzzy Lop.
Since at 20161008 there wasn’t an upstream response, after an investigation it looks like an invalid memory access instead of a NULL pointer, so I’m changing a bit the description, but I’m keeping the permalink as-is.

Permalink:

libav: invalid memory access in ff_put_pixels8_mmx (fpel_mmx.c)

This entry was posted in advisories, security. Bookmark the permalink.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.