graphicsmagick: memory allocation failure in ReadPCXImage (pcx.c)

Description:
Graphicsmagick is an Image Processing System.

After the first round of fuzzing where I discovered some slowness issues that make the fuzz hard, the second round revealed a memory allocation failure.

The complete ASan output:

# gm identify $FILE
==10139==ERROR: AddressSanitizer failed to allocate 0x4cd6a6000 (20626169856) bytes of LargeMmapAllocator (error code: 12)                                                                                                                                                     
==10139==Process memory map follows:                                                                                                                                                                                                                                           
        0x000000400000-0x00000051f000   /usr/bin/gm                                                                                                                                                                                                                            
        0x00000071e000-0x00000071f000   /usr/bin/gm                                                                                                                                                                                                                            
        0x00000071f000-0x000000722000   /usr/bin/gm                                                                                                                                                                                                                            
        0x000000722000-0x000001394000                                                                                                                                                                                                                                          
        0x00007fff7000-0x00008fff7000                                                                                                                                                                                                                                          
        0x00008fff7000-0x02008fff7000                                                                                                                                                                                                                                          
        0x02008fff7000-0x10007fff8000                                                                                                                                                                                                                                          
        0x600000000000-0x602000000000                                                                                                                                                                                                                                          
        0x602000000000-0x602000010000                                                                                                                                                                                                                                          
        0x602000010000-0x603000000000                                                                                                                                                                                                                                          
        0x603000000000-0x603000010000                                                                                                                                                                                                                                          
        0x603000010000-0x604000000000                                                                                                                                                                                                                                          
        0x604000000000-0x604000010000                                                                                                                                                                                                                                          
        0x604000010000-0x606000000000                                                                                                                                                                                                                                          
        0x606000000000-0x606000010000                                                                                                                                                                                                                                          
        0x606000010000-0x607000000000                                                                                                                                                                                                                                          
        0x607000000000-0x607000010000                                                                                                                                                                                                                                          
        0x607000010000-0x608000000000                                                                                                                                                                                                                                          
        0x608000000000-0x608000010000                                                                                                                                                                                                                                          
        0x608000010000-0x60a000000000                                                                                                                                                                                                                                          
        0x60a000000000-0x60a000010000                                                                                                                                                                                                                                          
        0x60a000010000-0x60b000000000                                                                                                                                                                                                                                          
        0x60b000000000-0x60b000010000                                                                                                                                                                                                                                          
        0x60b000010000-0x60c000000000                                                                                                                                                                                                                                          
        0x60c000000000-0x60c000010000                                                                                                                                                                                                                                          
        0x60c000010000-0x60f000000000                                                                                                                                                                                                                                          
        0x60f000000000-0x60f000010000                                                                                                                                                                                                                                          
        0x60f000010000-0x610000000000                                                                                                                                                                                                                                          
        0x610000000000-0x610000010000                                                                                                                                                                                                                                          
        0x610000010000-0x611000000000                                                                                                                                                                                                                                          
        0x611000000000-0x611000010000                                                                                                                                                                                                                                          
        0x611000010000-0x612000000000                                                                                                                                                                                                                                          
        0x612000000000-0x612000010000                                                                                                                                                                                                                                          
        0x612000010000-0x614000000000                                                                                                                                                                                                                                          
        0x614000000000-0x614000020000                                                                                                                                                                                                                                          
        0x614000020000-0x616000000000                                                                                                                                                                                                                                          
        0x616000000000-0x616000020000                                                                                                                                                                                                                                          
        0x616000020000-0x618000000000                                                                                                                                                                                                                                          
        0x618000000000-0x618000020000                                                                                                                                                                                                                                          
        0x618000020000-0x619000000000                                                                                                                                                                                                                                          
        0x619000000000-0x619000020000                                                                                                                                                                                                                                          
        0x619000020000-0x61a000000000
        0x61a000000000-0x61a000020000
        0x61a000020000-0x61b000000000
        0x61b000000000-0x61b000020000
        0x61b000020000-0x61d000000000
        0x61d000000000-0x61d000020000
        0x61d000020000-0x61e000000000
        0x61e000000000-0x61e000020000
        0x61e000020000-0x621000000000
        0x621000000000-0x621000020000
        0x621000020000-0x623000000000
        0x623000000000-0x623000020000
        0x623000020000-0x624000000000
        0x624000000000-0x624000020000
        0x624000020000-0x625000000000
        0x625000000000-0x625000020000
        0x625000020000-0x640000000000
        0x640000000000-0x640000003000
        0x7ff8e8877000-0x7ff8e888c000   /usr/lib64/GraphicsMagick-1.3.25/modules-Q32/coders/pcx.so
        0x7ff8e888c000-0x7ff8e8a8c000   /usr/lib64/GraphicsMagick-1.3.25/modules-Q32/coders/pcx.so
        0x7ff8e8a8c000-0x7ff8e8a8d000   /usr/lib64/GraphicsMagick-1.3.25/modules-Q32/coders/pcx.so
        0x7ff8e8a8d000-0x7ff8e8a8e000   /usr/lib64/GraphicsMagick-1.3.25/modules-Q32/coders/pcx.so
        0x7ff8e8a8e000-0x7ff8ef100000   /usr/lib64/locale/locale-archive
        0x7ff8ef100000-0x7ff8ef200000
        0x7ff8ef300000-0x7ff8ef400000
        0x7ff8ef4ab000-0x7ff8f17fd000
        0x7ff8f17fd000-0x7ff8f1806000   /usr/lib64/libltdl.so.7.3.1
        0x7ff8f1806000-0x7ff8f1a05000   /usr/lib64/libltdl.so.7.3.1
        0x7ff8f1a05000-0x7ff8f1a06000   /usr/lib64/libltdl.so.7.3.1
        0x7ff8f1a06000-0x7ff8f1a07000   /usr/lib64/libltdl.so.7.3.1
        0x7ff8f1a07000-0x7ff8f1a1c000   /lib64/libz.so.1.2.8
        0x7ff8f1a1c000-0x7ff8f1c1b000   /lib64/libz.so.1.2.8
        0x7ff8f1c1b000-0x7ff8f1c1c000   /lib64/libz.so.1.2.8
        0x7ff8f1c1c000-0x7ff8f1c1d000   /lib64/libz.so.1.2.8
        0x7ff8f1c1d000-0x7ff8f1c2c000   /lib64/libbz2.so.1.0.6
        0x7ff8f1c2c000-0x7ff8f1e2b000   /lib64/libbz2.so.1.0.6
        0x7ff8f1e2b000-0x7ff8f1e2c000   /lib64/libbz2.so.1.0.6
        0x7ff8f1e2c000-0x7ff8f1e2d000   /lib64/libbz2.so.1.0.6
        0x7ff8f1e2d000-0x7ff8f1ed4000   /usr/lib64/libfreetype.so.6.12.3
        0x7ff8f1ed4000-0x7ff8f20d4000   /usr/lib64/libfreetype.so.6.12.3
        0x7ff8f20d4000-0x7ff8f20da000   /usr/lib64/libfreetype.so.6.12.3
        0x7ff8f20da000-0x7ff8f20db000   /usr/lib64/libfreetype.so.6.12.3
        0x7ff8f20db000-0x7ff8f212f000   /usr/lib64/liblcms2.so.2.0.6
        0x7ff8f212f000-0x7ff8f232e000   /usr/lib64/liblcms2.so.2.0.6
        0x7ff8f232e000-0x7ff8f232f000   /usr/lib64/liblcms2.so.2.0.6
        0x7ff8f232f000-0x7ff8f2334000   /usr/lib64/liblcms2.so.2.0.6
        0x7ff8f2334000-0x7ff8f24c7000   /lib64/libc-2.22.so
        0x7ff8f24c7000-0x7ff8f26c7000   /lib64/libc-2.22.so
        0x7ff8f26c7000-0x7ff8f26cb000   /lib64/libc-2.22.so
        0x7ff8f26cb000-0x7ff8f26cd000   /lib64/libc-2.22.so
        0x7ff8f26cd000-0x7ff8f26d1000
        0x7ff8f26d1000-0x7ff8f26e7000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
        0x7ff8f26e7000-0x7ff8f28e6000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
        0x7ff8f28e6000-0x7ff8f28e7000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
        0x7ff8f28e7000-0x7ff8f28e8000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
        0x7ff8f28e8000-0x7ff8f28ee000   /lib64/librt-2.22.so
        0x7ff8f28ee000-0x7ff8f2aee000   /lib64/librt-2.22.so
        0x7ff8f2aee000-0x7ff8f2aef000   /lib64/librt-2.22.so
        0x7ff8f2aef000-0x7ff8f2af0000   /lib64/librt-2.22.so
        0x7ff8f2af0000-0x7ff8f2b07000   /lib64/libpthread-2.22.so
        0x7ff8f2b07000-0x7ff8f2d06000   /lib64/libpthread-2.22.so
        0x7ff8f2d06000-0x7ff8f2d07000   /lib64/libpthread-2.22.so
        0x7ff8f2d07000-0x7ff8f2d08000   /lib64/libpthread-2.22.so
        0x7ff8f2d08000-0x7ff8f2d0c000
        0x7ff8f2d0c000-0x7ff8f2e09000   /lib64/libm-2.22.so
        0x7ff8f2e09000-0x7ff8f3008000   /lib64/libm-2.22.so
        0x7ff8f3008000-0x7ff8f3009000   /lib64/libm-2.22.so
        0x7ff8f3009000-0x7ff8f300a000   /lib64/libm-2.22.so
        0x7ff8f300a000-0x7ff8f300c000   /lib64/libdl-2.22.so
        0x7ff8f300c000-0x7ff8f320c000   /lib64/libdl-2.22.so
        0x7ff8f320c000-0x7ff8f320d000   /lib64/libdl-2.22.so
        0x7ff8f320d000-0x7ff8f320e000   /lib64/libdl-2.22.so
        0x7ff8f320e000-0x7ff8f387c000   /usr/lib64/libGraphicsMagick.so.3.15.1
        0x7ff8f387c000-0x7ff8f3a7b000   /usr/lib64/libGraphicsMagick.so.3.15.1
        0x7ff8f3a7b000-0x7ff8f3aa3000   /usr/lib64/libGraphicsMagick.so.3.15.1
        0x7ff8f3aa3000-0x7ff8f3afd000   /usr/lib64/libGraphicsMagick.so.3.15.1
        0x7ff8f3afd000-0x7ff8f3b01000
        0x7ff8f3b01000-0x7ff8f3b23000   /lib64/ld-2.22.so
        0x7ff8f3c79000-0x7ff8f3c8e000
        0x7ff8f3c8e000-0x7ff8f3c95000   /usr/lib64/gconv/gconv-modules.cache
        0x7ff8f3c95000-0x7ff8f3cb8000   /usr/share/locale/it/LC_MESSAGES/libc.mo
        0x7ff8f3cb8000-0x7ff8f3d16000
        0x7ff8f3d16000-0x7ff8f3d22000
        0x7ff8f3d22000-0x7ff8f3d23000   /lib64/ld-2.22.so
        0x7ff8f3d23000-0x7ff8f3d24000   /lib64/ld-2.22.so
        0x7ff8f3d24000-0x7ff8f3d25000
        0x7fffd09c8000-0x7fffd09e9000   [stack]
        0x7fffd09f0000-0x7fffd09f2000   [vvar]
        0x7fffd09f2000-0x7fffd09f4000   [vdso]
        0xffffffffff600000-0xffffffffff601000   [vsyscall]
==10139==End of process memory map.
==10139==AddressSanitizer CHECK failed: /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
    #0 0x4c973d in AsanCheckFailed /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_rtl.cc:67
    #1 0x4d0273 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:159
    #2 0x4d0461 in __sanitizer::ReportMmapFailureAndDie(unsigned long, char const*, char const*, int, bool) /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183
    #3 0x4d949a in __sanitizer::MmapOrDie(unsigned long, char const*, bool) /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_posix.cc:122
    #4 0x42182f in __sanitizer::LargeMmapAllocator::Allocate(__sanitizer::AllocatorStats*, unsigned long, unsigned long) /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1033
    #5 0x42182f in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback>, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >, __sanitizer::LargeMmapAllocator >::Allocate(__sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >*, unsigned long, unsigned long, bool, bool) /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1302
    #6 0x42182f in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:368
    #7 0x42182f in __asan::asan_malloc(unsigned long, __sanitizer::BufferedStackTrace*) /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:718
    #8 0x4bfe01 in malloc /var/tmp/portage/sys-devel/llvm-3.8.1/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:53
    #9 0x7ff8e887beba in ReadPCXImage /var/tmp/portage/media-gfx/graphicsmagick-1.3.25/work/GraphicsMagick-1.3.25/coders/pcx.c:467:16
    #10 0x7ff8f34a4c4e in ReadImage /var/tmp/portage/media-gfx/graphicsmagick-1.3.25/work/GraphicsMagick-1.3.25/magick/constitute.c:1607:13
    #11 0x7ff8f34a4294 in PingImage /var/tmp/portage/media-gfx/graphicsmagick-1.3.25/work/GraphicsMagick-1.3.25/magick/constitute.c:1370:9
    #12 0x7ff8f33f5836 in IdentifyImageCommand /var/tmp/portage/media-gfx/graphicsmagick-1.3.25/work/GraphicsMagick-1.3.25/magick/command.c:8375:17
    #13 0x7ff8f33f9e23 in MagickCommand /var/tmp/portage/media-gfx/graphicsmagick-1.3.25/work/GraphicsMagick-1.3.25/magick/command.c:8865:17
    #14 0x7ff8f344fc3e in GMCommandSingle /var/tmp/portage/media-gfx/graphicsmagick-1.3.25/work/GraphicsMagick-1.3.25/magick/command.c:17379:10
    #15 0x7ff8f344e5d1 in GMCommand /var/tmp/portage/media-gfx/graphicsmagick-1.3.25/work/GraphicsMagick-1.3.25/magick/command.c:17432:16
    #16 0x7ff8f235461f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
    #17 0x4188d8 in _init (/usr/bin/gm+0x4188d8)

Affected version:
1.3.25

Fixed version:
1.3.26 (not yet released)

Commit fix:
http://hg.code.sf.net/p/graphicsmagick/code/rev/b9edafd479b9

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2016-8683

Timeline:
2016-09-09: bug discovered
2016-09-09: bug reported privately to upstream
2016-09-10: no upstream response
2016-09-15: blog post about the issue
2016-10-16: CVE Assigned

Note:
This bug was found with American Fuzzy Lop.

Permalink:

graphicsmagick: memory allocation failure in ReadPCXImage (pcx.c)

This entry was posted in advisories, security. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.