graphicsmagick: memory allocation failure in MagickRealloc (memory.c)

Description:
Graphicsmagick is an Image Processing System.

This is an old memory failure, discovered time ago. The maintainer, Mr. Bob Friesenhahn was able to reproduce the issue; I’m quoting his feedback about:

The problem is that the embedded JPEG data claims to have dimensions 59395×56833 and
this is only learned after we are in the JPEG reader.

But for some reasons (maybe not easy to fix) it is still not fixed.
EDIT: the patch was added but I was not aware of that.

The complete ASan output:

# gm identify $FILE
==12404==ERROR: AddressSanitizer failed to allocate 0xfb8065000 (67511930880) bytes of LargeMmapAllocator (error code: 12)
==12404==Process memory map follows:
	0x000000400000-0x000000522000	/usr/bin/gm
	0x000000722000-0x000000723000	/usr/bin/gm
	0x000000723000-0x000000726000	/usr/bin/gm
	0x000000726000-0x0000013a9000	
	0x00007fff7000-0x00008fff7000	
	0x00008fff7000-0x02008fff7000	
	0x02008fff7000-0x10007fff8000	
	0x600000000000-0x602000000000	
	0x602000000000-0x602000010000	
	0x602000010000-0x603000000000	
	0x603000000000-0x603000010000	
	0x603000010000-0x604000000000	
	0x604000000000-0x604000010000	
	0x604000010000-0x606000000000	
	0x606000000000-0x606000010000	
	0x606000010000-0x607000000000	
	0x607000000000-0x607000010000	
	0x607000010000-0x608000000000	
	0x608000000000-0x608000010000	
	0x608000010000-0x60a000000000	
	0x60a000000000-0x60a000010000	
	0x60a000010000-0x60b000000000	
	0x60b000000000-0x60b000010000	
	0x60b000010000-0x60c000000000	
	0x60c000000000-0x60c000010000	
	0x60c000010000-0x60d000000000	
	0x60d000000000-0x60d000010000	
	0x60d000010000-0x60e000000000	
	0x60e000000000-0x60e000010000	
	0x60e000010000-0x60f000000000	
	0x60f000000000-0x60f000010000	
	0x60f000010000-0x610000000000	
	0x610000000000-0x610000010000	
	0x610000010000-0x611000000000	
	0x611000000000-0x611000010000	
	0x611000010000-0x612000000000	
	0x612000000000-0x612000010000	
	0x612000010000-0x614000000000	
	0x614000000000-0x614000020000	
	0x614000020000-0x616000000000	
	0x616000000000-0x616000020000	
	0x616000020000-0x618000000000	
	0x618000000000-0x618000020000	
	0x618000020000-0x619000000000	
	0x619000000000-0x619000020000	
	0x619000020000-0x61a000000000	
	0x61a000000000-0x61a000020000	
	0x61a000020000-0x61b000000000	
	0x61b000000000-0x61b000020000	
	0x61b000020000-0x61c000000000	
	0x61c000000000-0x61c000020000	
	0x61c000020000-0x61d000000000	
	0x61d000000000-0x61d000020000	
	0x61d000020000-0x61e000000000	
	0x61e000000000-0x61e000020000	
	0x61e000020000-0x621000000000	
	0x621000000000-0x621000020000	
	0x621000020000-0x623000000000	
	0x623000000000-0x623000020000	
	0x623000020000-0x624000000000	
	0x624000000000-0x624000020000	
	0x624000020000-0x625000000000	
	0x625000000000-0x625000030000	
	0x625000030000-0x628000000000	
	0x628000000000-0x628000010000	
	0x628000010000-0x62a000000000	
	0x62a000000000-0x62a000010000	
	0x62a000010000-0x630000000000	
	0x630000000000-0x630000020000	
	0x630000020000-0x640000000000	
	0x640000000000-0x640000003000	
	0x7fcc55fbe000-0x7fcc56027000	/usr/lib64/libjpeg.so.62.2.0
	0x7fcc56027000-0x7fcc56226000	/usr/lib64/libjpeg.so.62.2.0
	0x7fcc56226000-0x7fcc56227000	/usr/lib64/libjpeg.so.62.2.0
	0x7fcc56227000-0x7fcc56228000	/usr/lib64/libjpeg.so.62.2.0
	0x7fcc56228000-0x7fcc56254000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/jpeg.so
	0x7fcc56254000-0x7fcc56453000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/jpeg.so
	0x7fcc56453000-0x7fcc56454000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/jpeg.so
	0x7fcc56454000-0x7fcc56457000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/jpeg.so
	0x7fcc56457000-0x7fcc5645b000	
	0x7fcc5645b000-0x7fcc5648c000	/usr/lib64/libpng16.so.16.21.0
	0x7fcc5648c000-0x7fcc5668b000	/usr/lib64/libpng16.so.16.21.0
	0x7fcc5668b000-0x7fcc5668c000	/usr/lib64/libpng16.so.16.21.0
	0x7fcc5668c000-0x7fcc5668d000	/usr/lib64/libpng16.so.16.21.0
	0x7fcc5668d000-0x7fcc5671d000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/png.so
	0x7fcc5671d000-0x7fcc5691d000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/png.so
	0x7fcc5691d000-0x7fcc5691f000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/png.so
	0x7fcc5691f000-0x7fcc56927000	/usr/lib64/GraphicsMagick-1.3.24/modules-Q32/coders/png.so
	0x7fcc56927000-0x7fcc56932000	
	0x7fcc56932000-0x7fcc5cfa4000	/usr/lib64/locale/locale-archive
	0x7fcc5cfa4000-0x7fcc5fdff000	
	0x7fcc5fdff000-0x7fcc5fe08000	/usr/lib64/libltdl.so.7.3.1
	0x7fcc5fe08000-0x7fcc60007000	/usr/lib64/libltdl.so.7.3.1
	0x7fcc60007000-0x7fcc60008000	/usr/lib64/libltdl.so.7.3.1
	0x7fcc60008000-0x7fcc60009000	/usr/lib64/libltdl.so.7.3.1
	0x7fcc60009000-0x7fcc6001e000	/lib64/libz.so.1.2.8
	0x7fcc6001e000-0x7fcc6021d000	/lib64/libz.so.1.2.8
	0x7fcc6021d000-0x7fcc6021e000	/lib64/libz.so.1.2.8
	0x7fcc6021e000-0x7fcc6021f000	/lib64/libz.so.1.2.8
	0x7fcc6021f000-0x7fcc6022e000	/lib64/libbz2.so.1.0.6
	0x7fcc6022e000-0x7fcc6042d000	/lib64/libbz2.so.1.0.6
	0x7fcc6042d000-0x7fcc6042e000	/lib64/libbz2.so.1.0.6
	0x7fcc6042e000-0x7fcc6042f000	/lib64/libbz2.so.1.0.6
	0x7fcc6042f000-0x7fcc604d6000	/usr/lib64/libfreetype.so.6.12.3
	0x7fcc604d6000-0x7fcc606d6000	/usr/lib64/libfreetype.so.6.12.3
	0x7fcc606d6000-0x7fcc606dc000	/usr/lib64/libfreetype.so.6.12.3
	0x7fcc606dc000-0x7fcc606dd000	/usr/lib64/libfreetype.so.6.12.3
	0x7fcc606dd000-0x7fcc60730000	/usr/lib64/liblcms2.so.2.0.6
	0x7fcc60730000-0x7fcc60930000	/usr/lib64/liblcms2.so.2.0.6
	0x7fcc60930000-0x7fcc60931000	/usr/lib64/liblcms2.so.2.0.6
	0x7fcc60931000-0x7fcc60936000	/usr/lib64/liblcms2.so.2.0.6
	0x7fcc60936000-0x7fcc60ac9000	/lib64/libc-2.22.so
	0x7fcc60ac9000-0x7fcc60cc9000	/lib64/libc-2.22.so
	0x7fcc60cc9000-0x7fcc60ccd000	/lib64/libc-2.22.so
	0x7fcc60ccd000-0x7fcc60ccf000	/lib64/libc-2.22.so
	0x7fcc60ccf000-0x7fcc60cd3000	
	0x7fcc60cd3000-0x7fcc60ce9000	/usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
	0x7fcc60ce9000-0x7fcc60ee8000	/usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
	0x7fcc60ee8000-0x7fcc60ee9000	/usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
	0x7fcc60ee9000-0x7fcc60eea000	/usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1
	0x7fcc60eea000-0x7fcc60ef0000	/lib64/librt-2.22.so
	0x7fcc60ef0000-0x7fcc610f0000	/lib64/librt-2.22.so
	0x7fcc610f0000-0x7fcc610f1000	/lib64/librt-2.22.so
	0x7fcc610f1000-0x7fcc610f2000	/lib64/librt-2.22.so
	0x7fcc610f2000-0x7fcc61109000	/lib64/libpthread-2.22.so
	0x7fcc61109000-0x7fcc61308000	/lib64/libpthread-2.22.so
	0x7fcc61308000-0x7fcc61309000	/lib64/libpthread-2.22.so
	0x7fcc61309000-0x7fcc6130a000	/lib64/libpthread-2.22.so
	0x7fcc6130a000-0x7fcc6130e000	
	0x7fcc6130e000-0x7fcc6140b000	/lib64/libm-2.22.so
	0x7fcc6140b000-0x7fcc6160a000	/lib64/libm-2.22.so
	0x7fcc6160a000-0x7fcc6160b000	/lib64/libm-2.22.so
	0x7fcc6160b000-0x7fcc6160c000	/lib64/libm-2.22.so
	0x7fcc6160c000-0x7fcc6160e000	/lib64/libdl-2.22.so
	0x7fcc6160e000-0x7fcc6180e000	/lib64/libdl-2.22.so
	0x7fcc6180e000-0x7fcc6180f000	/lib64/libdl-2.22.so
	0x7fcc6180f000-0x7fcc61810000	/lib64/libdl-2.22.so
	0x7fcc61810000-0x7fcc61e6e000	/usr/lib64/libGraphicsMagick.so.3.15.0
	0x7fcc61e6e000-0x7fcc6206e000	/usr/lib64/libGraphicsMagick.so.3.15.0
	0x7fcc6206e000-0x7fcc6209f000	/usr/lib64/libGraphicsMagick.so.3.15.0
	0x7fcc6209f000-0x7fcc62125000	/usr/lib64/libGraphicsMagick.so.3.15.0
	0x7fcc62125000-0x7fcc621a0000	
	0x7fcc621a0000-0x7fcc621c2000	/lib64/ld-2.22.so
	0x7fcc6228e000-0x7fcc62317000	
	0x7fcc6231b000-0x7fcc62322000	
	0x7fcc62322000-0x7fcc62329000	/usr/lib64/gconv/gconv-modules.cache
	0x7fcc62329000-0x7fcc6234c000	/usr/share/locale/it/LC_MESSAGES/libc.mo
	0x7fcc6234c000-0x7fcc623b6000	
	0x7fcc623b6000-0x7fcc623c1000	
	0x7fcc623c1000-0x7fcc623c2000	/lib64/ld-2.22.so
	0x7fcc623c2000-0x7fcc623c3000	/lib64/ld-2.22.so
	0x7fcc623c3000-0x7fcc623c4000	
	0x7ffcfee34000-0x7ffcfee55000	[stack]
	0x7ffcfef4c000-0x7ffcfef4e000	[vvar]
	0x7ffcfef4e000-0x7ffcfef50000	[vdso]
	0xffffffffff600000-0xffffffffff601000	[vsyscall]
==12404==End of process memory map.
==12404==AddressSanitizer CHECK failed: /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
    #0 0x4c9b3d in AsanCheckFailed /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_rtl.cc:67
    #1 0x4d0673 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:159
    #2 0x4d0861 in __sanitizer::ReportMmapFailureAndDie(unsigned long, char const*, char const*, int, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183
    #3 0x4d989a in __sanitizer::MmapOrDie(unsigned long, char const*, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_posix.cc:122
    #4 0x421c2f in __sanitizer::LargeMmapAllocator::Allocate(__sanitizer::AllocatorStats*, unsigned long, unsigned long) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1033
    #5 0x421c2f in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback>, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >, __sanitizer::LargeMmapAllocator >::Allocate(__sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >*, unsigned long, unsigned long, bool, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1302
    #6 0x421c2f in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:368
    #7 0x421c2f in __asan::asan_malloc(unsigned long, __sanitizer::BufferedStackTrace*) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:718
    #8 0x4c0201 in malloc /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:53
    #9 0x7fcc61c6a3f2 in MagickRealloc /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/memory.c:471:18
    #10 0x7fcc61cbb2b0 in OpenCache /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/pixel_cache.c:3155:7
    #11 0x7fcc61cb98fd in ModifyCache /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/pixel_cache.c:2955:18
    #12 0x7fcc61cbee4c in SetCacheNexus /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/pixel_cache.c:3878:7
    #13 0x7fcc61cbf5e1 in SetCacheViewPixels /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/pixel_cache.c:3957:10
    #14 0x7fcc61cbf5e1 in SetImagePixels /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/pixel_cache.c:4023
    #15 0x7fcc56235483 in ReadJPEGImage /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/coders/jpeg.c:1344:9
    #16 0x7fcc61ad3a8a in ReadImage /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/constitute.c:1607:13
    #17 0x7fcc566ed13e in ReadOneJNGImage /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/coders/png.c:3308:17
    #18 0x7fcc566d6f72 in ReadJNGImage /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/coders/png.c:3516:9
    #19 0x7fcc61ad3a8a in ReadImage /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/constitute.c:1607:13
    #20 0x7fcc61ad1a4b in PingImage /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/constitute.c:1370:9
    #21 0x7fcc61a23240 in IdentifyImageCommand /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:8372:17
    #22 0x7fcc61a27786 in MagickCommand /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:8862:17
    #23 0x7fcc61a81740 in GMCommandSingle /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:17370:10
    #24 0x7fcc61a7fce3 in GMCommand /tmp/portage/media-gfx/graphicsmagick-1.3.24/work/GraphicsMagick-1.3.24/magick/command.c:17423:16
    #25 0x7fcc6095661f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
    #26 0x418cd8 in _init (/usr/bin/gm+0x418cd8)

/usr/bin/gm identify: abort due to signal 6 (SIGABRT) "Abort"...

Affected version:
1.3.25

Fixed version:
1.3.26 (not released atm)

Commit fix:
http://hg.code.sf.net/p/graphicsmagick/code/rev/38d0f281e8c8

Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.

CVE:
CVE-2016-9830

Reproducer:
https://github.com/asarubbo/poc/blob/master/00096-graphicsmagick-memalloc-MagickRealloc

Timeline:
2016-10-19: bug discovered and reported privately to upstream
2016-10-21: upstream released a patch
2016-12-01: blog post about the issue
2016-12-05: CVE assigned

Note:
This bug was found with American Fuzzy Lop.

Permalink:

graphicsmagick: memory allocation failure in MagickRealloc (memory.c)