mp3splt: invalid free in free_options (options_manager.c)

16 Responses to mp3splt: invalid free in free_options (options_manager.c)

1. Jonas Meurer says: February 22, 2017 at 7:38 pm

   Hi, unfortunately I fail to reproduce the crash as the reproducer file is not accepted as input by mp3splt:

   $ mp3splt -P -f -t 0.1 -a 00130-mp3splt-badfree-free_options
   mp3splt 2.6.2 (09/11/14) – using libmp3splt 0.9.2
   Matteo Trotta
   Alexandru Munteanu
   THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
   Pretending to split file ‘00130-mp3splt-badfree-free_options’ …
   error: no plugin matches the file ‘00130-mp3splt-badfree-free_options’

   $ cat 00130-mp3splt-badfree-free_options | mp3splt -P -f -t 0.1 -a m-
   mp3splt 2.6.2 (09/11/14) – using libmp3splt 0.9.2
   Matteo Trotta
   Alexandru Munteanu
   THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
   Pretending to split file ‘m-‘ …
   info: file matches the plugin ‘mp3 (libmad)’
   error: invalid input file ‘m-‘ for ‘mp3 (libmad)’ plugin

   Reply
   • ago says: February 23, 2017 at 12:17 pm

     Are you sure you have downloaded the file instead of the html page? i.e.:
     # wget https://github.com/asarubbo/poc/blob/master/00130-mp3splt-badfree-free_options
     # file 00130-mp3splt-badfree-free_options
     00130-mp3splt-badfree-free_options: HTML document, UTF-8 Unicode text, with very long lines

     # wget https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options
     # file 00130-mp3splt-badfree-free_options
     00130-mp3splt-badfree-free_options: data

     Reply
2. Jonas Meurer says: February 23, 2017 at 3:53 pm

   @ago: I’m sure that I downloaded the *file*, not the HTML page. I also checked with ‘file …’ and it gave ‘data’. Are you able to run ‘mp3splt -P -f -t 0.1 -a 00130-mp3splt-badfree-free_options’ successfully?

   Reply
   • ago says: February 23, 2017 at 5:49 pm

     yes.

     # wget https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options && mp3splt -P -f -t 0.1 -a 00130*
     –2017-02-23 15:48:14– https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options
     Resolving github.com… 192.30.253.113, 192.30.253.112
     Connecting to github.com|192.30.253.113|:443… connected.
     HTTP request sent, awaiting response… 302 Found
     Location: https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options [following]
     –2017-02-23 15:48:15– https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options
     Resolving raw.githubusercontent.com… 151.101.64.133, 151.101.128.133, 151.101.192.133, …
     Connecting to raw.githubusercontent.com|151.101.64.133|:443… connected.
     HTTP request sent, awaiting response… 200 OK
     Length: 26968 (26K) [application/octet-stream]
     Saving to: ‘00130-mp3splt-badfree-free_options’

     00130-mp3splt-badfree-free_options 100%[==================================================================================================================================================================>] 26.34K –.-KB/s in 0.05s

     2017-02-23 15:48:15 (579 KB/s) – ‘00130-mp3splt-badfree-free_options’ saved [26968/26968]

     mp3splt 2.6.1a (19/07/14) – using libmp3splt 0.9.1a
     Matteo Trotta
     Alexandru Munteanu
     THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
     Pretending to split file ‘00130-mp3splt-badfree-free_options’ …
     error: no plugin matches the file ‘00130-mp3splt-badfree-free_options’
     =================================================================
     ==26813==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x000000d3ef65 in thread T0
     #0 0x4d35d0 (/usr/bin/mp3splt+0x4d35d0)
     #1 0x50c598 (/usr/bin/mp3splt+0x50c598)
     #2 0x511a94 (/usr/bin/mp3splt+0x511a94)
     #3 0x50df37 (/usr/bin/mp3splt+0x50df37)
     #4 0x517567 (/usr/bin/mp3splt+0x517567)
     #5 0x7fa66fb8378f (/lib64/libc.so.6+0x2078f)
     #6 0x41ab68 (/usr/bin/mp3splt+0x41ab68)

     AddressSanitizer can not describe address in more detail (wild memory access suspected).
     SUMMARY: AddressSanitizer: bad-free (/usr/bin/mp3splt+0x4d35d0)
     ==26813==ABORTING
     Aborted

     As you can see 2.6.1 is affected too.

     Reply
3. Jonas Meurer says: February 23, 2017 at 8:27 pm

   Interesting, at least on Debian the reproducer doesn’t work. I tried it on Debian Stretch with mp3plt 2.6.2, on Debian Jessie with mp3splt 2.4.2 and on Debian Wheezy, also with mp3splt 2.4.2.

   Unfortunately that doesn’t mean that the mp3splt Debian packages aren’t affected, only it makes debugging the issue more complicated. I guess that you use mp3splt 2.6.1a compiled from the Gentoo port sources?

   Below is the output of the exact same command like you ran:

   “`
   $ wget https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options && mp3splt -P -f -t 0.1 -a 00130*
   –2017-02-23 18:24:58– https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options
   Resolving github.com (github.com)… 192.30.253.113, 192.30.253.112
   Connecting to github.com (github.com)|192.30.253.113|:443… connected.
   HTTP request sent, awaiting response… 302 Found
   Location: https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options [following]
   –2017-02-23 18:24:59– https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options
   Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.112.133
   Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.112.133|:443… connected.
   HTTP request sent, awaiting response… 200 OK
   Length: 26968 (26K) [application/octet-stream]
   Saving to: `00130-mp3splt-badfree-free_options’

   100%[=================================================>] 26,968 –.-K/s in 0.04s

   2017-02-23 18:25:00 (738 KB/s) – `00130-mp3splt-badfree-free_options’ saved [26968/26968]

   mp3splt 2.4.2 (13/05/12) – using libmp3splt 0.7.2
   Matteo Trotta
   Alexandru Munteanu
   THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
   Pretending to split file ‘00130-mp3splt-badfree-free_options’ …
   error: no plugin matches the file ‘00130-mp3splt-badfree-free_options’
   “`

   Reply
   • ago says: February 24, 2017 at 12:16 pm

     Your package, if comes from the apt repo, is not compiled with any special flags like -fsanitize=address. So you won’t see what I’m able to see here. However I tried to build without -fsanitize=address and I see something like:
     *** Error in `mp3splt’: munmap_chunk(): invalid pointer: 0x0000000000a6a110 ***

     Reply
4. Jan Naumann says: February 25, 2017 at 4:06 pm

   I also have tried to reproduce the bug, but cannot get the bug even if enabling -fsanitize=address.
   Could you post your complete gcc/clang command line for reference?

   Reply
5. Dave Kennedy says: February 27, 2017 at 9:31 am

   This bug is marked as a release critical in Debian, and therefore mp3splt is marked for autoremoval from testing on 2017-03-06. This means that it won’t be included in the upcoming release of Debian stable (codename stretch) unless we can fix this bug.

   I am NOT able to reproduce the original error message in the bug report. Following is a list of symptoms that I see on Gentoo (default/linux/amd64/13.0).

   1.) With default optimizations and AddressSanitizer enabled (but no _FORTIFY_SOURCE or stack protector):
   AddressSanitizer: heap-buffer-overflow on address 0x61e00000ee87 at pc 0x7f3944bd4b43 bp 0x7ffcf2d62e40 sp 0x7ffcf2d625e8
   The backtrace starts with mad_layer_III (/usr/lib64/libmad.so.0+0x1b8bd)

   2.) With -Og (debugging compatible optimizations) and -g enabled for libmad, libmp3splt, mp3splt, and with AddressSanitizer enabled (but no _FORTIFY_SOURCE or stack protector) for only mp3splt:

   Program received signal SIGABRT, Aborted.
   0x00007ffff693d167 in raise () from /lib64/libc.so.6
   (gdb) bt
   #0 0x00007ffff693d167 in raise () from /lib64/libc.so.6
   #1 0x00007ffff693e5ea in abort () from /lib64/libc.so.6
   #2 0x00007ffff693619d in ?? () from /lib64/libc.so.6
   #3 0x00007ffff6936252 in __assert_fail () from /lib64/libc.so.6
   #4 0x00007ffff2858d36 in mad_layer_III (stream=0x62c000000a28, frame=0x62c000000aa0) at /var/tmp/portage/media-libs/libmad-0.15.1b-r8/work/libmad-0.15.1b/layer3.c:2632
   #5 0x00007ffff283deb0 in mad_frame_decode (frame=0x62c000000aa0, stream=0x62c000000a28) at /var/tmp/portage/media-libs/libmad-0.15.1b-r8/work/libmad-0.15.1b/frame.c:453
   #6 0x00007ffff2a88c02 in splt_mp3_get_frame (mp3state=mp3state@entry=0x62c000000200) at mp3_utils.c:1109
   #7 0x00007ffff2a7a72f in splt_mp3_info (file_input=file_input@entry=0x61600000e180, state=state@entry=0x62300000e100, framemode=, error=error@entry=0x7fffffffdbe0) at mp3.c:1247
   #8 0x00007ffff2a7b747 in splt_mp3_get_info (state=state@entry=0x62300000e100, file_input=, error=error@entry=0x7fffffffdbe0) at mp3.c:1479
   #9 0x00007ffff2a82956 in splt_mp3_init (state=state@entry=0x62300000e100, error=error@entry=0x7fffffffdbe0) at mp3.c:3377
   #10 0x00007ffff2a82fd6 in splt_pl_check_plugin_is_for_file (state=0x62300000e100, error=0x7fffffffdbe0) at mp3.c:3496
   #11 0x00007ffff6cc4d26 in splt_p_check_plugin_is_for_file (state=state@entry=0x62300000e100, error=error@entry=0x7fffffffdbe0) at plugins.c:748
   #12 0x00007ffff6cc0f6c in splt_check_file_type_and_set_plugin (state=state@entry=0x62300000e100, force_check_by_extension=force_check_by_extension@entry=0, show_warnings=show_warnings@entry=1, error=error@entry=0x7fffffffdca0)
   at checks.c:317
   #13 0x00007ffff6cbcaa3 in mp3splt_split (state=state@entry=0x62300000e100) at mp3splt.c:1226
   #14 0x000000000040ed88 in main (argc=, orig_argv=) at mp3splt.c:872

   This assert fails on the second call to mad_layer_III(). An initial inspection of various variables reveals only one clue so far: stream->main_data = 0x0. That means it’s a NULL pointer at the time of crash, even though it IS valid (nonzero) at the beginning of the second call to the function.

   In my opinion, the next step is to try feeding the same data to libmad with a minimal test case (ie. remove mp3splt from the test case).

   agostino, could you please rebuild the relevant packages with debugging enabled and provide a complete backtrace and AddressSanitizer log? And if you want extra credit then you could do some fuzzing on libmad, as that might be the root cause anyway.

   On Gentoo in order to build packages with debugging, edit make.conf
   CFLAGS=”-Og -g
   CXXFLAGS=”${CFLAGS}

   If you want AddressSanitizer enabled then you can use these CFLAGS.

   CFLAGS=”${CFLAGS} -fsanitize=address -U_FORTIFY_SOURCE”
   CXXFLAGS=”${CXXFLAGS} -fsanitize=address -U_FORTIFY_SOURCE”
   LDFLAGS=”${LDFLAGS} -fsanitize=address -U_FORTIFY_SOURCE”

   You will need to patch configure.ac in libmad to prevent stripping of -fsanitize=address. (Remove |-f from the case statement.) You also need to patch ltmain.sh in all packages so that it doesn’t strip -fsanitize=address from the LDFLAGS. Search AddressSanitizer on the Gentoo wiki for instructions (the Asantoo overlay also configures AddressSanitizer with the ASAN_OPTIONS environment variable so that logs are stored on the disk).

   Then emerge with the following command.

   USE=”debug c-debug no-optimise” FEATURES=nostrip emerge –ask –verbose libmad libmp3splt mp3splt

   And to start debugging, use the following command.

   gdb $(which mp3splt)
   set args -P -f -t 0.1 -a 00130-mp3splt-badfree-free_options
   set logging file gdb_output.txt
   set logging on
   start
   cont
   (when it crashes then type the following to get a backtrace including debugging symbols)
   bt

   The output will be stored in gdb_output.txt.

   Reply
   • ago says: February 27, 2017 at 10:21 am

     Hello Dave,

     the fuzzing stuff is done into a dedicated chroot where all packages have the debug symbols, i.e. -O2 -mtune=generic -march=x86-64 -fno-stack-protector -g3 -ggdb3 -U_FORTIFY_SOURCE ( I didn’t investigate if I can use just one between -g3/-ggdb3 but for now it is fine as is).
     The asan stuff is delegated via package.env, so thanks for let me know how to do it but I’m already doing this since a lot of time 🙂
     I’ll try to work on it and make you able to reproduce the issue.

     Reply
     • ago says: February 27, 2017 at 3:16 pm

       I investigate a bit.
       When you launch mp3splt, the order where thing go are: mp3splt init -> libmad stuff -> bad free. While you compile libmad with debug, it hangs before you can see the bad free for some reasons. So you can compile libmad with -g/-ggdb but you shouldn’t use –enable-debugging. Does it helps?
       In the meantime I’m investigating on the libmad status…

       Reply
6. Dave K says: March 5, 2017 at 8:13 am

   I tried using the following make.conf settings. This is not a complete list. See the links below for a complete list of make.conf settings.

   CFLAGS=-O2 -pipe -mtune=generic -march=x86-64 -g3 -ggdb3 -U_FORTIFY_SOURCE -fsanitize=address CXXFLAGS=${CFLAGS}
   LDFLAGS=-U_FORTIFY_SOURCE -fsanitize=address

   And I modified libmad in the following ways.

   1. FEATURES=nostrip ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild prepare
   2. As a side effect of the above command, the asantoo overlay automatically runs a script that fixes libtool CFLAGS filtering for asan.
   3. cd /var/tmp/portage/media-libs/libmad/libmad-0.15.1b-r8/work/libmad-0.15.1b-r8
   4. autoupdate
   5. Patch configure.ac and Makefile.in to ensure that the CFLAGS from make.conf are used during the build (below is a link to the patch file that I wrote).
   6. touch NEWS AUTHOR ChangeLog
   7. Copy autogen.sh from mp3splt and run it so that the changes in step 3 take effect.
   8. FEATURES=nostrip ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild compile
   9. In order to allow gdb to read the source code, make a backup copy of /var/tmp/portage/media-libs/libmad/libmad-0.15.1b-r8
   10. FEATURES=nostrip ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild merge
   11. Restore the backup of /var/tmp/portage/media-libs/libmad/libmad-0.15.1b-r8
   12. Now the gdb backtrace and debugging include the source code

   I followed a similar process for libmp3splt and mp3splt, except that I built them with the environment variables USE=-optimise FEATURES=nostrip in order to prevent the use of -O3. I don’t know why the Makefile sets LDFLAGS= -Wl,-O1 because it isn’t in any of the autoconf files, but one can edit the Makefile before compiling (I did not).

   Anyway, I again ran the test case that you provided, but I still haven’t reproduced the symptoms that you described. Instead, the assert failed and the program aborted without any errors from AddressSanitizer.

   Following are relevant settings.

   PACKAGE.USE settings
   http://pastebin.com/raw/QGKf3ae7

   MAKE.CONF settings
   http://pastebin.com/raw/tc2ug1J1

   patch to ensure the correct build flags with libmad (the comments explain how it can be improved).
   http://pastebin.com/raw/p27bDBMt

   Reply
7. Dave K says: March 5, 2017 at 9:00 am

   UPDATE. I did more debugging and found that libmp3splt loops on calls to mad_frame_decode() without doing any operations on memory between said calls.

   Also, what I said earlier about main_data being a NULL pointer is not correct. As long as mp3splt, libmp3splt and libmad were built with debugging symbols and working assert(), whenever I checked main_data it was valid if it was supposed to be valid.

   So in my opinion this bug should be filed with libmad. Sure enough, there is a bug already reported with libmad that in my opinion matches the symptoms exactly: Debian Bug #287519.

   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=287519

   Glenn Maynard said that “…the assertion isn’t compiled into release
   libraries, which means that it’s clobbering memory and crashing later–causing
   hard-to-trace memory corruption later on.”

   I don’t know what the solution to this problem is immediately, but for now can we agree to request that this bug be reassigned to libmad instead of mp3splt? My guess is that the reproducer file would cause the same symptoms in madplay if it were patched to continue processing the file despite synchronization failure.

   Here is some of the output from my debugging session and a link to the ISO/IEC 11172-3 standard (MP3 decoding guidelines).

   Debugging session with the values of relevant variables just before the assert fails
   http://pastebin.com/raw/PSb8SDiD

   ISO/IEC 11172 – Part 3: Audio
   http://www.mp3-tech.org/programmer/docs/iso11172-3.zip

   More MP3 technical documents
   http://www.mp3-tech.org/programmer/docs/index.php

   Reply
   • ago says: March 5, 2017 at 1:38 pm

     Dave,

     it is great that you spend time on it, but you made a process that I never suggester and/or described.

     When I do fuzzing, I use more advanced features but basically you _need_ to have:

     # cat /etc/portage/env/asan
     CC=”clang”
     CXX=”clang++”
     CFLAGS=”${CFLAGS} -fsanitize=address”
     CXXFLAGS=”${CXXFLAGS} -fsanitize=address”
     LDFLAGS=”${LDFLAGS} -fsanitize=address”

     # cat /etc/portage/package.env
     media-sound/mp3splt asan

     # grep mp3splt /etc/portage/package.keywords
     media-sound/mp3splt

     and FEATURES=”splitdebug” in make.conf

     then, simply emerge -v mp3splt

     Apart that. I worked on it too this morning and I discovered that if you compile it with gcc you see an overflow in libmad (which I already reported days ago after your input to fuzz libmad too) and if you compile with clang-3.9.1 you can see the bad free. Would at this point it help? If not, send me an email and I will provide a chroot where you will able to reproduce what I described. Thanks.

     Reply
8. Dave K says: March 6, 2017 at 6:11 am

   I am able to reproduce the symptoms (invalid free in options_manager.c) after building mp3splt with clang and the other portage settings that you listed.

   Let us call this test “Test A”. This particular test does not determine which package is at fault for causing memory corruption. That’s because the stack protection and AddressSanitizer are disabled for everything except the main mp3splt binary. Yet mp3splt loads several dynamic libraries that could be the culprit.

   Test A also does not enable the critical assert() in libmad. Therefore a buffer overflow occurs, overwriting memory that later causes options_manager.c to attempt to free data that was not malloc’d. The reason that Test A does not enable the critical assert() is because the default USE flags are -debug, which eventually passes –disable-debugging to configure.ac. This in turn causes the NDEBUG variable to be set, which causes assert to be defined as /* nothing */.

   Sure enough, when I build with the following command

   USE=debug ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild merge

   which we shall call “Test B”, the assert fails and the program aborts with no buffer overflow and no invalid free. And yes, mp3splt is built with clang.

   In conclusion, the configure flags of libmad are the root cause of the problem, and mp3splt is only affected in the case that all protections are disabled (stack protection and AddressSanitizer).

   Do you think that the solution is to enable the asserts in libmad regardless of the debug flags?

   Reply
   • ago says: March 6, 2017 at 10:41 am

     Dave, I’m happy you finally reached the bug 🙂

     I didn’t look deeper in the code as I’m devolving the time only to fuzz and I let to do analysis to upstream. However as I explained in the past comment, the process looks like “mp3splt -> $OTHER_LIBS -> bad free”. While you enable the asserts in libmad it denies you to reach the bad free bug with the provided testcase, so consider the following:
     1) It may be possible that the bug comes up with another round of fuzzing where libmad has debug symbols (I’ll try);
     2) An attacker that know this story, can provide a crafted audio which can bypass what fails in the assert, but reach the bad free.

     Reply

9. Pingback: SB17-065: Vulnerability Summary for the Week of February 27, 2017 – sec.uno

This site uses Akismet to reduce spam. Learn how your comment data is processed.