Description:
mp3splt is a command line utility to split mp3 and ogg files without decoding.
A fuzz on it discovered an invalid free.
The complete ASan output:
# mp3splt -P -f -t 0.1 -a $FILE
==2631==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x000000d3ef65 in thread T0
#0 0x4d3770 in free /tmp/portage/sys-devel/llvm-3.9.0-r1/work/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:47
#1 0x50dbaa in free_options /tmp/portage/media-sound/mp3splt-2.6.2/work/mp3splt-2.6.2/src/options_manager.c:67:9
#2 0x515623 in free_main_struct /tmp/portage/media-sound/mp3splt-2.6.2/work/mp3splt-2.6.2/src/data_manager.c:74:7
#3 0x50ffa5 in process_confirmation_error /tmp/portage/media-sound/mp3splt-2.6.2/work/mp3splt-2.6.2/src/print_utils.c:266:7
#4 0x51df29 in main /tmp/portage/media-sound/mp3splt-2.6.2/work/mp3splt-2.6.2/src/mp3splt.c:873:9
#5 0x7f783aba361f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
#6 0x41ad08 in _init (/usr/bin/mp3splt+0x41ad08)
AddressSanitizer can not describe address in more detail (wild memory access suspected).
SUMMARY: AddressSanitizer: bad-free /tmp/portage/sys-devel/llvm-3.9.0-r1/work/llvm-3.9.0.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:47 in free
==2631==ABORTING
Affected version:
0.9.2
Fixed version:
N/A
Commit fix:
N/A
Credit:
This bug was discovered by Agostino Sarubbo of Gentoo.
CVE:
CVE-2017-5666
Reproducer:
https://github.com/asarubbo/poc/blob/master/00130-mp3splt-badfree-free_options
Timeline:
2017-01-01: private report to upstream via mail
2017-01-29: public upstream report on sourceforge
2017-01-29: blog post about the issue
2017-01-31: CVE assigned
Note:
This bug was found with American Fuzzy Lop.
Permalink:
https://blogs.gentoo.org/ago/2017/01/29/mp3splt-invalid-free-in-free_options-options_manager-c
Hi, unfortunately I fail to reproduce the crash as the reproducer file is not accepted as input by mp3splt:
$ mp3splt -P -f -t 0.1 -a 00130-mp3splt-badfree-free_options
mp3splt 2.6.2 (09/11/14) – using libmp3splt 0.9.2
Matteo Trotta
Alexandru Munteanu
THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
Pretending to split file ‘00130-mp3splt-badfree-free_options’ …
error: no plugin matches the file ‘00130-mp3splt-badfree-free_options’
$ cat 00130-mp3splt-badfree-free_options | mp3splt -P -f -t 0.1 -a m-
mp3splt 2.6.2 (09/11/14) – using libmp3splt 0.9.2
Matteo Trotta
Alexandru Munteanu
THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
Pretending to split file ‘m-‘ …
info: file matches the plugin ‘mp3 (libmad)’
error: invalid input file ‘m-‘ for ‘mp3 (libmad)’ plugin
Are you sure you have downloaded the file instead of the html page? i.e.:
# wget https://github.com/asarubbo/poc/blob/master/00130-mp3splt-badfree-free_options
# file 00130-mp3splt-badfree-free_options
00130-mp3splt-badfree-free_options: HTML document, UTF-8 Unicode text, with very long lines
# wget https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options
# file 00130-mp3splt-badfree-free_options
00130-mp3splt-badfree-free_options: data
@ago: I’m sure that I downloaded the *file*, not the HTML page. I also checked with ‘file …’ and it gave ‘data’. Are you able to run ‘mp3splt -P -f -t 0.1 -a 00130-mp3splt-badfree-free_options’ successfully?
yes.
# wget https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options && mp3splt -P -f -t 0.1 -a 00130*
–2017-02-23 15:48:14– https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options
Resolving github.com… 192.30.253.113, 192.30.253.112
Connecting to github.com|192.30.253.113|:443… connected.
HTTP request sent, awaiting response… 302 Found
Location: https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options [following]
–2017-02-23 15:48:15– https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options
Resolving raw.githubusercontent.com… 151.101.64.133, 151.101.128.133, 151.101.192.133, …
Connecting to raw.githubusercontent.com|151.101.64.133|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 26968 (26K) [application/octet-stream]
Saving to: ‘00130-mp3splt-badfree-free_options’
00130-mp3splt-badfree-free_options 100%[==================================================================================================================================================================>] 26.34K –.-KB/s in 0.05s
2017-02-23 15:48:15 (579 KB/s) – ‘00130-mp3splt-badfree-free_options’ saved [26968/26968]
mp3splt 2.6.1a (19/07/14) – using libmp3splt 0.9.1a
Matteo Trotta
Alexandru Munteanu
THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
Pretending to split file ‘00130-mp3splt-badfree-free_options’ …
error: no plugin matches the file ‘00130-mp3splt-badfree-free_options’
=================================================================
==26813==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x000000d3ef65 in thread T0
#0 0x4d35d0 (/usr/bin/mp3splt+0x4d35d0)
#1 0x50c598 (/usr/bin/mp3splt+0x50c598)
#2 0x511a94 (/usr/bin/mp3splt+0x511a94)
#3 0x50df37 (/usr/bin/mp3splt+0x50df37)
#4 0x517567 (/usr/bin/mp3splt+0x517567)
#5 0x7fa66fb8378f (/lib64/libc.so.6+0x2078f)
#6 0x41ab68 (/usr/bin/mp3splt+0x41ab68)
AddressSanitizer can not describe address in more detail (wild memory access suspected).
SUMMARY: AddressSanitizer: bad-free (/usr/bin/mp3splt+0x4d35d0)
==26813==ABORTING
Aborted
As you can see 2.6.1 is affected too.
Interesting, at least on Debian the reproducer doesn’t work. I tried it on Debian Stretch with mp3plt 2.6.2, on Debian Jessie with mp3splt 2.4.2 and on Debian Wheezy, also with mp3splt 2.4.2.
Unfortunately that doesn’t mean that the mp3splt Debian packages aren’t affected, only it makes debugging the issue more complicated. I guess that you use mp3splt 2.6.1a compiled from the Gentoo port sources?
Below is the output of the exact same command like you ran:
“`
$ wget https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options && mp3splt -P -f -t 0.1 -a 00130*
–2017-02-23 18:24:58– https://github.com/asarubbo/poc/raw/master/00130-mp3splt-badfree-free_options
Resolving github.com (github.com)… 192.30.253.113, 192.30.253.112
Connecting to github.com (github.com)|192.30.253.113|:443… connected.
HTTP request sent, awaiting response… 302 Found
Location: https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options [following]
–2017-02-23 18:24:59– https://raw.githubusercontent.com/asarubbo/poc/master/00130-mp3splt-badfree-free_options
Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 151.101.112.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.112.133|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 26968 (26K) [application/octet-stream]
Saving to: `00130-mp3splt-badfree-free_options’
100%[=================================================>] 26,968 –.-K/s in 0.04s
2017-02-23 18:25:00 (738 KB/s) – `00130-mp3splt-badfree-free_options’ saved [26968/26968]
mp3splt 2.4.2 (13/05/12) – using libmp3splt 0.7.2
Matteo Trotta
Alexandru Munteanu
THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
Pretending to split file ‘00130-mp3splt-badfree-free_options’ …
error: no plugin matches the file ‘00130-mp3splt-badfree-free_options’
“`
Your package, if comes from the apt repo, is not compiled with any special flags like -fsanitize=address. So you won’t see what I’m able to see here. However I tried to build without -fsanitize=address and I see something like:
*** Error in `mp3splt’: munmap_chunk(): invalid pointer: 0x0000000000a6a110 ***
I also have tried to reproduce the bug, but cannot get the bug even if enabling -fsanitize=address.
Could you post your complete gcc/clang command line for reference?
This bug is marked as a release critical in Debian, and therefore mp3splt is marked for autoremoval from testing on 2017-03-06. This means that it won’t be included in the upcoming release of Debian stable (codename stretch) unless we can fix this bug.
I am NOT able to reproduce the original error message in the bug report. Following is a list of symptoms that I see on Gentoo (default/linux/amd64/13.0).
1.) With default optimizations and AddressSanitizer enabled (but no _FORTIFY_SOURCE or stack protector):
AddressSanitizer: heap-buffer-overflow on address 0x61e00000ee87 at pc 0x7f3944bd4b43 bp 0x7ffcf2d62e40 sp 0x7ffcf2d625e8
The backtrace starts with mad_layer_III (/usr/lib64/libmad.so.0+0x1b8bd)
2.) With -Og (debugging compatible optimizations) and -g enabled for libmad, libmp3splt, mp3splt, and with AddressSanitizer enabled (but no _FORTIFY_SOURCE or stack protector) for only mp3splt:
Program received signal SIGABRT, Aborted.
0x00007ffff693d167 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x00007ffff693d167 in raise () from /lib64/libc.so.6
#1 0x00007ffff693e5ea in abort () from /lib64/libc.so.6
#2 0x00007ffff693619d in ?? () from /lib64/libc.so.6
#3 0x00007ffff6936252 in __assert_fail () from /lib64/libc.so.6
#4 0x00007ffff2858d36 in mad_layer_III (stream=0x62c000000a28, frame=0x62c000000aa0) at /var/tmp/portage/media-libs/libmad-0.15.1b-r8/work/libmad-0.15.1b/layer3.c:2632
#5 0x00007ffff283deb0 in mad_frame_decode (frame=0x62c000000aa0, stream=0x62c000000a28) at /var/tmp/portage/media-libs/libmad-0.15.1b-r8/work/libmad-0.15.1b/frame.c:453
#6 0x00007ffff2a88c02 in splt_mp3_get_frame (mp3state=mp3state@entry=0x62c000000200) at mp3_utils.c:1109
#7 0x00007ffff2a7a72f in splt_mp3_info (file_input=file_input@entry=0x61600000e180, state=state@entry=0x62300000e100, framemode=, error=error@entry=0x7fffffffdbe0) at mp3.c:1247
#8 0x00007ffff2a7b747 in splt_mp3_get_info (state=state@entry=0x62300000e100, file_input=, error=error@entry=0x7fffffffdbe0) at mp3.c:1479
#9 0x00007ffff2a82956 in splt_mp3_init (state=state@entry=0x62300000e100, error=error@entry=0x7fffffffdbe0) at mp3.c:3377
#10 0x00007ffff2a82fd6 in splt_pl_check_plugin_is_for_file (state=0x62300000e100, error=0x7fffffffdbe0) at mp3.c:3496
#11 0x00007ffff6cc4d26 in splt_p_check_plugin_is_for_file (state=state@entry=0x62300000e100, error=error@entry=0x7fffffffdbe0) at plugins.c:748
#12 0x00007ffff6cc0f6c in splt_check_file_type_and_set_plugin (state=state@entry=0x62300000e100, force_check_by_extension=force_check_by_extension@entry=0, show_warnings=show_warnings@entry=1, error=error@entry=0x7fffffffdca0)
at checks.c:317
#13 0x00007ffff6cbcaa3 in mp3splt_split (state=state@entry=0x62300000e100) at mp3splt.c:1226
#14 0x000000000040ed88 in main (argc=, orig_argv=) at mp3splt.c:872
This assert fails on the second call to mad_layer_III(). An initial inspection of various variables reveals only one clue so far: stream->main_data = 0x0. That means it’s a NULL pointer at the time of crash, even though it IS valid (nonzero) at the beginning of the second call to the function.
In my opinion, the next step is to try feeding the same data to libmad with a minimal test case (ie. remove mp3splt from the test case).
agostino, could you please rebuild the relevant packages with debugging enabled and provide a complete backtrace and AddressSanitizer log? And if you want extra credit then you could do some fuzzing on libmad, as that might be the root cause anyway.
On Gentoo in order to build packages with debugging, edit make.conf
CFLAGS=”-Og -g
CXXFLAGS=”${CFLAGS}
If you want AddressSanitizer enabled then you can use these CFLAGS.
CFLAGS=”${CFLAGS} -fsanitize=address -U_FORTIFY_SOURCE”
CXXFLAGS=”${CXXFLAGS} -fsanitize=address -U_FORTIFY_SOURCE”
LDFLAGS=”${LDFLAGS} -fsanitize=address -U_FORTIFY_SOURCE”
You will need to patch configure.ac in libmad to prevent stripping of -fsanitize=address. (Remove |-f from the case statement.) You also need to patch ltmain.sh in all packages so that it doesn’t strip -fsanitize=address from the LDFLAGS. Search AddressSanitizer on the Gentoo wiki for instructions (the Asantoo overlay also configures AddressSanitizer with the ASAN_OPTIONS environment variable so that logs are stored on the disk).
Then emerge with the following command.
USE=”debug c-debug no-optimise” FEATURES=nostrip emerge –ask –verbose libmad libmp3splt mp3splt
And to start debugging, use the following command.
gdb $(which mp3splt)
set args -P -f -t 0.1 -a 00130-mp3splt-badfree-free_options
set logging file gdb_output.txt
set logging on
start
cont
(when it crashes then type the following to get a backtrace including debugging symbols)
bt
The output will be stored in gdb_output.txt.
Hello Dave,
the fuzzing stuff is done into a dedicated chroot where all packages have the debug symbols, i.e. -O2 -mtune=generic -march=x86-64 -fno-stack-protector -g3 -ggdb3 -U_FORTIFY_SOURCE ( I didn’t investigate if I can use just one between -g3/-ggdb3 but for now it is fine as is).
The asan stuff is delegated via package.env, so thanks for let me know how to do it but I’m already doing this since a lot of time 🙂
I’ll try to work on it and make you able to reproduce the issue.
I investigate a bit.
When you launch mp3splt, the order where thing go are: mp3splt init -> libmad stuff -> bad free. While you compile libmad with debug, it hangs before you can see the bad free for some reasons. So you can compile libmad with -g/-ggdb but you shouldn’t use –enable-debugging. Does it helps?
In the meantime I’m investigating on the libmad status…
I tried using the following make.conf settings. This is not a complete list. See the links below for a complete list of make.conf settings.
CFLAGS=-O2 -pipe -mtune=generic -march=x86-64 -g3 -ggdb3 -U_FORTIFY_SOURCE -fsanitize=address CXXFLAGS=${CFLAGS}
LDFLAGS=-U_FORTIFY_SOURCE -fsanitize=address
And I modified libmad in the following ways.
1. FEATURES=nostrip ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild prepare
2. As a side effect of the above command, the asantoo overlay automatically runs a script that fixes libtool CFLAGS filtering for asan.
3. cd /var/tmp/portage/media-libs/libmad/libmad-0.15.1b-r8/work/libmad-0.15.1b-r8
4. autoupdate
5. Patch configure.ac and Makefile.in to ensure that the CFLAGS from make.conf are used during the build (below is a link to the patch file that I wrote).
6. touch NEWS AUTHOR ChangeLog
7. Copy autogen.sh from mp3splt and run it so that the changes in step 3 take effect.
8. FEATURES=nostrip ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild compile
9. In order to allow gdb to read the source code, make a backup copy of /var/tmp/portage/media-libs/libmad/libmad-0.15.1b-r8
10. FEATURES=nostrip ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild merge
11. Restore the backup of /var/tmp/portage/media-libs/libmad/libmad-0.15.1b-r8
12. Now the gdb backtrace and debugging include the source code
I followed a similar process for libmp3splt and mp3splt, except that I built them with the environment variables USE=-optimise FEATURES=nostrip in order to prevent the use of -O3. I don’t know why the Makefile sets LDFLAGS= -Wl,-O1 because it isn’t in any of the autoconf files, but one can edit the Makefile before compiling (I did not).
Anyway, I again ran the test case that you provided, but I still haven’t reproduced the symptoms that you described. Instead, the assert failed and the program aborted without any errors from AddressSanitizer.
Following are relevant settings.
PACKAGE.USE settings
http://pastebin.com/raw/QGKf3ae7
MAKE.CONF settings
http://pastebin.com/raw/tc2ug1J1
patch to ensure the correct build flags with libmad (the comments explain how it can be improved).
http://pastebin.com/raw/p27bDBMt
UPDATE. I did more debugging and found that libmp3splt loops on calls to mad_frame_decode() without doing any operations on memory between said calls.
Also, what I said earlier about main_data being a NULL pointer is not correct. As long as mp3splt, libmp3splt and libmad were built with debugging symbols and working assert(), whenever I checked main_data it was valid if it was supposed to be valid.
So in my opinion this bug should be filed with libmad. Sure enough, there is a bug already reported with libmad that in my opinion matches the symptoms exactly: Debian Bug #287519.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=287519
Glenn Maynard said that “…the assertion isn’t compiled into release
libraries, which means that it’s clobbering memory and crashing later–causing
hard-to-trace memory corruption later on.”
I don’t know what the solution to this problem is immediately, but for now can we agree to request that this bug be reassigned to libmad instead of mp3splt? My guess is that the reproducer file would cause the same symptoms in madplay if it were patched to continue processing the file despite synchronization failure.
Here is some of the output from my debugging session and a link to the ISO/IEC 11172-3 standard (MP3 decoding guidelines).
Debugging session with the values of relevant variables just before the assert fails
http://pastebin.com/raw/PSb8SDiD
ISO/IEC 11172 – Part 3: Audio
http://www.mp3-tech.org/programmer/docs/iso11172-3.zip
More MP3 technical documents
http://www.mp3-tech.org/programmer/docs/index.php
Dave,
it is great that you spend time on it, but you made a process that I never suggester and/or described.
When I do fuzzing, I use more advanced features but basically you _need_ to have:
# cat /etc/portage/env/asan
CC=”clang”
CXX=”clang++”
CFLAGS=”${CFLAGS} -fsanitize=address”
CXXFLAGS=”${CXXFLAGS} -fsanitize=address”
LDFLAGS=”${LDFLAGS} -fsanitize=address”
# cat /etc/portage/package.env
media-sound/mp3splt asan
# grep mp3splt /etc/portage/package.keywords
media-sound/mp3splt
and FEATURES=”splitdebug” in make.conf
then, simply emerge -v mp3splt
Apart that. I worked on it too this morning and I discovered that if you compile it with gcc you see an overflow in libmad (which I already reported days ago after your input to fuzz libmad too) and if you compile with clang-3.9.1 you can see the bad free. Would at this point it help? If not, send me an email and I will provide a chroot where you will able to reproduce what I described. Thanks.
I am able to reproduce the symptoms (invalid free in options_manager.c) after building mp3splt with clang and the other portage settings that you listed.
Let us call this test “Test A”. This particular test does not determine which package is at fault for causing memory corruption. That’s because the stack protection and AddressSanitizer are disabled for everything except the main mp3splt binary. Yet mp3splt loads several dynamic libraries that could be the culprit.
Test A also does not enable the critical assert() in libmad. Therefore a buffer overflow occurs, overwriting memory that later causes options_manager.c to attempt to free data that was not malloc’d. The reason that Test A does not enable the critical assert() is because the default USE flags are -debug, which eventually passes –disable-debugging to configure.ac. This in turn causes the NDEBUG variable to be set, which causes assert to be defined as /* nothing */.
Sure enough, when I build with the following command
USE=debug ebuild /usr/portage/media-libs/libmad/libmad-0.15.1b-r8.ebuild merge
which we shall call “Test B”, the assert fails and the program aborts with no buffer overflow and no invalid free. And yes, mp3splt is built with clang.
In conclusion, the configure flags of libmad are the root cause of the problem, and mp3splt is only affected in the case that all protections are disabled (stack protection and AddressSanitizer).
Do you think that the solution is to enable the asserts in libmad regardless of the debug flags?
Dave, I’m happy you finally reached the bug 🙂
I didn’t look deeper in the code as I’m devolving the time only to fuzz and I let to do analysis to upstream. However as I explained in the past comment, the process looks like “mp3splt -> $OTHER_LIBS -> bad free”. While you enable the asserts in libmad it denies you to reach the bad free bug with the provided testcase, so consider the following:
1) It may be possible that the bug comes up with another round of fuzzing where libmad has debug symbols (I’ll try);
2) An attacker that know this story, can provide a crafted audio which can bypass what fails in the assert, but reach the bad free.
Pingback: SB17-065: Vulnerability Summary for the Week of February 27, 2017 – sec.uno