{"id":359,"date":"2014-11-15T13:40:58","date_gmt":"2014-11-15T13:40:58","guid":{"rendered":"http:\/\/blogs.gentoo.org\/lu_zero\/?p=359"},"modified":"2014-12-02T16:46:22","modified_gmt":"2014-12-02T16:46:22","slug":"making-a-new-demuxer","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/lu_zero\/2014\/11\/15\/making-a-new-demuxer\/","title":{"rendered":"Making a new demuxer"},"content":{"rendered":"

Maxim<\/a> asked me to to check a stream from a security camera that he could not decode with avconv<\/a> without forcing<\/a> the format<\/strong> to mjpeg<\/code>.<\/p>\n

Mysterious stream<\/h2>\n

Since it is served as http<\/strong> the first step had been checking the mime<\/strong> type. Time to use curl -I<\/code>.<\/p>\n

# curl -I \"http:\/\/host\/some.cgi?user=admin&amp;pwd=pwd\" | grep Content-Type\n<\/code><\/pre>\n
Interesting enough it is a multipart\/x-mixed-replace<\/code><\/p>\n
Content-Type: multipart\/x-mixed-replace;boundary=object-ipcamera\n<\/code><\/pre>\n
Basically the cgi sends a jpeg<\/strong> images one after the other, we even have a (old and ugly) muxer<\/strong> for it!<\/p>\n
Time to write a demuxer.<\/p>\n
Libav demuxers<\/h2>\n
We already have some documentation<\/a> on how to write a demuxer, but it is not complete so this blogpost will provide an example.<\/p>\n
Basics<\/h3>\n
Libav code is quite object oriented: every component is a C structure<\/strong> containing a description of it and pointers to a set of functions and there are fixed pattern to make easier to make new code fit in.<\/p>\n
Every major library has an all${components}.c<\/code> in which the components are registered to be used. In our case we talk about libavformat<\/strong> so we have allformats.c<\/code>.<\/p>\n
The components are built according to CONFIG_${name}_${component}<\/code> variables generated by configure<\/strong>. The actual code reside in the ${component}<\/code> directory with a pattern such as ${name}.c<\/code> or ${name}dec.c<\/code>\/${name}enc.c<\/code> if both demuxer and muxer are available.<\/p>\n
The code can be split in multiple files if it starts growing to an excess of 500-1000<\/strong> LOCs.<\/p>\n
Registration<\/h4>\n
We have some REGISTER_<\/code> macros that abstract some logic to make every component selectable at configure time since in Libav you can enable\/disable every muxer, demuxer, codec, IO\/protocol from configure<\/strong>.<\/p>\n
We had already have a muxer<\/strong> for the format.<\/p>\n
    REGISTER_MUXER   (MPJPEG,           mpjpeg);\n<\/code><\/pre>\n
Now we register both in a single line:<\/p>\n
    REGISTER_MUXDEMUX(MPJPEG,           mpjpeg);\n<\/code><\/pre>\n
The all${components}<\/code> files are parsed by configure<\/strong> to generate the appropriate Makefile<\/strong> and C<\/strong> definitions. The next run we’ll get a new
\nCONFIG_MPJPEG_DEMUXER<\/code> variable in config.mak<\/code> and config.h<\/code>.<\/p>\n
Now we can add to libavformat\/Makefile<\/code> a line like<\/p>\n
OBJS-$(CONFIG_MPJPEG_DEMUXER)            += mpjpegdec.o\n<\/code><\/pre>\n
and put our mpjpegdec.c<\/code> in libavformat<\/code> and we are ready to write some code!<\/p>\n
Demuxer structure<\/h3>\n
Usually I start putting down a skeleton file with the bare minimum:<\/p>\n
The AVInputFormat<\/a> and the core _read_probe<\/code>, _read_header<\/code> and _read_packet<\/code> callbacks.<\/p>\n
#include \"avformat.h\"\n\nstatic int ${name}_read_probe(AVProbeData *p)\n{\n    return 0;\n}\n\nstatic int ${name}_read_header(AVFormatContext *s)\n{\n    return AVERROR(ENOSYS);\n}\n\nstatic int ${name}_read_packet(AVFormatContext *s, AVPacket *pkt)\n{\n    return AVERROR(ENOSYS);\n}\n\nAVInputFormat ff_${name}_demuxer = {\n    .name           = \"${name}\",\n    .long_name      = NULL_IF_CONFIG_SMALL(\"Longer ${name} description\"),\n    .read_probe     = ${name}_read_probe,\n    .read_header    = ${name}_read_header,\n    .read_packet    = ${name}_read_packet,\n<\/code><\/pre>\n
I make so that all the functions return a no-op<\/strong> value.<\/p>\n
_read_probe<\/h3>\n
This function will be called by the av_probe_input<\/a> functions, it receives some probe information<\/strong> in the form of a buffer<\/strong>. The function return a score<\/strong> between 0 and 100; AVPROBE_SCORE_MAX<\/code>, AVPROBE_SCORE_MIME<\/code> and AVPROBE_SCORE_EXTENSION<\/code> are provided to make more evident what is the expected confidence. 0 means that we are sure that the probed stream is not<\/strong> parsable by this demuxer.<\/p>\n
_read_header<\/h3>\n
This function will be called by avformat_open_input<\/a>. It reads the initial format information (e.g. number and kind of streams) when available, in this function the initial set of streams<\/strong> should be mapped with avformat_new_stream<\/a>. Must return 0<\/code> on success. The skeleton is made to return ENOSYS<\/code> so it can be run and just exit cleanly.<\/p>\n
_read_packet<\/h3>\n
This function will be called by av_read_frame<\/a>. It should return an AVPacket<\/a> containing demuxed data as contained in the bytestream. It will be parsed and collated (or splitted) to a frame-worth amount of data by the optional parsers<\/a>. Must return 0<\/code> on success. The skeleton again returns ENOSYS<\/code>.<\/p>\n
Implementation<\/h2>\n
Now let’s implement the mpjpeg support! The format in itself is quite simple:
\n– a boundary line starting with --<\/code>
\n– a Content-Type<\/code> line stating image\/jpeg<\/code>.
\n– a Content-Length<\/code> line with the actual buffer length.
\n– the jpeg data<\/p>\n
Probe function<\/h3>\n
We just want to check if the Content-Type<\/code> is what we expect basically, so we just
\ngo over the lines (\\n\\r<\/code>-separated) and check if there is a tag<\/strong> Content-Type<\/code> with a value<\/strong> image\/jpeg<\/code>.<\/p>\n
static int get_line(AVIOContext *pb, char *line, int line_size)\n{\n    int i, ch;\n    char *q = line;\n\n    for (i = 0; !pb->eof_reached; i++) {\n        ch = avio_r8(pb);\n        if (ch == 'n') {\n            if (q > line && q[-1] == 'r')\n                q--;\n            *q = '';\n\n            return 0;\n        } else {\n            if ((q - line) < line_size - 1)\n                *q++ = ch;\n        }\n    }\n\n    if (pb->error)\n        return pb->error;\n    return AVERROR_EOF;\n}\n\nstatic int split_tag_value(char **tag, char **value, char *line)\n{\n    char *p = line;\n\n    while (*p != '' && *p != ':')\n        p++;\n    if (*p != ':')\n        return AVERROR_INVALIDDATA;\n\n    *p   = '';\n    *tag = line;\n\n    p++;\n\n    while (av_isspace(*p))\n        p++;\n\n    *value = p;\n\n    return 0;\n}\n\nstatic int check_content_type(char *line)\n{\n    char *tag, *value;\n    int ret = split_tag_value(&tag, &value, line);\n\n    if (ret < 0)\n        return ret;\n\n    if (av_strcasecmp(tag, \"Content-type\") ||\n        av_strcasecmp(value, \"image\/jpeg\"))\n        return AVERROR_INVALIDDATA;\n\n    return 0;\n}\n\nstatic int mpjpeg_read_probe(AVProbeData *p)\n{\n    AVIOContext *pb;\n    char line[128] = { 0 };\n    int ret;\n\n    pb = avio_alloc_context(p->buf, p->buf_size, 0, NULL, NULL, NULL, NULL);\n    if (!pb)\n        return AVERROR(ENOMEM);\n\n    while (!pb->eof_reached) {\n        ret = get_line(pb, line, sizeof(line));\n        if (ret < 0)\n            break;\n\n        ret = check_content_type(line);\n        if (!ret)\n            return AVPROBE_SCORE_MAX;\n    }\n\n    return 0;\n}\n<\/code><\/pre>\n
Here we are using avio<\/a> to be able to reuse get_line<\/code> later.<\/p>\n
Reading the header<\/h3>\n
The format is pretty much header-less, we just check for the boundary for now and
\nset up the minimum amount of information regarding the stream: media type<\/strong>, codec id<\/strong> and frame rate<\/strong>. The boundary by specification is less than 70 characters with --<\/code> as initial marker.<\/p>\n
static int mpjpeg_read_header(AVFormatContext *s)\n{\n    MPJpegContext *mp = s->priv_data;\n    AVStream *st;\n    char boundary[70 + 2 + 1];\n    int ret;\n\n    ret = get_line(s->pb, boundary, sizeof(boundary));\n    if (ret < 0)\n        return ret;\n\n    if (strncmp(boundary, \"--\", 2))\n        return AVERROR_INVALIDDATA;\n\n    st = avformat_new_stream(s, NULL);\n\n    st->codec->codec_type = AVMEDIA_TYPE_VIDEO;\n    st->codec->codec_id   = AV_CODEC_ID_MJPEG;\n\n    avpriv_set_pts_info(st, 60, 1, 25);\n\n    return 0;\n}\n<\/code><\/pre>\n
Reading packets<\/h3>\n
Even this function is quite simple, please note that AVFormatContext<\/code> provides an
\nAVIOContext<\/code>. The bulk of the function boils down to reading the size of the frame,
\nallocate a packet using av_new_packet<\/code> and write down if using avio_read<\/code>.<\/p>\n
static int parse_content_length(char *line)\n{\n    char *tag, *value;\n    int ret = split_tag_value(&tag, &value, line);\n    long int val;\n\n    if (ret < 0)\n        return ret;\n\n    if (av_strcasecmp(tag, \"Content-Length\"))\n        return AVERROR_INVALIDDATA;\n\n    val = strtol(value, NULL, 10);\n    if (val == LONG_MIN || val == LONG_MAX)\n        return AVERROR(errno);\n    if (val > INT_MAX)\n        return AVERROR(ERANGE);\n    return val;\n}\n\nstatic int mpjpeg_read_packet(AVFormatContext *s, AVPacket *pkt)\n{\n    char line[128];\n    int ret, size;\n\n    ret = get_line(s->pb, line, sizeof(line));\n    if (ret < 0)\n        return ret;\n\n    ret = check_content_type(line);\n    if (ret < 0)\n        return ret;\n\n    ret = get_line(s->pb, line, sizeof(line));\n    if (ret < 0)\n        return ret;\n\n    size = parse_content_length(line);\n    if (size < 0)\n        return size;\n\n    ret = get_line(s->pb, line, sizeof(line));\n    if (ret < 0)\n        goto fail;\n\n    ret = av_new_packet(pkt, size);\n    if (ret < 0)\n        return ret;\n\n    ret = avio_read(s->pb, pkt->data, size);\n    if (ret < 0)\n        goto fail;\n\n    ret = get_line(s->pb, line, sizeof(line));\n    if (ret < 0)\n        goto fail;\n\n    \/\/ Consume the boundary marker\n    ret = get_line(s->pb, line, sizeof(line));\n    if (ret < 0)\n        goto fail;\n\n    return ret;\n\nfail:\n    av_free_packet(pkt);\n    return ret;\n}\n\n<\/code><\/pre>\n
What next<\/h2>\n
For now I walked you through on the fundamentals, hopefully next week I’ll show you some additional features I’ll need to implement in this simple demuxer to make it land in Libav: AVOptions<\/a> to make possible overriding the framerate<\/strong> and some additional code to be able to do without Content-Length<\/code> and just use the boundary<\/strong> line.<\/p>\n
\n  PS: wordpress support for syntax highlight is quite subpar, if somebody has a blog engine that can use pygments or equivalent please tell me and I’d switch to it.\n<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"
Maxim asked me to to check a stream from a security camera that he could not decode with avconv without forcing the format to mjpeg. Mysterious stream Since it is served as http the first step had been checking the mime type. Time to use curl -I. # curl -I “http:\/\/host\/some.cgi?user=admin&amp;pwd=pwd” | grep Content-Type Interesting … Continue reading Making a new demuxer<\/span><\/a><\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[14],"tags":[18],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1aGWH-5N","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/359"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/comments?post=359"}],"version-history":[{"count":11,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/359\/revisions"}],"predecessor-version":[{"id":385,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/359\/revisions\/385"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/media?parent=359"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/categories?post=359"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/tags?post=359"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}