{"id":587,"date":"2016-04-01T18:53:34","date_gmt":"2016-04-01T18:53:34","guid":{"rendered":"http:\/\/blogs.gentoo.org\/lu_zero\/?p=587"},"modified":"2016-04-01T19:29:23","modified_gmt":"2016-04-01T19:29:23","slug":"avscale-part1","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/lu_zero\/2016\/04\/01\/avscale-part1\/","title":{"rendered":"AVScale – part1"},"content":{"rendered":"

swscale<\/a> is one of the most annoying part of Libav, after a couple of years since the initial blueprint<\/a> we have something almost functional you can play with.<\/p>\n

Colorspace conversion and Scaling<\/h2>\n

Before delving in the library architecture and the outher API probably might be good to make a extra quick summary of what this library is about.<\/p>\n

Most multimedia concepts are more or less intuitive:
\n– encoding<\/strong> is taking some data (e.g. video frames, audio samples) and compress<\/strong> it by leaving out unimportant details
\n– muxing<\/strong> is the act of storing<\/strong> such compressed data and timestamps so that audio and video can play back in sync
\n– demuxing<\/strong> is getting back the compressed data with the timing information stored in the container format
\n– decoding<\/strong> inflates somehow the data so that video frames can be rendered on screen and the audio played on the speakers<\/p>\n

After the decoding step would seem that all the hard work is done, but since there isn’t a single way to store video pixels<\/strong> or audio samples<\/strong> you need to process them so they work with your output devices.<\/p>\n

That process is usually called resampling<\/strong> for audio and for video we have colorspace conversion<\/strong> to change the pixel information and scaling<\/strong> to change the amount of pixels in the image.<\/p>\n

Today I’ll introduce you to the new library for colorspace conversion and scaling we are working on.<\/p>\n

AVScale<\/h2>\n

The library aims to be as simple as possible and hide all the gory details from the user, you won’t need to figure the heads and tails of functions with a quite large<\/a> amount<\/a> of arguments<\/a> nor special-purpose<\/a> functions.<\/p>\n

The API itself is modelled after avresample<\/a> and approaches the problem of conversion and scaling in a way quite different from swscale<\/a>, following the same design of NAScale<\/a>.<\/p>\n

Everything is a Kernel<\/h3>\n

One of the key concept of AVScale<\/strong> is that the conversion chain is assembled out of different components, separating the concerns.<\/p>\n

Those components are called kernels.<\/p>\n

The kernels can be conceptually divided in two kinds:
\n– Conversion<\/strong> kernels, taking an input in a certain format and providing an output in another (e.g. rgb2yuv<\/strong>) without changing any other property.
\n– Process<\/strong> kernels, modifying the data while keeping the format itself unchanged (e.g. scale<\/strong>)<\/p>\n

This pipeline approach gets great flexibility and helps code reuse.<\/p>\n

The most common use-cases (such as scaling without conversion or conversion with out scaling) can be faster than solutions trying to merge together scaling and conversion in a single step.<\/p>\n

API<\/h3>\n

AVScale works with two kind of structures:
\n– AVPixelFormaton<\/strong>: A full description of the pixel format
\n– AVFrame<\/strong>: The frame data, its dimension and a reference to its format details (aka AVPixelFormaton<\/strong>)<\/p>\n

The library will have an AVOption<\/strong>-based system to tune specific options (e.g. selecting the scaling algorithm).<\/p>\n

For now only avscale_config<\/code> and avscale_convert_frame<\/code> are implemented.<\/p>\n

So if the input and output are pre-determined the context can be configured like this:<\/p>\n

\n
AVScaleContext<\/span> *<\/span>ctx<\/span> =<\/span> avscale_alloc_context<\/span>();<\/span>\n\nif<\/span> (<\/span>!<\/span>ctx<\/span>)<\/span>\n    ...<\/span>\n\nret<\/span> =<\/span> avscale_config<\/span>(<\/span>ctx<\/span>,<\/span> out<\/span>,<\/span> in<\/span>);<\/span>\nif<\/span> (<\/span>ret<\/span> <<\/span> 0<\/span>)<\/span>\n    ...<\/span>\n<\/pre>\n<\/div>\n
But you can skip it and scale and\/or convert from a input to an output like this:<\/p>\n
\n
AVScaleContext<\/span> *<\/span>ctx<\/span> =<\/span> avscale_alloc_context<\/span>();<\/span>\n\nif<\/span> (<\/span>!<\/span>ctx<\/span>)<\/span>\n    ...<\/span>\n\nret<\/span> =<\/span> avscale_convert_frame<\/span>(<\/span>ctx<\/span>,<\/span> out<\/span>,<\/span> in<\/span>);<\/span>\nif<\/span> (<\/span>ret<\/span> <<\/span> 0<\/span>)<\/span>\n    ...<\/span>\n\navscale_free<\/span>(<\/span>&<\/span>ctx<\/span>);<\/span>\n<\/pre>\n<\/div>\n
The context gets lazily configured on the first call.<\/p>\n
Notice that avscale_free()<\/code> takes a pointer to a pointer, to make sure the context pointer does not stay dangling.<\/p>\n
As said the API is really<\/em> simple and essential.<\/p>\n
Help welcome!<\/h2>\n
Kostya<\/a> kindly provided an initial proof of concept<\/a> and me, Vittorio and Anton prepared this preview<\/a> on the spare time. There is plenty left to do, if you like the idea (since many<\/strong> kept telling they would love a swscale<\/strong> replacement) we even have a fundraiser<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
swscale is one of the most annoying part of Libav, after a couple of years since the initial blueprint we have something almost functional you can play with. Colorspace conversion and Scaling Before delving in the library architecture and the outher API probably might be good to make a extra quick summary of what this … Continue reading AVScale – part1<\/span><\/a><\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[14],"tags":[19],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1aGWH-9t","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/587"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/comments?post=587"}],"version-history":[{"count":3,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/587\/revisions"}],"predecessor-version":[{"id":590,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/587\/revisions\/590"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/media?parent=587"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/categories?post=587"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/tags?post=587"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}