{"id":591,"date":"2016-10-31T14:24:41","date_gmt":"2016-10-31T14:24:41","guid":{"rendered":"http:\/\/blogs.gentoo.org\/lu_zero\/?p=591"},"modified":"2016-12-14T10:35:11","modified_gmt":"2016-12-14T10:35:11","slug":"intel-mediasdk-mini-walkthrough","status":"publish","type":"post","link":"https:\/\/blogs.gentoo.org\/lu_zero\/2016\/10\/31\/intel-mediasdk-mini-walkthrough\/","title":{"rendered":"Intel MediaSDK mini-walkthrough"},"content":{"rendered":"<h1>Using hwaccel<\/h1>\n<p>Had been a while since I mentioned the topic and we made a <strong>huge<\/strong> progress on this field.<\/p>\n<p>Currently with Libav12 we already have nice support for multiple different hardware support for decoding, scaling, deinterlacing and encoding.<\/p>\n<p>The whole thing works nicely but it isn&#8217;t foolproof yet so I&#8217;ll start describing how to setup and use it for some common tasks.<\/p>\n<p>This post will be about <a href=\"https:\/\/software.intel.com\/media-sdk\">Intel MediaSDK<\/a>, the next post will be about <a href=\"https:\/\/developer.nvidia.com\/nvidia-video-codec-sdk\">NVIDIA Video Codec SDK<\/a>.<\/p>\n<h2>Setup<\/h2>\n<h3>Prerequisites<\/h3>\n<ul>\n<li>A machine with QSV hardware, Haswell, Skylake or better.<\/li>\n<li>The ability to compile your own kernel and modules<\/li>\n<li>The MediaSDK <a href=\"https:\/\/github.com\/lu-zero\/mfx_dispatch\">mfx_dispatch<\/a><\/li>\n<\/ul>\n<p>It works nicely both on Linux and Windows. If you happen to have other platforms feel free to contact Intel and let them know, they&#8217;ll be delighted.<\/p>\n<h3>Installation<\/h3>\n<p>The MediaSDK comes with either the usual Windows setup binary or a Linux bash script that tries its best to install the prerequisites.<\/p>\n<pre><code># tar -xvf MediaServerStudioEssentials2017.tar.gz\nMediaServerStudioEssentials2017\/\nMediaServerStudioEssentials2017\/Intel(R)_Media_Server_Studio_EULA.pdf\nMediaServerStudioEssentials2017\/MediaSamples_Linux_2017.tar.gz\nMediaServerStudioEssentials2017\/intel_sdk_for_opencl_2016_6.2.0.1760_x64.tgz\nMediaServerStudioEssentials2017\/site_license_materials.txt\nMediaServerStudioEssentials2017\/third_party_programs.txt\nMediaServerStudioEssentials2017\/redist.txt\nMediaServerStudioEssentials2017\/FEI2017-16.5.tar.gz\nMediaServerStudioEssentials2017\/SDK2017Production16.5.tar.gz\nMediaServerStudioEssentials2017\/media_server_studio_essentials_release_notes.pdf\n<\/code><\/pre>\n<p>Focus on <code>SDK2017Production16.5.tar.gz<\/code>.<\/p>\n<pre><code>tar -xvf SDK2017Production16.5.tar.gz\nSDK2017Production16.5\/\nSDK2017Production16.5\/Generic\/\nSDK2017Production16.5\/Generic\/intel-opencl-16.5-55964.x86_64.tar.xz.sig\nSDK2017Production16.5\/Generic\/intel-opencl-devel-16.5-55964.x86_64.tar.xz.sig\nSDK2017Production16.5\/Generic\/intel-opencl-devel-16.5-55964.x86_64.tar.xz\nSDK2017Production16.5\/Generic\/intel-linux-media_generic_16.5-55964_64bit.tar.gz\nSDK2017Production16.5\/Generic\/intel-opencl-16.5-55964.x86_64.tar.xz\nSDK2017Production16.5\/Generic\/vpg_ocl_linux_rpmdeb.public\nSDK2017Production16.5\/media_server_studio_getting_started_guide.pdf\nSDK2017Production16.5\/intel-opencl-16.5-release-notes.pdf\nSDK2017Production16.5\/intel-opencl-16.5-installation.pdf\nSDK2017Production16.5\/CentOS\/\nSDK2017Production16.5\/CentOS\/libva-1.67.0.pre1-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/libdrm-devel-2.4.66-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/intel-linux-media-devel-16.5-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/intel-i915-firmware-16.5-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/install_scripts_centos_16.5-55964.tar.gz\nSDK2017Production16.5\/CentOS\/intel-opencl-devel-16.5-55964.x86_64.rpm\nSDK2017Production16.5\/CentOS\/ukmd-kmod-16.5-55964.el7.src.rpm\nSDK2017Production16.5\/CentOS\/libdrm-2.4.66-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/libva-utils-1.67.0.pre1-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/intel-linux-media-16.5-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/kmod-ukmd-16.5-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/intel-opencl-16.5-55964.x86_64.rpm\nSDK2017Production16.5\/CentOS\/libva-devel-1.67.0.pre1-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/drm-utils-2.4.66-55964.el7.x86_64.rpm\nSDK2017Production16.5\/CentOS\/MediaSamples_Linux_bin-16.5-55964.tar.gz\nSDK2017Production16.5\/CentOS\/vpg_ocl_linux_rpmdeb.public\nSDK2017Production16.5\/media_server_studio_sdk_release_notes.pdf\n<\/code><\/pre>\n<h4>Libraries<\/h4>\n<p>The <strong>MediaSDK<\/strong> leverages <em>libva<\/em> to access the hardware together with an highly extended DRI kernel module.<br \/>\nThey support CentOS with rpms and all the other distros with a tarball.<\/p>\n<blockquote><p>\n  <strong>BEWARE<\/strong>: if you use the installer script the custom libva would override your system one, you might not want that.\n<\/p><\/blockquote>\n<p>I&#8217;m using Gentoo so it is <code>intel-linux-media_generic_16.5-55964_64bit.tar.gz<\/code> for me.<\/p>\n<p>The bits of this tarball you <strong>really<\/strong> want to install in the system no matter what is the firmware:<\/p>\n<pre><code>.\/lib\/firmware\/i915\/skl_dmc_ver1_26.bin\n<\/code><\/pre>\n<p>If you are afraid of adding custom stuff on your system I advise to offset the whole installation and then override the LD paths to use that only for Libav.<\/p>\n<blockquote><p>\n  <strong>BEWARE<\/strong>: you must use the custom <strong>iHD<\/strong> libva driver with the custom <strong>i915<\/strong> kernel module.\n<\/p><\/blockquote>\n<p>If you want to install using the provided script on Gentoo you should first <code>emerge lsb-release<\/code>.<\/p>\n<pre><code>emerge lsb-release\nbash install_media.sh\nsource \/etc\/profile.d\/*.sh\necho \/opt\/intel\/mediasdk\/lib64\/ &gt;&gt; \/etc\/ld.so.conf.d\/intel-msdk.conf\nldconfig\n<\/code><\/pre>\n<h4>Kernel Modules<\/h4>\n<p>The patchset resides in:<\/p>\n<pre><code>opt\/intel\/mediasdk\/opensource\/patches\/kmd\/4.4\/intel-kernel-patches.tar.bz2\n<\/code><\/pre>\n<p>The current set is 143 patches against linux 4.4, trying to apply on a more recent kernel requires patience and care.<\/p>\n<p>The 4.4.27 works almost fine (even btrfs does not seem to have many horrible bugs).<\/p>\n<h3>Libav<\/h3>\n<p>In order to use the Media SDK with Libav you should use the <a href=\"https:\/\/github.com\/lu-zero\/mfx_dispatch\">mfx_dispatch<\/a> from yours truly since it provides a default for Linux so it behaves in an uniform way compared to Windows.<\/p>\n<h4>Building the dispatcher<\/h4>\n<p>It is a standard <strong>autotools<\/strong> package.<\/p>\n<pre><code>git clone git:\/\/github.com\/lu-zero\/mfx_dispatch\ncd mfx_dispatch\nautoreconf -ifv\n.\/configure --prefix=\/some\/where\nmake -j 8\nmake install\n<\/code><\/pre>\n<h4>Building Libav<\/h4>\n<p>If you want to use the advanced <strong>hwcontext<\/strong> features on Linux you must enable both the <strong>vaapi<\/strong> and the <strong>mfx<\/strong> support.<\/p>\n<pre><code>git clone git:\/\/github.com\/libav\/libav\ncd libav\nexport PKG_CONFIG_PATH=\/some\/where\/lib\/pkg-config\n.\/configure --enable-libmfx --enable-vaapi --prefix=\/that\/you\/like\nmake -j 8\nmake install\n<\/code><\/pre>\n<h2>Troubleshooting<\/h2>\n<p>Media SDK is sort of temperamental and the setup process requires <a href=\"https:\/\/www.xkcd.com\/1742\/\">manual tweaking<\/a> so the odds of having to do debug and investigate are high.<\/p>\n<p>If something misbehave here is a checklist:<br \/>\n&#8211; Make sure you are using the right kernel and you are loading the module.<\/p>\n<pre><code>uname -a\nlsmod\ndmesg\n<\/code><\/pre>\n<ul>\n<li>Make sure libva is the correct one and it is loading the right thing.<\/li>\n<\/ul>\n<pre><code>vainfo\nstrace -e open .\/avconv -c:v h264_qsv -i test.h264 -f null -\n<\/code><\/pre>\n<ul>\n<li>Make sure you aren&#8217;t using the wrong ratecontrol or not passing all the parameters required<\/li>\n<\/ul>\n<pre><code>.\/avconv -v verbose -filter_complex testsrc -c:v h264_qsv {ratecontrol params omitted} out.mkv\n<\/code><\/pre>\n<p>See below for some examples of working rate-control settings.<br \/>\n&#8211; Use the MediaSDK examples provided with the distribution to confirm that everything works in case the SDK is more recent than the updates.<\/p>\n<h2>Usage<\/h2>\n<p>The Media SDK support in Libav covers <strong>decoding<\/strong>, <strong>encoding<\/strong>, <strong>scaling<\/strong> and <strong>deinterlacing<\/strong>.<\/p>\n<p>Decoding is straightforward, the rest has still quite a bit of rough edges and this blog post had been written mainly to explain them.<\/p>\n<p>Currently the most interesting format supported are <strong>h264<\/strong> and <strong>hevc<\/strong>, but even other formats such as vp8 and vc1 are supported.<\/p>\n<pre><code>.\/avconv -codecs | grep qsv\n<\/code><\/pre>\n<h3>Decoding<\/h3>\n<p>The decoders can output directly to <strong>system memory<\/strong> and can be used as normal decoders and feed a software implementation just fine.<\/p>\n<pre><code>.\/avconv -c:v h264_qsv -i input.h264 -c:v av1 output.mkv\n<\/code><\/pre>\n<p>Or they can decode to opaque (gpu backed) buffers so further processing can happen<\/p>\n<pre><code>.\/avconv -hwaccel qsv -c:v h264_qsv -vf deinterlace_qsv,hwdownload,format=nv12 -c:v x265 output.mov\n<\/code><\/pre>\n<blockquote><p>\n  <strong>NOTICE<\/strong>: you have to explicitly pass the filterchain <code>hwdownload,format=nv12<\/code> not have mysterious failures.\n<\/p><\/blockquote>\n<h3>Encoding<\/h3>\n<p>The encoders are almost as straightforward beside the fact that the MediaSDK provides multiple rate-control systems and they do require explicit parameters to work.<\/p>\n<pre><code>.\/avconv -i input.mkv -c:v h264_qsv -q 20 output.mkv\n<\/code><\/pre>\n<p>Failing to set the nominal framerate or the bitrate would make the look-ahead rate control not happy at all.<\/p>\n<h4>Rate controls<\/h4>\n<p>The rate control is one of the most rough edges of the current MediaSDK support, most of them do require a nominal frame rate and that requires an explicit <code>-r<\/code> to be passed.<\/p>\n<p>There isn&#8217;t a default bitrate so also <code>-b:v<\/code> should be passed if you want to use a rate-control that has a bitrate target.<\/p>\n<p>Is it possible to use a look-ahead rate-control aiming to a quality metric passing <code>-global_quality  -la_depth<\/code>.<\/p>\n<p>The full list is <a href=\"https:\/\/libav.org\/avconv.html#QSV-encoders\">documented<\/a>.<\/p>\n<h3>Transcoding<\/h3>\n<p>It is possible to have a full hardware transcoding pipeline with Media SDK.<\/p>\n<p>Deinterlacing<\/p>\n<pre><code>.\/avconv -hwaccel qsv -c:v h264_qsv -i input.mkv -vf deinterlace_qsv -c:v h264_qsv -r 25 -b:v 2M output.mov\n<\/code><\/pre>\n<p>Scaling<\/p>\n<pre><code>.\/avconv -hwaccel qsv -c:v h264_qsv -i input.mkv -vf scale_qsv=640:480 -c:v h264_qsv -r 25 -b:v 2M -la_depth 10 output.mov\n\n<\/code><\/pre>\n<p>Both at the same time<\/p>\n<pre><code>.\/avconv -hwaccel qsv -c:v h264_qsv -i input.mkv -vf deinterlace_qsv,scale_qsv=640:480 -c:v h264_qsv -r 25 -b:v 2M -la_depth 10 output.mov\n<\/code><\/pre>\n<h3>Hardware filtering caveats<\/h3>\n<p>The hardware filtering system is quite new and introducing it shown a number of shortcomings in the Libavfilter architecture regarding format autonegotiation so for hybrid pipelines (those that do not keep using hardware frames all over) it is necessary to explicitly call for <code>hwupload<\/code> and <code>hwdownload<\/code> explictitly in such ways:<\/p>\n<pre><code>.\/avconv -hwaccel qsv -c:v h264_qsv -i in.mkv -vf deinterlace_qsv,hwdownload,format=nv12 -c:v vp9 out.mkv\n<\/code><\/pre>\n<h2>Future for MediaSDK in Libav<\/h2>\n<p>The Media SDK supports already a good number of interesting codecs (h264, hevc, vp8\/vp9) and Intel seems to be quite receptive regarding what codecs support.<br \/>\nThe Libav support for it will improve over time as we improve the hardware acceleration support in the filtering layer and we make the <strong>libmfx<\/strong> interface richer.<\/p>\n<p>We&#8217;d need more people testing and helping us to figure use-cases and corner-cases that hadn&#8217;t been thought of yet, your feedback is important!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Using hwaccel Had been a while since I mentioned the topic and we made a huge progress on this field. Currently with Libav12 we already have nice support for multiple different hardware support for decoding, scaling, deinterlacing and encoding. The whole thing works nicely but it isn&#8217;t foolproof yet so I&#8217;ll start describing how to &hellip; <a href=\"https:\/\/blogs.gentoo.org\/lu_zero\/2016\/10\/31\/intel-mediasdk-mini-walkthrough\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Intel MediaSDK mini-walkthrough<\/span><\/a><\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[14],"tags":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1aGWH-9x","_links":{"self":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/591"}],"collection":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/comments?post=591"}],"version-history":[{"count":7,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/591\/revisions"}],"predecessor-version":[{"id":601,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/posts\/591\/revisions\/601"}],"wp:attachment":[{"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/media?parent=591"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/categories?post=591"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.gentoo.org\/lu_zero\/wp-json\/wp\/v2\/tags?post=591"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}