Intel MediaSDK mini-walkthrough

Using hwaccel

Had been a while since I mentioned the topic and we made a huge progress on this field.

Currently with Libav12 we already have nice support for multiple different hardware support for decoding, scaling, deinterlacing and encoding.

The whole thing works nicely but it isn’t foolproof yet so I’ll start describing how to setup and use it for some common tasks.

This post will be about Intel MediaSDK, the next post will be about NVIDIA Video Codec SDK.

Setup

Prerequisites

  • A machine with QSV hardware, Haswell, Skylake or better.
  • The ability to compile your own kernel and modules
  • The MediaSDK mfx_dispatch

It works nicely both on Linux and Windows. If you happen to have other platforms feel free to contact Intel and let them know, they’ll be delighted.

Installation

The MediaSDK comes with either the usual Windows setup binary or a Linux bash script that tries its best to install the prerequisites.

# tar -xvf MediaServerStudioEssentials2017.tar.gz
MediaServerStudioEssentials2017/
MediaServerStudioEssentials2017/Intel(R)_Media_Server_Studio_EULA.pdf
MediaServerStudioEssentials2017/MediaSamples_Linux_2017.tar.gz
MediaServerStudioEssentials2017/intel_sdk_for_opencl_2016_6.2.0.1760_x64.tgz
MediaServerStudioEssentials2017/site_license_materials.txt
MediaServerStudioEssentials2017/third_party_programs.txt
MediaServerStudioEssentials2017/redist.txt
MediaServerStudioEssentials2017/FEI2017-16.5.tar.gz
MediaServerStudioEssentials2017/SDK2017Production16.5.tar.gz
MediaServerStudioEssentials2017/media_server_studio_essentials_release_notes.pdf

Focus on SDK2017Production16.5.tar.gz.

tar -xvf SDK2017Production16.5.tar.gz
SDK2017Production16.5/
SDK2017Production16.5/Generic/
SDK2017Production16.5/Generic/intel-opencl-16.5-55964.x86_64.tar.xz.sig
SDK2017Production16.5/Generic/intel-opencl-devel-16.5-55964.x86_64.tar.xz.sig
SDK2017Production16.5/Generic/intel-opencl-devel-16.5-55964.x86_64.tar.xz
SDK2017Production16.5/Generic/intel-linux-media_generic_16.5-55964_64bit.tar.gz
SDK2017Production16.5/Generic/intel-opencl-16.5-55964.x86_64.tar.xz
SDK2017Production16.5/Generic/vpg_ocl_linux_rpmdeb.public
SDK2017Production16.5/media_server_studio_getting_started_guide.pdf
SDK2017Production16.5/intel-opencl-16.5-release-notes.pdf
SDK2017Production16.5/intel-opencl-16.5-installation.pdf
SDK2017Production16.5/CentOS/
SDK2017Production16.5/CentOS/libva-1.67.0.pre1-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/libdrm-devel-2.4.66-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/intel-linux-media-devel-16.5-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/intel-i915-firmware-16.5-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/install_scripts_centos_16.5-55964.tar.gz
SDK2017Production16.5/CentOS/intel-opencl-devel-16.5-55964.x86_64.rpm
SDK2017Production16.5/CentOS/ukmd-kmod-16.5-55964.el7.src.rpm
SDK2017Production16.5/CentOS/libdrm-2.4.66-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/libva-utils-1.67.0.pre1-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/intel-linux-media-16.5-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/kmod-ukmd-16.5-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/intel-opencl-16.5-55964.x86_64.rpm
SDK2017Production16.5/CentOS/libva-devel-1.67.0.pre1-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/drm-utils-2.4.66-55964.el7.x86_64.rpm
SDK2017Production16.5/CentOS/MediaSamples_Linux_bin-16.5-55964.tar.gz
SDK2017Production16.5/CentOS/vpg_ocl_linux_rpmdeb.public
SDK2017Production16.5/media_server_studio_sdk_release_notes.pdf

Libraries

The MediaSDK leverages libva to access the hardware together with an highly extended DRI kernel module.
They support CentOS with rpms and all the other distros with a tarball.

BEWARE: if you use the installer script the custom libva would override your system one, you might not want that.

I’m using Gentoo so it is intel-linux-media_generic_16.5-55964_64bit.tar.gz for me.

The bits of this tarball you really want to install in the system no matter what is the firmware:

./lib/firmware/i915/skl_dmc_ver1_26.bin

If you are afraid of adding custom stuff on your system I advise to offset the whole installation and then override the LD paths to use that only for Libav.

BEWARE: you must use the custom iHD libva driver with the custom i915 kernel module.

If you want to install using the provided script on Gentoo you should first emerge lsb-release.

emerge lsb-release
bash install_media.sh
source /etc/profile.d/*.sh
echo /opt/intel/mediasdk/lib64/ >> /etc/ld.so.conf.d/intel-msdk.conf
ldconfig

Kernel Modules

The patchset resides in:

opt/intel/mediasdk/opensource/patches/kmd/4.4/intel-kernel-patches.tar.bz2

The current set is 143 patches against linux 4.4, trying to apply on a more recent kernel requires patience and care.

The 4.4.27 works almost fine (even btrfs does not seem to have many horrible bugs).

Libav

In order to use the Media SDK with Libav you should use the mfx_dispatch from yours truly since it provides a default for Linux so it behaves in an uniform way compared to Windows.

Building the dispatcher

It is a standard autotools package.

git clone git://github.com/lu-zero/mfx_dispatch
cd mfx_dispatch
autoreconf -ifv
./configure --prefix=/some/where
make -j 8
make install

Building Libav

If you want to use the advanced hwcontext features on Linux you must enable both the vaapi and the mfx support.

git clone git://github.com/libav/libav
cd libav
export PKG_CONFIG_PATH=/some/where/lib/pkg-config
./configure --enable-libmfx --enable-vaapi --prefix=/that/you/like
make -j 8
make install

Troubleshooting

Media SDK is sort of temperamental and the setup process requires manual tweaking so the odds of having to do debug and investigate are high.

If something misbehave here is a checklist:
– Make sure you are using the right kernel and you are loading the module.

uname -a
lsmod
dmesg
  • Make sure libva is the correct one and it is loading the right thing.
vainfo
strace -e open ./avconv -c:v h264_qsv -i test.h264 -f null -
  • Make sure you aren’t using the wrong ratecontrol or not passing all the parameters required
./avconv -v verbose -filter_complex testsrc -c:v h264_qsv {ratecontrol params omitted} out.mkv

See below for some examples of working rate-control settings.
– Use the MediaSDK examples provided with the distribution to confirm that everything works in case the SDK is more recent than the updates.

Usage

The Media SDK support in Libav covers decoding, encoding, scaling and deinterlacing.

Decoding is straightforward, the rest has still quite a bit of rough edges and this blog post had been written mainly to explain them.

Currently the most interesting format supported are h264 and hevc, but even other formats such as vp8 and vc1 are supported.

./avconv -codecs | grep qsv

Decoding

The decoders can output directly to system memory and can be used as normal decoders and feed a software implementation just fine.

./avconv -c:v h264_qsv -i input.h264 -c:v av1 output.mkv

Or they can decode to opaque (gpu backed) buffers so further processing can happen

./avconv -hwaccel qsv -c:v h264_qsv -vf deinterlace_qsv,hwdownload,format=nv12 -c:v x265 output.mov

NOTICE: you have to explicitly pass the filterchain hwdownload,format=nv12 not have mysterious failures.

Encoding

The encoders are almost as straightforward beside the fact that the MediaSDK provides multiple rate-control systems and they do require explicit parameters to work.

./avconv -i input.mkv -c:v h264_qsv -q 20 output.mkv

Failing to set the nominal framerate or the bitrate would make the look-ahead rate control not happy at all.

Rate controls

The rate control is one of the most rough edges of the current MediaSDK support, most of them do require a nominal frame rate and that requires an explicit -r to be passed.

There isn’t a default bitrate so also -b:v should be passed if you want to use a rate-control that has a bitrate target.

Is it possible to use a look-ahead rate-control aiming to a quality metric passing -global_quality -la_depth.

The full list is documented.

Transcoding

It is possible to have a full hardware transcoding pipeline with Media SDK.

Deinterlacing

./avconv -hwaccel qsv -c:v h264_qsv -i input.mkv -vf deinterlace_qsv -c:v h264_qsv -r 25 -b:v 2M output.mov

Scaling

./avconv -hwaccel qsv -c:v h264_qsv -i input.mkv -vf scale_qsv=640:480 -c:v h264_qsv -r 25 -b:v 2M -la_depth 10 output.mov

Both at the same time

./avconv -hwaccel qsv -c:v h264_qsv -i input.mkv -vf deinterlace_qsv,scale_qsv=640:480 -c:v h264_qsv -r 25 -b:v 2M -la_depth 10 output.mov

Hardware filtering caveats

The hardware filtering system is quite new and introducing it shown a number of shortcomings in the Libavfilter architecture regarding format autonegotiation so for hybrid pipelines (those that do not keep using hardware frames all over) it is necessary to explicitly call for hwupload and hwdownload explictitly in such ways:

./avconv -hwaccel qsv -c:v h264_qsv -i in.mkv -vf deinterlace_qsv,hwdownload,format=nv12 -c:v vp9 out.mkv

Future for MediaSDK in Libav

The Media SDK supports already a good number of interesting codecs (h264, hevc, vp8/vp9) and Intel seems to be quite receptive regarding what codecs support.
The Libav support for it will improve over time as we improve the hardware acceleration support in the filtering layer and we make the libmfx interface richer.

We’d need more people testing and helping us to figure use-cases and corner-cases that hadn’t been thought of yet, your feedback is important!

One thought on “Intel MediaSDK mini-walkthrough”

Leave a Reply

Your email address will not be published. Required fields are marked *