Year Two

Live Streaming with Hardware Acceleration using a Raspberry Pi and RTMP/HLS

If you’ve been following my blog post series on the development of my ever so useful cat cam, powered by a Raspberry Pi, you’ll know I’ve made several attempts at a more stable and scalable streaming solution for my Cat Cam. As it stands today, I’ve been using Motion. While it’s a decent tool, Bandwidth has been my primary concern and I’d like to be able to stream real-time without sucking up what measly bits my ISP gives me if more than a few folks decide to show interest.

So far we’ve tried ffmpeg => ffserver and that turned out exactly how you probably thought it would. Next, I tried swapping ffserver with an Nginx-powered RTMP server. While not an entirely fruitless endeavor, there were some blockages that I just couldn’t get past.

I received a suggestion from a colleague to fire up the Raspberry Pi’s hardware encoder/decoder. Up until yesterday, I didn’t know this was a thing. Shame on me for not looking into it. So that’s what we’re going to cover in tonight’s post: taking some of what we learned from our first RTMP attempt and make the hardware do all the work. With any luck, we should see some real perf gains, possibly enough for live streams to start instantly (which would make web players happy).

Since I felt like including it here would deviate from the purpose of this post too much, I wrote up how to Add RTMP Support to Nginx if you installed it via apt-get like me. If you’re in that boat, take a moment to read over that post then come back to this one.

Setting up ffmpeg to use hardware H.264 encoding used to be a fat challenge, but they’ve since added support to the official codebase. If you followed my original ffmpeg post, you’ll have a recent enough version that includes this code, but we’ll still need to compile it.

What we’re looking for this time is the OpenMAX IL (Integration Layer) acceleration module.

pi@pi:/usr/src/ffmpeg $ sudo ./configure --enable-omx --enable-omx-rpi
sudo make
sudo make install

That’ll take some time, as I’ve said before. You’ll have enough free time on your hands to get make something to eat. Come back in an hour or so and it should be done.

NOTE:  If you run into ERROR: OpenMAX IL headers not found, then you’ll need to run apt-get install libomxil-bellagio-bin libomxil-bellagio-dev. Thanks, lordofduct in the comments for that one!

From this point forward, we’ll be starting ffmpeg similarly to how we did it before but with a slightly different codec.

ffmpeg -i /dev/video0 -framerate 30 -video_size 720x404 -vcodec h264_omx -maxrate 768k -bufsize 8080k -vf "format=yuv420p" -g 60 -f flv rtmp://example.com:8081/hls/live

I confirmed VLC is able to play the stream, which is excellent, and there are no lag or jitter issues. It’s about 10-15 seconds behind live, which is totally fine.

I was able to set up an HTML5 player using tools from Bitmovin. I’m not entirely happy with this setup, though, as the player isn’t free and only HLS is supported, right now1. In my next post I’ll cover a new idea that came to mind when looking into the coolness of Ruby on Rails 5: WebSockets.


Also published on Medium.

Discussion

11 responses to ‘Live Streaming with Hardware Acceleration using a Raspberry Pi and RTMP/HLS

  1. It’s only decode with hwaccel
    ffmpeg -decoders | grep mmal
    V….. h264_mmal h264 (mmal) (codec h264)
    V….. mpeg2_mmal mpeg2 (mmal) (codec mpeg2video)
    V….. mpeg4_mmal mpeg4 (mmal) (codec mpeg4)
    V….. vc1_mmal vc1 (mmal) (codec vc1)

    ffmpeg -encoders | grep mmal
    nothing listed

  2. question… I can’t find anything on this anywhere

    But I’m running ubuntu on a raspberry pi, and when I go to configure ffmpeg with the –enable-omx and –enable-omx-rpi options I get the response:

    ERROR: OpenMAX IL headers not found

    In the configure file it’s throwing this because it fails to find the OMX_core.h header… is this not in the repository for ffmpeg? Am I supposed to get it from elsewhere?

  3. Hello,

    I am glad that I can find a thread talking on hardware encoding on RPi.

    I am playing with hls_variant on NGINX+RTMP with FFMPEG.
    This command gives out error. May you please help?
    ffmpeg -i rtmp://127.0.0.1/src/video -framerate 30 -video_size 1280×720 -vcodec h264_omx -maxrate 768k -c:a libfdk_aac -b:a 128k -f flv rtmp://127.0.0.1/hls/video_mid

    Output:
    ffmpeg version N-82785-g6b95da9 Copyright (c) 2000-2016 the FFmpeg developers
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
    configuration: –enable-libfdk-aac –enable-omx –enable-omx-rpi
    libavutil 55. 41.101 / 55. 41.101
    libavcodec 57. 66.109 / 57. 66.109
    libavformat 57. 58.101 / 57. 58.101
    libavdevice 57. 2.100 / 57. 2.100
    libavfilter 6. 68.100 / 6. 68.100
    libswscale 4. 3.101 / 4. 3.101
    libswresample 2. 4.100 / 2. 4.100
    [flv @ 0x2aeb2a0] audio stream discovered after head already parsed
    [flv @ 0x2aeb2a0] video stream discovered after head already parsed
    Input #0, flv, from ‘rtmp://127.0.0.1/src/video’:
    Metadata:
    Server : NGINX RTMP (github.com/arut/nginx-rtmp-module)
    displayWidth : 720
    displayHeight : 1280
    fps : 0
    profile :
    level :
    Duration: 00:00:00.00, start: 1372.178000, bitrate: N/A
    Stream #0:0: Audio: aac (LC), 44100 Hz, stereo, fltp
    Stream #0:1: Video: h264 (Constrained Baseline), yuv420p(progressive), 1280×720, 1k tbr, 1k tbn
    [h264_omx @ 0x2b1c4f0] Using OMX.broadcom.video_encode
    [h264_omx @ 0x2b1c4f0] OMX error 80001000
    [h264_omx @ 0x2b1c4f0] err 80001018 (-2147479528) on line 547
    Stream mapping:
    Stream #0:1 -> #0:0 (h264 (native) -> h264 (h264_omx))
    Stream #0:0 -> #0:1 (aac (native) -> aac (libfdk_aac))
    Error while opening encoder for output stream #0:0 – maybe incorrect parameters such as bit_rate, rate, width or height

    FFMPEG is complied with omx and also libfdk_aac:
    ./configure –enable-libfdk-aac –enable-omx –enable-omx-rpi

    May you please point out my problem?
    Thanks.

    • From what I can see, it doesn’t look like your Pi has enough resources to start up OMX for the encode (See the OMX 80001000 error in red toward the end). I’d suspect one of two things are happening:

      1. The Pi doesn’t have enough RAM left at all, or
      2. Not enough RAM is dedicated to the GPU.

      Take a look at http://raspberrypi.stackexchange.com/a/674 on how to change the amount of ram allotted to your device. Everything I’ve read indicates being able to have enough free RAM (128MB) to set aside is important for camera/video type tasks, even though we’re using the hardware encoder–the video feed has to be piped through something, and the video buffer is that pipe.

Leave a Reply

Your email address will not be published. Required fields are marked *