FFmpeg is a free and open-source software project consisting of a suite of libraries and programs for handling video, audio, and other multimedia files and streams. At its core is the command-line FFmpeg tool itself, designed for processing of video and audio files. It is widely used for format transcoding, basic editing (trimming and concatenation), video scaling, video post-production effects, and standards compliance.

Wikipedia

Streaming Audio and Video Once a Recording Begins

In our first blog post in this series, we provided some detail about how we integrated FFmpeg into a custom Claris FileMaker solution. This article discusses the next step in that solution – streaming the audio and video from the cameras after starting a recording. Once we had the authentication figured out, we then moved to integrating the audio and video streams into FileMaker. 

The Choice to Use AXIS Cameras, with Built-in VAPIX API

The client chose to use cameras from AXIS Communications, a well-known manufacturer of video surveillance solutions, specifically because of the robust VAPIX API built into the devices. The API allowed us to interact with the cameras in a variety of ways, including:

• Remote camera control
• Video streaming
• Event handling, with triggers and notifications
• Image and video retrieval
• Camera configuration 

In some AXIS cameras the API even allows for pan-tilt-zoom control, along with metadata and analytics features.

The M10 box camera style proved a great choice due to its affordability and compact size (smaller than a can of soda). But it’s worth noting there are a ton of tech and security features packed into these little things – even a deep learning processing unit.

Sorting Out That Separate Streaming

Since the cameras are RTSP (Real Time Streaming Protocol) enabled, our first thought was to stream RTSP in a web viewer. But we quickly discovered that RTSP is not supported in web viewers without converting the stream into something the web viewer can read, like an HLS stream.

We went down the rabbit hole of embedding a JavaScript library into the solution, but could never get the RTSP stream to function within FileMaker, even though it worked well in any browser.

Our next thought was to use an external player like VLC to stream the RTSP, but that option gave us a number of new problems to solve, such as ensuring VLC was installed, opening multiple VLC windows for each camera and defining the locations for each window in conjunction with the FileMaker application window.

Ultimately, since we knew that the user wanted to select the audio from one camera during the recording session, we decided to split the audio and video into separate functions. From that point, the scope became much more clear.

FileMaker Web Viewers Handle Video Streams Exceptionally Well

FileMaker web viewers can handle video streams surprisingly well, so we were able to stream multiple cameras at the same time with multiple web viewers. Even at 4k resolution each web viewer handled the video streams exceptionally, so we were comfortable building an interface which displayed all video streams from all cameras at the same time.

We drew our design ideas from the movies. Every movie reference to video surveillance seems to land on some form of grid interface, so we ran with that look as well. (And we may or may not have built an “enhance” easter egg into the solution).

Utilizing the FFplay Media Player Inside the FFmpeg Stream

We were happy with our video process, but we went back to the drawing board for audio. Since RTSP streams for both video and audio in FileMaker were out of the equation, we explored a few other ways to stream the camera audio. We were able to stream audio in FileMaker within another web viewer, but the available layout space was getting a bit cramped. After trying a number of combinations of pop-overs and document windows, we still weren’t fully satisfied with the UI experience.

Then we hit on the question, “What if we could use RTSP, but only for audio?” We discovered we could utilize the FFplay media player inside FFmpeg to handle a single camera’s audio via RTSP. The trick here was to call FFplay via command line and have it stream the audio, but in a hidden background window.

In the end, we were able to create a layered camera interface, appropriately displaying the assigned number of cameras for the selected recording session in progress, while a background service streamed audio from the selected camera. We were delighted to solve that puzzle.

Upcoming: Handling the MP4 Conversion and Video Manipulation

In the next article in this four-part series, we’ll talk about how we figured out how to download video files from each camera, convert those files into a usable format, and then stitch them together into a grid-style video. Then in our final article we’ll review the successes we found during this project and the important lessons we learned.

• PART 1: Capturing Live Video and Audio in FileMaker
• PART 3: Handling the MP4 Conversion and Video Manipulation
• PART 4: Successes Found and Important Lessons Learned

Contact Us

Do you have a sticky problem that you’re trying to solve? We’re great at that. We prioritize simplicity and efficiency, but in smart ways – looking at an issue from lots of angles so we can consider various approaches. We can provide a robust solution that lasts the test of time, but that also has a straightforward framework for future maintenance and improvements. Send us a message below or schedule a 15-minute complimentary consultation.

About the Author

Joe Ranne is a Senior Application Developer at Portage Bay Solutions. Whether on a bicycle in a remote forest or 10 hours into a solution refactor, he always enjoys a good problem to solve.