Matt Jones details how 7thSense leverages JUCE to power their cutting-edge video hardware running high-end media installations.
7thSense makes hardware boxes for large format media. Do you think of yourselves as a hardware company?
Yes and no. We’re a hybrid, in that we specialize in custom software on custom hardware. In recent times, we do sell hardware without software but traditionally we have always sold hardware with our software.
“Integrated solutions?”
Exactly, yeah.
Ian and Adam, the guys who founded the company, and a lot of us in senior management come from an aircraft simulation background. What Adam and Ian have done is take a lot of the quality visuals required in aviation simulation into the entertainment world.
They were right on the cusp of first using GPUs to do jobs which you would need specialized hardware to do — things like warp and blending projectors. For multiple projectors making up a single image, you have to blend them together so they look like a single image.
They started with planetariums. Then quickly moved into the entertainment world with theme park rides. We’ve always erred on the high-end of the absolute top-quality image. We run uncompressed media.
Do you have different pieces of hardware and software for each of these specialized media industries?
We have a number of different hardware platforms. We run the same software on all of them. The reason for the tight integration between hardware and software is the driver differences and performance. In the early days, the guys offered software-only. But what they found is they were constantly having to debug problems which resided in drivers or hardware.
When you’re at that very high level of quality, people are talking about the difference in 8 and 10-bit color and making sure Superman’s blue is the blue that guys are designing in the studio. These companies want their characters portrayed in the exact way they design them. So that’s why our high end boxes get used.
And because it’s media, it’s also about timing and sync and managing many devices?
Exactly. At a recent large install, we’ve got probably 150, if not more devices in the venue. And the majority of those are talking on a network, synced together, playing media back in sync.
Do you personally get to go on-site to the installations?
Our Head Office is in the UK. We do have an office in Orlando and a satellite office in Burbank, California.
But yeah, I’ve been lucky enough to spend a lot of time on location.
Also, our CEO Bryan Hinckley is based in Pasadena. Bryan used to work for a big integrator who’s actually a big customer of ours.
Integrators tend to be our biggest customer. In the AV industry, most of our business is what we call fixed install. So, theme park rides, visitor attractions, stadiums. They tend to be installed by what we call integrators.
So there’s a collection of companies who will bid for the project. They’re speccing the audio system, the lighting, the health and safety systems and delivering the project as a whole. We constitute a slice of the overall project.
Your role at 7thSense is Software Director. You started out as a software developer, has the company grown a fair bit?
Yeah, it’s grown a lot. When I joined, I think there was eight of us? We’re around 70 now. Multiple offices.
I started in aviation and simulation. The fact that I’d worked with companies in that same sector was why they took a chance on me. So I’ve been a developer, development manager, and the head of software.
I’m still a developer. As is Rich, our CTO. And Adam, one of the founders. They’re all still developers that work on the product.
That’s a dream. Especially for a CTO. I know some CTOs who say “I wish could develop more.”
It comes with its challenges. Finding the time to do it is tough, but I think it’s great that we’re able to pass on that industry knowledge.
When we were small, and I was a developer, I’d write the code for a project, and then I’d be on-site installing it, delivering it, seeing the whole process through. You don’t realize it at the time, but you’re getting exposed to the whole industry from start to beginning.
Do you miss that? The good old days?
A little bit. The quick turnaround we had back then was really fun. Not that it’s not fun these days — we have really exciting projects. That big installation I mentioned was a four-year project.
And we built brand new products. That project allowed us to develop our new software, which is all based on JUCE — Compere and Actor.
Oh, does that mean JUCE is running on the hardware too? I originally thought it was just being used for software control or audio.
All of our GUI is based on JUCE. And our whole backend is based on JUCE.
We don’t utilize the whole framework for everything we’re doing. But our product is compiled and built for a bunch of different platforms: Linux ARM, Linux x86, macOS, Windows. We’re able to do that off the backbone of JUCE.
As we’ve evolved, we’re using things like CMake more, but JUCE was our starting point for cross-platform development. We were Windows-only before we started with JUCE.
JUCE was our starting point for cross-platform development. We were Windows-only before we started with JUCE
—Matt Jones, 7thSense
So there’s software running on the hardware, but then there’s also a cross-platform client talking to the hardware from a computer?
Yeah, it’s a server-client architecture. The GUI is a client. The hardware has a client. And then there’s the server, which is like the dataset owner, controlling everything.
Our video rendering engine is Vulkan, but we have some GUI elements which are JUCE-based.
Do you custom build hardware for that recent big install?
It was development that we were already doing, but it’s fair to say that our roadmap was accelerated.
They were all new products that went in there. A brand new media server, a new pixel processor, Juggler. It’s not PC-based, it’s firmware and FPGA. It’s dedicated hardware with lots of inputs, lots of outputs, picture in picture, being able to process images, splice them together, do processing on it. Not provide the media – because you can imagine a media server has storage, it’s playing back a movie.
The media server might come into a Juggler where you’re doing picture-in-picture. That’s also got JUCE software running. Thats’s an embedded ARM-based platform.
Wow. What’s JUCE doing on the Juggler?
We use the ValueTree for our dataset. So it’s integral to the way we construct our dataset.
That’s really cool. I’ve been talking to a lot of people lately and ValueTrees seem to be at the heart of a lot of different businesses.
We looked at a similar Google protocol…
Yeah, we were sort of getting into JUCE because we wanted a cross-platform GUI and our previous server was Windows. And being in the creative industry, you’ve got a lot of guys who make video on Macs. They go on-site and want to run the frontend on their local machine.
For years it was “Oh, now you have to install Bootcamp, boot into Windows.” So when we started looking at new products, we said “Ok, from the start, we’re going to make it cross-platform.”
So, the latest used Juggler and the new Media Server. Did you do audio too?
We didn’t do the sound. That’s a separate system.
So it’s all about video.
Yeah, a brand-new product going into a very high-profile venue. It’s been a lot of ups and downs but we’re all very proud of what we’ve done and achieved what we did.
There was a lot of brand-new technologies going into that venue. All the media in that venue is 12-bit imagery running at 4k 60fps. That’s all running on a format called SMPTE ST 2110. So that’s uncompressed media over IP, all networked video.
There’s no HDMI with converters anywhere. It’s all super high-bandwidth networking and streamed off a central NAS. There’s crazy networking stuff going on to make sure that happens. And it’s all brand-new technology, leading-edge.
Is that using 10 Gigabit ethernet or something fancier?
Every machine’s got at least 200 Gigabit ports. The bandwidth for the media is about 20 gigs a second, per machine. At any one time, there are dozens of machines serving the media and processing the pixels to drive the one screen.
There’s nothing more leading edge than this, I guess.
It’s an amazing project to work on. We’ve been working with SMPTE, the standard organization, to create new standards. When we first started the project, the standard didn’t exist. No one had ever done 12-bit media in that format. So we had to work with SMPTE to define that video playback standard: ST 2110.
Where can you go from here? I guess this is the biggest project of it’s kind…
Our journey right now is making these new products open to a wider market. A lot of other customers of ours perhaps aren’t ready to go in at the level used at our latest install. They wanted to make sure that when that venue opened, it was still on the cutting edge of technology.
A lot of the work we are doing right now is carving out new workflows for general use.
So the Juggler would be appropriate for any fixed event that need needs to juggle multiple video screens?
Yeah, we’re trying to open ourselves up for more live events as well. The Juggler’s really for anyone who (hence the name) is juggling a lot of inputs, lots of outputs. They’ve got third-party systems playing into them, lots of cameras they’re having to manage, an effects engine. It’s a scalable system where you can stack as many Jugglers as you need. And it’s really fast because of the high-speed bus which connects them all up.
That’s cool. It sounds like work got pretty exciting these last few years.
It’s been a bit of a whirlwind! What’s been great recently is starting to look at our core markets, getting back to what got us here in the first place.
Our old Media Server you programmed a box at a time. It meant the amount of work to program a system scaled with it. A lot of the work now is programming a system, not programming a box. It doesn’t matter how many boxes you have. A little bit of config to set them up…
That’s what JUCE has brought to us. We have a distributed ValueTree. You’re programming it in one place and those updates propagate around our system.
We’re utilizing ZMQ as a transport with JUCE ValueTrees. So it’s really that which is the power of our new products. Moving away from box-by-box programming to programming a system. It’s all in a single frontend.
Thanks for sharing all the technical details!
I’ve enjoyed JUCE. We definitely enjoyed creating a more rich UI. With our previous framework, every UI we created almost felt like you had to make this thing from scratch every time. We found JUCE to be really powerful.
Matt was interviewed by Sudara.
Have comments on this interview? Ideas for who we should talk to next? Let us know on the JUCE forum.