Jake Mills is part of the team that built the latest iteration of the popular Spitfire LABS plugin using WebView and JUCE. We talk about his career path, and the myriad advantages of WebView audio plugins.
So, you said running into JUCE made you change careers?
I think a lot of people go into university and don’t quite know what they’ll end up doing as a role. They just choose a course that’s kind of interesting. I was very big into music, producing stuff, just enjoyed that whole process. I wanted to do something in this industry, but didn’t know what that was.
I initially thought I wanted to go into post-production or live sound. Then I went to university and they introduced us to software engineering almost right off the bat. Like, it was first year, let’s do some C++.
I knew what a plugin was. The thought of making one myself didn’t cross my mind. It was not even a thought that it was possible. I don’t know why.
Was this a music university? What was your degree in?
The degree was Audio Music Tech at Bristol, UWE University.
Wow and they slapped you with C++ as a freshman!
It was very unexpected. I’m obviously super grateful for it. A lot of people saw that and were immediately like “Okay, this is not for me” and just dropped out straight away, which was a bit of a surprise. Because obviously, they do all the other stuff as well. The post-production, the live sound.
It’s a fun way to flush people out! “Are you hardcore enough to tolerate C++?” [laughs]
[laughs] Yeah, I don’t know if that was intended or not. But I think the audio industry is one of those industries that is so niche and so competitive, right? If you want to do live sound, that’s a hard industry to get into. So, I think as a course they have a bit of a responsibility to temper that a bit and be like, “Hey, there’s also this whole other avenue of the audio industry that you probably don’t know.”
Back then, programming wasn’t a subject taught in secondary school / high school. The idea of making a plugin just gelled with me so much.
You are the second person I’ve interviewed that worked with JUCE at university, which blows my mind. What was the course like? Was it JUCE-centric or just a part of the curriculum?
It was clever, the way they did it. They’re not expecting people to come into this course with any software engineering background at all. So the first year, it was fundamentals. What is a variable? What is your basic software engineering stuff?
They did tailor it to audio pretty early on. They gave us this little environment that was obviously written in JUCE, but they hadn’t told us what JUCE was at that stage. It was like, “Here’s a really simple plugin that you’ve got on screen. Here’s how you can build it.” And then you could do something with it. I think it was hooking up some controls that they pre-configured or something like that. So they very gently eased you into it.
At the end it was like, “Oh by the way, all the stuff that you’ve been using is with this framework called JUCE, go learn it.” And lots of people used it for their final year projects.
That’s cool. Do you know other people from your courses that stuck with JUCE or are you the outlier?
I’m definitely not the outlier. It’s pretty common. A few of my university friends are working at audio tech companies. Or they became software engineers for various other companies.
After university did you immediately get a job in audio?
I was pretty lucky. While I was still at university, I got an opportunity to do some freelance work for a very small plugin company. Couple hours a week sort of thing, just in between studies.
I took a year out before doing my final year and did an internship with Digico who do the big mixing consoles. A really good group of people there. Some insanely clever people as well. It’s very intimidating. There wasn’t any JUCE there. Again, a separate industry, not plugins. So it was very different in that regard. I did the internship and then became a full blown software engineer for them for about four years.
That’s a decent amount of time, what kinds of things did you do there?
I was able to do a little bit of everything. It was the start of my career and I just wanted to experience as much as I could. So whether that was the frontend UI stuff on the consoles themselves, or making a build system. Dabbled with a bit of the embedded stuff, but I think very quickly found out that embedded was a bit too low level for me.
What’s it like, dealing with UI on consoles? Do they have their own framework to display UI?
Some use Linux, some use Windows. You’ve configured the OS to just display the application. But then you’ve got a lot of work involved going from the frontend to communicating with some kind of intermediary embedded service that then communicates with an FPGA chip to actually control stuff. The FPGA work especially, I didn’t touch that at all. But I saw it and was utterly mind-blown. I don’t know how those guys write that. Huge respect for them.
Did you go from Digico to Spitfire?
Yeah! I’d done lots of different things and enjoyed it, but it was all wrapped up in hardware and I thought maybe I could try something totally different. “What’s a company that doesn’t have any kind of hardware restrictions. What’s that like?” I was lucky enough to land the position at Spitfire, I’ve been there nearly three years now.
What was the role you applied for? Was it advertised as full-stack or did you sort of make your own role up as you grew there?
Spitfire advertised it as a software engineering role, generally. They were just looking for people that had JUCE experience, people who had made plugins before. I think I’ve stayed fairly true to that role. It’s sort of inherently full-stack to some degree, especially in the world of audio. Everyone’s full-stack to some degree.
Did you get to visit some recording halls where Spitfire does recordings?
We’ve had a couple opportunities to look at how that stuff works. But yeah, it’s very much just the software, which is enough for me, to be honest. There’s enough depth there!
Do you do mainly JUCE work? Only JUCE work?
I guess mainly JUCE. Like a lot of companies that use JUCE, Spitfire kind of made their own audio engine on top of it. They’ve got their own slightly different UI system. It was a lot of that to start with. But it very quickly went on to experimenting with web tech and WebView.
Does Spitfire currently have products out using WebView in JUCE?
Well, we’ve got Air Reverb that is probably one of our biggest unique releases, which uses web technologies for the UI. This was started before we developed our WebView UI system.. Also one of our nicest UIs recently. I didn’t work on it personally, so I can say that without sounding cocky. [laughs]
[laughs] Ah, that’s Spitfire’s first audio effect, right? Do you know anything about the decision to move to WebView? It looks like most of this could have been built in “normal” JUCE as well.
It’s a mix of things. The other product that uses WebView is probably more relevant, because it’s the revamped version of our LABS plugin. That plugin now has all of the content management. So if you’re downloading a LABS pack or installing something or moving it around, that’s all been put into LABS itself. You don’t need to open up a separate app to download new content.
If you go to the LABS website and see all the packs laid out, the copy etc — that’s now very easily translated over and sits in the plugin as well. I think that’s probably one of the main pushes for Web Tech. Not only the technical advantages you get from using tech that’s been around for so long, solely design for making websites, UIs, frontends. You can more easily get parity between a website and a product.
Right, the content is dynamic. You’re in the instrument, you want to keep people there, in the flow. Did you work on UI for LABS?
I was part of the team that brought all of that together. It was a lot of fun. A very small team, but very tight knit. It’s got to be the best team I’ve worked on throughout my career so far, it’s been a really good project.
That’s awesome. Did you keep the sampler engine and redo the UI stuff?
Pretty much. The playback of audio stayed exactly the same. It’s the same plugin with the same IDs. It was very much a replacement of the old UI with WebView. And obviously, there’s all the rehashing of the installation, download management and CMS service for lack of a better word. That was a separate app that got ported over and embedded.
Do you like building frontend in web tech? I built a Vue app the other day and was like “Oh my god, this is so fast to iterate.” Compared with recompiling and so on…
There’s just so many advantages. The speed of development is a complete game changer. That realization of “I’ve spent so much time of my life….” I mean, I’m drinking a lot less coffee now. Like, you know, I go and get a coffee while it builds.
There’s engineers and developers out there that purely do frontend. That’s their whole job. To be able to leverage those people with those experiences to just make a UI super quickly and super eloquently, it’s just another world.
So, definitely the future for plugins, you think?
I’d hope so! I’m not saying every audio app is going to end up being a web UI. With JUCE doubling down on doing their own implementation, it’s going to be a bit of an industry change. I’m sure lots of companies have been experimenting with WebView and dipping their toes in, like “Is this the way to do? Not quite sure. Is it fast enough?” But as soon as a big player like JUCE does it, it’s like, “Ok, this is probably the way to go”.
Do you know what the support load looked like when switching to WebView? I guess it’s also a new product, in a way. But was it a problem when people have a hundred instances of a WebView plugin, have you heard anything around that stuff?
With all products, there’s teething issues. I’ve not heard of anything horrendous with opening up lots of different WebView or anything like that. Like with all products, you release something, you test it as best you can, but it’s impossible to test on every machine unless you are a giant conglomerate of a company, So you have to be pragmatic there a bit. There were definitely some optimizations we had to make, and that’s going to be an ongoing thing.
I was just wondering, because there are WebView cynics. I come from web, I understand all the benefits of hot reloading. But I also understand the reluctance to migrate, which is usually around “it’s new” and “plugins need to be stable” and “web stuff can get bloated, especially if you bring in people who don’t have experience with optimization.” In a WebView-dominant future, do you think there’s any risk of DAWs bloating out?
There’s always going to be risk with any of these things. The main thing, all of this stuff is separate. The audio processing is still happening in C++. Unless you’re sitting there with literally a hundred instances of a web plugin open, with the UI displayed. As soon as you close that down, it’s gone. It doesn’t exist anymore.
Maybe we should get into the weeds. Do you persist the WebView when you close the plugin? I’ve heard of some people doing that, as an optimization for reopening the UI.
We don’t. We obviously persist and cache all the data. But whenever the UI is populated it’s reactive anyway. It’s like “Oh, the UI is open, it needs to fetch this data from the backend.” So it’s all kind of separated out.
I’m sure you guys have had to deal with keypresses on different DAWs, right? Like, re-bubbling up keypresses to the DAW, or suppressing them. Most people doing WebView plugins before JUCE 8 were using CHOC. But you guys rolled your own glue code, etc?
That’s the bit we did ourselves. I didn’t personally work on that, so it’s a little tricky for me to comment. We didn’t know that JUCE was going to roll out improvements to the WebView. So we didn’t do it because we wanted to do it ourselves, necessarily. We did it because we didn’t have an alternative.
We basically ended up doing something very similar. We’ve leveraged the ValueTree system. So we’ve got all of our parameter data, values, value text in a big value tree model. And that gets recreated on the TypeScript side. So we’ve got a similar ValueTree-esque recreation of that data on our frontend. And when you change that value in C++, that gets picked up by TypeScript and that gets picked up by whatever kind of reactivity engine the frontend is using. But you wouldn’t be able to do it the other way around. And that’s a good thing, right? That means your data flow is in the same cycling pattern, all in one direction.
And did you say TypeScript? Do you like TypeScript?
I love TypeScript. As opposed to JavaScript?
Yeah, as opposed to Javascript. Trying to see which side you are on. [laughs]
I’m definitely pro-types. I don’t think you can take a C++ engineer and force them to do JavaScript and not TypeScript. We’re so used to having types and knowing what everything is at all times. It’s very scary to not know what a variable could be.
That’s funny, because there’s this other breed, which I guess I’m a part of! [laughs] C++ is “raw” and “low level” and so of course I want to use the most “raw” and “low level” version of JavaScript — I don’t want to use an abstraction over the real thing!
[laughs] I hadn’t thought of it that way! I’m used to C++ and as much as I love the flexibility of JavaScript, I’m like, “Give me some types please!”
Yeah, I feel that. When I worked on this recent Vue thing, I was like “What is this function even taking? It could be anything.” You are using something reactive on the frontend. Does that mean React or Vue? Does it depend on the project?
I can’t say, but they’re both very similar. I don’t think we’re locked into one or the other, particularly. If our website changes, we will probably do the same because it makes code sharing and team sharing so much easier.
Is all of your parameter glue written for the particular framework?
It’s pretty framework agnostic. Just TypeScript and JavaScript.
What about WebView and accessibility, should we talk about that?
It’s something we’re looking at more now. JUCE has its own accessibility layer. You can do a lot of stuff for accessibility in JUCE. With WebView and HTML, because it is a browser, it’s just all sort of there for you. As long as you write your HTML code correctly and it’s intact and semantically sort of correct, it just works. It’s really surprising. And we can make it even better now because the foundation’s already there.
You know, we’ll have a parameter that is controlling semitones or something and it will have a st suffix. That’s the abbreviation for semitone. A screen reader will see that and be like “ah, you mean ‘street’”. So this is “zero street” or “two street.”
If you look at web standards for accessibility, it’s a big thing. A lot of audio companies have done accessibility well, but with it being a bit of a niche industry, it’s sort of easily forgotten about. It shouldn’t be forgotten about as much as it is. So my hope for any code that I write is that it should be as accessible as it can be, within reason.
While we are on serious topics, I wanted to chat testing. Are you testing frontend too?
Yes. It’s easier to unit test frontend stuff using web tech. You get Jest and all of these little modules and frameworks that make unit testing so much easier. That’s been a big plus.
On the C++ side, I’ve been trying out the paradigm of “lazy tester, lazy developer” where you’re wearing two different hats. You write the most basic simple unit tests you could possible think of for your feature, then write the simplest thing to make that test pass and then very quickly go between to and fro.
When you’re on the testing side, you are like, “Ok, how can I write a test that breaks this functionality.” And then you fix it in the laziest way, go back and forth until you get to a point where you’ve made an if statement that’s a thousand lines long — obviously you don’t want that, so you optimize it.
Funny, that’s related to something I’ve been practicing recently, which is to always plan to throw away the first implementation of a feature. The intention being to learn what the boundaries are before committing to a shape in code. Otherwise I can get a bit of paralysis, too focused on trying to cook up the optimized solution ahead of time before constraints and edge cases are fully known. Similar vibe to get it working, get it nice, get it fast.
Yeah, I think it forces two things really. It forces you to write code that is well tested. But it’s also kind of note taking as well. That’s part of the process. You’re trying to think of those things, get them in a test, fix them, get the next one in the test. It’s definitely changed me as an engineer. There’s an ADC 2024 talk about that exact paradigm that Spitfire is doing called Unit Testing Like a Lazy Pro.
Love this stuff. Testing can sometimes feel like a trade-off in terms of time investment. There’s a temptation to bang it out and move on. Maybe you didn’t even want to be in this code to begin with, or have a deadline to meet. All sorts of pressures that work against being methodical.
Like a lot of devs, I hate writing tests after the fact. I hate having made something really cool and being like, “Oh, I should probably write some tests.” That’s why this process works so well. You’re doing the tests first, but you’re not doing the tests just to do the tests. You’re doing the tests to think about it, write notes and just to have the process.
If you’ve made some gnarly code, or something you are relieved to finish. It’s like “Okay, I don’t have to go back to this for a while.” But you do come back to it and you’ve got this big list of tests. It serves as a bit of a reminder of the process you went through to get to that end. And if you change it, you’ve got loads of tests to make you not feel like you’ve broken everything.
Yeah, those two things are are my favorite things about writing tests. It forces me to be clear while building, and it ends up as documentation. But yeah, writing tests after the fact, that’s like instant tech debt…
Then again if you’ve got a legacy codebase and it doesn’t have tests, it’s still important to go through that. But that can be quite painful. It’s probably more important just to test the new stuff. You could argue the legacy stuff is field tested.
Yeah. So you are testing view stuff, too, that’s interesting…
It was a bit of a learning curve. Because it’s so easy to trigger a click event on a button or something. Is that now an end-to-end test? Not really. It’s still worth testing, especially because frontend changes so quickly. It’s nice to have those backup tests.
I’ve made features in frontend without even looking at them because the tests are all there. It’s a bit of a surreal experience to spend a whole day or two writing loads of code, testing loads of code and then just fire it up at the end. And it’s like, “That feature’s there and it mostly works!”
What kinds of things are you testing? On-screen positions? Or if you give a component some data, you are inspecting the html rendered?
We don’t ever test the positioning or styling of elements, because that stuff changes so quickly, it’s feels not really worth it. The goal is to have any logic tested. Like, whether a reactive element is rendered or not rendered based on a certain property or certain effect. Anything in TypeScript should be unit-tested.
So like, adding something to a list?
Yeah, or you click a button then an overlay opens. We don’t care about where the overlay is or what it looks like. We just want to be sure that the overlay exists at that point. There’s a lot of tests that I have written in web tech that I wouldn’t have bothered doing in C++ because it’s so much easier and quicker to do.
Yeah, in C++ I’ve said no to view testing. I’m religious about unit tests but I saw a friend go down that view path, saw what it took, and I don’t have time for that! I need to work on my plugin! [laughs]
Yeah, make the thing you want to make! [laugh]
Featured photo credit: Ben K. Adams. Jake was interviewed by Sudara, who is building an additive synth with JUCE.
Have comments on this interview? Ideas for who we should talk to next? Let us know on the JUCE forum.