Baconpaul from the Surge Synthesizer Team

"It's fun working on stuff that works!" Best known as a member of the Surge Synthesizer Team, Baconpaul is an open source force of nature. He sits down with the JUCE team to chat microtonality, accessibility and open source, such as the new audio plugin standard CLAP.

To some degree, you are like audio development's Batman. Showed up out of nowhere to clean up Gotham. Nobody knows your background, what your motivations are, if it's a deeply sad tragedy...

My parents were killed by a falling Jupiter 8, right?

[laughs]

I'm semi-retired and was looking for projects which combined programming, music, and giving something away to the music community.

I've been playing music my whole life and decided to start screwing around with audio software a bit more more seriously. My wife got me a Roli Seaboard for my birthday. This was in 2018, the same time that Claes [Bitwig cofounder and lead developer] had open sourced Surge, a synth he sold commercially in the 2000s and early 2010s. One of the features in Surge that was coming was MPE support. I go to download the source. I'm like "I should be able to build this thing? How hard is it? You press build." It didn't build at all, so I ignored it. Played the Roli for a little.

Then Thanksgiving that year came around. I'm like "I should try building that synth again, see if anything's happened on it." A bunch of people were saying "Why doesn't it build on Mac? Where's the Mac binary?" I downloaded it and I'm like "Oh, it's missing this, it's missing that." I got it working on Mac. Mostly by looking at what Dexed and a bunch of other open source synths were doing. This is before it was JUCE. It was VSTGUI. Got it to make sound in Logic. Sent in a pull request: "This makes it work on Mac, I think."

Claes' response was to press merge, go into settings, go to users, find my user ID, add me as an admin and then send me an email saying "Good Luck."

[laughs] That is amazing. I like to do that with my open source projects, too. The minute someone contributes something decent, it's like "it's yours now, too!"

But you know, that's a very me-centric story. That's how I got involved in Surge. But there were a group of people kicking around who loved the synth. And so immediately there were 7 or 8 people working on it in various ways.

One person (who's much less active in the community now) was a professional tester. A software tester by trade. He wrangled all the bug reports from KVR, wrote them into excellent bug reports on GitHub. I realized right away that this could not only be a fun way to spend some time, but also a little secret project, which is can you actually build a sort of unmanaged anarchist collective that makes something useful? The consciously no-ones-in-charge part of it was appealing to me.

You don't want to be characterized as the leader of the project. But by contribution volume aren't you kind of pulling everything along?

I joke about this on our Discord all the time. Where I apply the Invisible Hand Of How It Should Be. Like, "maybe lets do it this way. No, like, really, lets maybe do it this way."

But let me push back on that a little. I'm not an instrument designer. I'm not a particularly good tester. I'm not a particularly good user interface person. I'm not a particularly good documentation person. I used to not know anything about microtuning and microtonality. I'd never written an accessible app before.

50 people have commits on Surge now. We've added Open Sound Control to Surge 1.3 and I wrote exactly none of that code. Microtonality is another great example. Someone showed up and said it would be great if Surge was microtonal and I'm like "Oh, that's always been something I wanted to figure out" but I didn't know anything about it. And so the group of us figured that out.

Baconpaul's github contribution graph.
Baconpaul's countless contributions to countless projects.

This is one of the real myths of software development: Typing in C++ is not the project. Shipping a thing that brings people joy is the project. And that is a many-faceted project.

What did you guys go with for microtonal stuff? Was it MTS-ESP or a combination of that and tuning files?

On the day that MTS-ESP was announced, we were one of the 5 synths that supported it.

We had microtonal support for a while before that. We did SCL KBM and in the course of doing that me and Jacky Ligon (who is one of collaborators who debugged a lot of it with me) realized we had a piece of software in Surge that could be used elsewhere. So we factored that into a library. That library is now running in Dexed, it's running in Odin, it's running in Vital. It's running in Decent Sampler. [Editor's note: See our interview with David Hilowitz]. So we Johnny Appleseeded SCL KBM support.

If you were in my boat and writing a new synth, what would you be looking to support these days?

I think a bare minimum is MTS-ESP. Both note on and continuous tuning with channel awareness. If you do that, almost every serious microtonal composer will be happy with you.

Here's the thing. Tuning an oscillator is easy. Tuning an instrument is hard. Here's one of the questions I often ask someone: "What does the pitch bend wheel do in a microtonal context?"

Those questions have no answers...

You have Pitch Bend Depth set to 2. You press C and move your wheel all the way up. Do you expect two semitones of change? Do you expect the note to be sounding the note you would get if you pressed the D key?

[laughs] Yes, that is the question!

So Surge can do both. We think of it as "do you tune the instrument at the MIDI keyboard or do you tune the instrument after the modulation has run?" What does a 1 key LFO do? Is it one semitone or is it moving up and down the keyboard by one?

Lots of interesting questions!

If you are writing a synth, you should do MTS-ESP support, drop it in the #tuning channel on the Surge Discord and you'll find a million people to test it for you.

Deal! Thanks! Let's talk about the open source audio development landscape. What inspires you?

One of the best critiques I've heard of open source audio development — I don't have a great answer to this other than to try to allow it to influence my work — if open source audio development is great and relieved of an ROI constraint, why isn't there more super weird creative stuff that everyone loves? If you take a look at Bespoke, I think it's a really interesting sort of quasi-DAW. I don't think it would have emerged from a commercial closed source process.

That's part of what we try to do with the Surge project. We do things that don't make economic sense, because we can. And then we see what happens. We take advantage of and contribute back to open source.

We're increasingly making all of our software available as libraries as opposed to a monolithic plugin. We're doing Short Circuit right now which is a project to rebuild a sampler. I'm not going to copy those effects from Surge to Short Circuit. I'm going to move them from Surge to a common library and then use them in both projects.

A screenshot of Surge XT Effects with the Airwindows Deb Center effect loaded.
The dozens of effects available in Surge are also available standalone

A mad-genius open source person allowing people to do crazy and wonderful stuff is Chris Johnson of Airwindows. I've collaborated with him for a while now, including bringing all his plugins into a single consolidated JUCE plugin.

Another great example of open source is Don Cross (CosineKitty) in the Rack ecosystem who is also a retired science person. He's writing these crazy physical modeling modules. You know, a 9 by 17 hexagonal grid of springs and masses with damping in a liquid. And here's what happens when you hit it on one side.

Ooh, give it to me, I want it! [laughs]

Exactly. They're fantastic! Rack being open source helped make it possible. But you know, let's try some crazy stuff in the ecosystem and see what happens. That's sort of one of my thrusts.

The problem you often have with open source is if you need to get paid to do it, it's very hard to get paid to do it. If you don't need to get paid to do it, you're doing it as a hobby, so that limits the number of people who get the opportunity to do it. So I definitely acknowledge and appreciate the privileged situation those of us doing it are in.

I want to peer into that a little bit more. To release a cross-platform plugin, even something trivial, you have a fair amount of work to do. There are attempts to shorten that work. But you need to know about a lot of things from code signing to DSP fundamentals. Even knowing what you should know can take months.

There's this really great open source project called Pamplejuce that gets you started, have you heard of it?

[laughs] The reason I say that is because I also feel privileged just to have the bandwidth to make the attempt...

There is a mass of activation energy to get started that I think is challenging. But I think projects like the one you are working on, projects like the one I'm working on, and frankly projects like JUCE (which I think is a high-quality piece of software) lower that activation energy.

But still not to zero. If you don't know what a compiler is and don't know what a DLL is and don't know what code signing is, you cannot write an instrument for a digital workstation.

I do feel like one contributing factor is the size of the audio community. It's not quite big enough to have mature open source projects with hundreds of contributors. Quality can vary due to sheer numbers... Surge is an outlier, right?

Plugins are sub-scale luxury goods, right? Or very particular professional goods.

I think we ship a pretty high quality product. I think it's a pretty good synth, despite you know, the more you know a piece of software the more you hate it. Despite that, I think it's actually pretty good.

The quality doesn't come from me writing bug-free code. The quality comes from a community of engaged individuals who make sure that we not only implement, but also design and test things. Good testers make good software.

I like that. I wanted to probe a bit about Bitwig, CLAP, Surge — it's like there's this whole big happy family over there.

Before I was doing audio, I spent a lot of time thinking about open source. The team at Bitwig, the team at u-he and a few of the rest of us thought that it was time to have a proper open standard. And by proper open, I mean reasonably well governed, licensed permissively, etc. I think we met those goals. No-one from u-he can unilaterally push a change to CLAP. There's a review process and we're inviting new people into that review process as the community grows.

It's fun working on stuff that works!

[laughs] That might have to be your pull quote!

It's fun working on stuff that works. There were a couple big changes to get to CLAP 1.0. A group of us were really collaborating on what this thing could be, right? We had a long discussion about note IDs vs. channel based note addressing and things like that. Real concrete discussion that changed the design of the platform. And we were able to work it out because both the CLAP community and the Surge community are just based on an idea of "it's fun to work on stuff that works."

It's now in Bitwig. It's in Reaper. It's in Fruity Loops. It's in a bunch of open source DAWs. On Surge Discord today, a dev for LMMS said they are working on CLAP support. I think there's going to be some other DAWs and plugins this year. [Editors note: Baconpaul also authored the unofficial CLAP plugin support for JUCE]

Let's talk about accessibility.

Surge didn't used to be a JUCE application. It used to be a VSTGUI application. One day I decided that was no longer tenable for a variety of reasons. I spent February of 2021 porting it to JUCE. One of the people who had worked on Surge was asking — they didn't have a visual impairment but they saw that the future may hold some diminished vision — if we had thought about this at all.

I actually even thought about a fork of VSTGUI to add support for NSAccessibility and realized that was ludicrously difficult. And then the JUCE team came along and did it. I think they did a nice job with it, by the way. I think the JUCE accessibility support is really really first-rate.

If people are reading this interview, the one thing I would like them to understand the most is just get a blind person testing your software and giving you feedback. You can test the accessibility features yourself but you will not realize when you accidentally crutch yourself to the mouse or the UI.

Getting reliable testers is hard already! How does one go about that?

There's a group associated with Surge who I'm sure would volunteer to give your software a test. Pop into the accessibility help channel in the Surge discord.

The other thing I would say about accessibility is that people are interacting with your application using the keyboard, not the mouse. And using spoken word, not visual display. It is absolutely critical that you think about that experience — keyboard plus spoken — as a user interface. We don't call it the UI and the screen reader. We call it the sighted UI and the spoken UI.

A screenshot of Surge synthesizer's graphical user interface.
Surge's sighted UI

All of a sudden you realize things that you never cared about at all, like tab order, matter a lot. You want to make sure that the tab key goes to the appropriate next control, the shift tab keys goes back. You want to make sure things are grouped properly, because there's group navigation in all these things. You want to make sure you can edit a value with the keyboard. Slider is responding to up arrow, down arrow, shift up/down arrow, home, end, delete.

And this is really important: You need to make sure that the spoken names of your controls make sense. I have no idea how you're going to get this into a written transcript, but it's a funny story: I get "lately bass 🐟" not "lately bass 🎸."

If you're not running voiceover and listening to your application and trying it with the keyboard then you're not actually doing it.

Now that I've done all the keyboard work to make it accessible, I use the keyboard all the time. Reaper is the main accessible DAW people use but Live 12 has a lot of accessibility in it as well, so people are getting pretty excited about these. And you need to test in those DAWs.

A photo of VoiceOver running in Surge.
VoiceOver Running in Surge. Starting VoiceOver on macOS is as easy as pushing cmd-F5.

I love navigating with my keyboard. But that's a different thing from accessible actions, they are two different UIs, right?

In the world of JUCE, there are three things: You need a KeyboardFocusTraverser, which is how you set your tab order. You need a key press handler. And you need an AccessibilityHandler to advertise the spoken name.

The keyboard has to trigger something as well as the accessibility action — is one triggering the other?

You can take a look at the Surge code. The keypress handler looks to see if it's an accessible bound action and then calls a method and the accessible bound action calls the same method. So, we bind SHIFT-F10 to be "Show Menu" in all of our sliders, but we also bind the accessible "Show Menu" action, so most of our things have a method called "Show Menu" that both paths call.

Surge is a big UI, so making it accessible was actually rather difficult. Accessibility for something like a virtual Minimoog would be a lot easier, you just tab across the knobs.

Yeah, my synth is a bit of a nightmare, it's on multiple screens, there's modals...

That might be hard. There is one feature in Surge we haven't made accessible yet, which is our MSEG editor. It's a purely visual editor right now. The plan I have for that is make a spreadsheet view, basically.

One of the things I learned is that if you have the "real UI" and the "accessible UI," the accessible UI will suffer and the community will rightly ask why you made that decision. So the more that you can have the UI be the UI and there's the sighted interface and the spoken and keyboard interface — the more you get into that mindset, the more successful you'll be.

Surge is by no means close to one of the better plugins in the world. But there's a couple niches where I think it does really well. Microtonality and accessibility are two of them. Shipping one of the most popular synths in the world with blind musicians is a cool thing to have done!

We collaborated on adding some accessibility inspection to Melatonin Inspector. There's some other way of inspecting accessibility on macOS.... what's it called?...

There's something called Accessibility Inspector. There are times when you will need to use that. I think the goal for Melatonin Inspector should be to reduce 80% of those uses, which is kind of what we put in, right?

A screenshot of the Melatonin Inspector JUCE module inspecting Surge's graphical user interface.
Inspecting Surge Accessibility with Melatonin Inspector (thanks to Baconpaul's contribution)

As a newcomer to accessibility, I also felt the need to see which accessibility actions are defined. Also, is the help text used by the community?

I have never had anyone ask me to put the help text in place.

The actions some people use, but on Windows especially, the actions traversal is nowhere near as useful as the key bindings. NVDA and JAWS don't seem to use it in the same way you'd expect. I originally thought you would have to support all the actions as well, but that doesn't seem to be the path people are using.

What is the path being used?

People are pressing tab, tab, tab, listening to what it says, and pressing the up and down arrow. There are value set methods, but as far as I can tell, one of the big Windows screen readers doesn't really use them.

Alright! Well, thanks so much for your time.

This is going to be a nightmare for you to turn into a coherent interview.

This is how it goes! Only 10% will make it in.

One thing I definitely want put in is that it's a real privilege to be able to work on Surge with the team of people I'm working on it with.


Download Surge and try adding CLAP support to your JUCE project. Baconpaul was interviewed by Sudara who is (still) building an additive synth.

Have comments on this interview? Ideas for who we should talk to next? Let us know on the JUCE forum

More "Made With JUCE"

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram