A master’s from CCRMA. A PhD in Computer Music from UCSD. Performing artist at Coachella. Senior Engineer at Output. We interview Jennifer Hsu about her multifaceted career.
I want to hear about everything! Maybe let’s start with how you found your way into the audio industry?
I started playing piano when I was a kid. When I was in the eighth grade, my brother showed me how to play guitar chords. I went crazy and played a bunch of guitar and recorded 20 layers of myself in Audacity every night through high school, trying to make covers of songs that I liked.
But my parents were like, “You are not doing music in life.” So I went to UC Berkeley for undergrad. And then I was like, “Okay, well, CS is fun because I can build anything with it”. So I double majored in that and cognitive science. But I took music classes when I could.
One class I took was at UC Berkeley’s CNMAT, where they teach you how to use Max/MSP. It was the first time I used the Fourier transform. When it was time to graduate, I didn’t want to get a job in the industry yet, so I was like, “I’ll just apply to grad schools.” I thought I wanted to do music and psychology. So I applied to CCRMA. I got my acceptance letter on Valentine’s Day and I was like, “Oh, what a love story.”
I didn’t know about CCRMA back when I was going through school, but ever since I learned about it, I’ve felt “Oh, this is where I wish I ended up!” I love that place.
It’s amazing.
At CCRMA the professors are always like, “Programming’s the easy part, music’s the hard part.”
Musical instrument practice is really hard! It takes so much discipline. Tougher than messing around on a computer. [laughs]
Yeah, and growing up, I was always taught that there’s no value in music, because it doesn’t make you money. It’s weird I was taught that, because I think there’s a lot of value in art and music.
Totally. So, Stanford was cool?
Yeah, that was where I started learning how to program plugins. I made a friend and he was like, “I’m going to teach you how to produce, since you’re making these plugins.” So we started doing collabs together.
Then I ended up working at Dolby for an internship for like a year and a half, I think.
Oh, nice!
Yeah, one of the professors at CCRMA was like, “Come work for us in SF.” That was more psychoacoustics kind of stuff. Doing experiments with people in order to inform the technology they were making at the time.
I was at Stanford for two years and then decided to apply to some PhD programs. At that point, I still thought I wanted to research music and emotions. I got into UCSD, which is a huge change from the Bay Area. More slow, you know? It was great though.
I started taking classes on designing studies for experiments on people, and I decided that I just didn’t really enjoy this kind of research.
Oh, was it just not interesting enough?
I didn’t like designing experiments. And on the other side, I was working for a professor and we were working on generating music that sounds like a certain style. I was also producing, making a lot of music, and I just felt like the things we were making from the research was taking away from what I love about making music, which comes from your experiences.
So I started talking to some of the professors there, like Tamara Smyth, Miller Puckett and Tom Erbe.
Wow, casually name dropping the heavy hitters!
They’re amazing! They’re really good with advising students. They got me into synthesis and physical modeling. Definitely learned a ton from them.
Speaking of, what does your PhD topic mean? “Percussion synthesis, physically-informed”… What were you researching?
I looked through all these physical modeling papers and implemented them. I’m not very good at reading a paper and understanding what’s going on. I have to re-implement it and it takes time. That’s when I finally understand.
So I was doing that for all these cool percussion synthesis papers. I was talking to Miller and he’s like, you know, cymbals are really cool, non-linear sounds are really cool. So I was like, “Ok, let’s look into it! What is this? Why is it cool?” He had recorded all of these cymbal hits for me, so he’s like, let’s analyze these and look at how crazy those sounds actually are.
All of these high frequencies get generated when you hit a cymbal or gong. There’s that big swell of sound at the end. So the PhD thesis started from that. Miller also showed me a different kind of feedback FM that we call loopback FM where something kind of loops back and creates a bunch of nonlinearities in the signal. And you don’t have to use as much computing power as physical modeling. It’s not that easy to control, so we were trying to parameterize it to make it easy to play with.
So yeah, it was taming loopback FM, in a way!
Is loopback FM for cymbal stuff, or all kinds of percussion stuff?
We made really artistic sounds, like non-realistic. There’s some examples where it just sounds like crazy metallic sounds. I had this sound library a friend gave me that had all of these metal banging sounds. I was like “I want to make cool ones like that.” So there are some sounds I made that were more metallic sounding. I still don’t understand what’s going on in some of them because they have a cool swell and a weird sound.
That’s cool. So you got your PhD, spat out into the world, and then you worked at places like Goodhertz?
I did internships every summer in-between school. I worked for Jawbone. They did Bluetooth speakers and headsets. They just let me do a research project where I studied people and how they responded to music (I was still into that at this point).
I did another one where I got to model heads and do HRTFs. And then I worked at Goodhertz for a summer. I just emailed them and they were like “Let’s talk!” I met them at a coffee shop in LA and they had me work on Megaverb.
A great piece of software! Those guys are part of the reason why I got into the industry.
Yeah, I love them. They’re a nice small team. I learned a lot from Devin Kerr. Actually, my partner just bought me the Chase Bliss Lossy pedal for my birthday!
I love Lossy. It’s one of my favorite effects.
Yeah, I like it! It’s taking all the great stuff of digital audio and bringing it out, you know?
I also interviewed Arden who made an open-source effect with actual MP3 encoders in there. I’m all about it. It’s time to romanticize the 90s!
I was thinking about those little Walkman headphones from back in the 90s.
I definitely had those! So, during this time, you were also making music as Black Jade?
I didn’t start doing that until 2017. I was making music but I didn’t put myself out there as much. I had a breakup in 2016 where I was like, “Nothing matters anymore!” and then I started posting music on my social media. I met this whole group of people in San Diego that make music. And they started inviting me out to play shows and collab with them. I’d rent out the music studios at UCSD and we’d all get together weekly.
Sounds really cool. And you played Coachella recently?!
Yes! That was after I came to LA! I moved here and then COVID happened. One of my buddies here at Output invited me to play a backyard party with all of his friends from the beat scene. That was when I was saying yes to everything…
I played it, and I met a bunch of people in the scene. I played a few shows, met more people, and now I’m a resident with Beat Cinema. They’re the ones who got me to play at Coachella. So big thank you to them!
Admirable! There’s such a big difference between doing something like that and juggling circular buffers.
I was trying to practice singing, piano, guitar, everything. And I was doing it after work and had this whole engineering mindset. It was a bad idea. I didn’t want to do practice anymore. So I switched it up. All music stuff is going to be about emotion — releasing and feeling free. And engineering daytime work is this straightforward thing. Otherwise, I was burning myself out.
So music making is not puzzle solving, it’s expressing.
Yeah!
I like that. So how did you get started at Output?
When I was graduating at UCSD, I wrote this list about everything I wanted in a job. It was like, be in LA, make this much, make tools for people to make music (because that’s what I believe in). And then I got a message from Spencer Salazar. Spencer was my TA at CCRMA and he messaged me like, “Hey, I work at Output. Do you want to come check out the office next time you’re in LA?”
So I came over to Output and hung out with Spencer for a day, met the whole team. And he was telling me about how much fun it is to work here. And I was like, “Okay, is this my interview or something?”
Those are the best kinds of interviews!
It was really, really cool. I’ve never had an experience where it was so simple and natural. So then I did a one-month contract at Output and went back to teach at UCSD for a summer and got hired for a full-time position after that. I’ve been here since, living in LA.
What were you teaching? I saw you did some psychoacoustics?
I did TA for some psychoacoustics courses. Some mixing, mastering, recording stuff. And then for teaching a full-on class, it was production in Ableton.
Prep must be intense for that. There’s an artistic element involved plus all the technical stuff.
I had TA’d that course maybe two times before actually teaching it. So I picked from what those professors taught. One of them went through a history of electronic music. It was about how we’re really shaped by recorded music, which only started a little over 100 years now. I think it was people who worked with film who were the first electronic musicians because they would cut up the stuff and play around with it.
So there was that and then I had also done some online artist building programs from this musician called Dot. She had a lot of cool projects to get people to make music. My favorite one is you take a song you like and recreate it as closely as possible, recreate 16 bars—
Ooh, that’s up my alley.
And then you take what you made and kind of flip it, remix it into your own. So that was the final project for them, because I thought that was how you learn the most.
That’s awesome. What role does working on music tools play for you? Is it the closest thing to music, and that’s why it’s cool? Or because you can have impact that goes beyond what you yourself are producing? Or is it about exploring the unknown?
There’s multiple things.
On one side side, it’s like a puzzle and I like puzzles a lot. I like room escape games. So that part of it is a fun challenge for that side of my brain.
On the other side, I experience a lot of joy when I make music. When I don’t make music, I get jealous of others and a little bitter. And I feel like that’s because of what I need to be doing — actually being creative and doing what I want. So I’d like others to also feel that joy that comes from creating music, and making music tools is a way to help that happen.
Output put out this tool recently, Co-Producer. One of the first subscribers wrote into the company and it was really touching. He just said he loves using these tools and it brightens up his day. And I was like, “Oh, that’s a reminder that I’m working on software that actually helps people feel better during the day.” It’s rewarding.
What have you been up to at Output? Are you cooking up algorithms?
Have you played with Arcade at all?
A bit!
I got to make the oscillators for Instruments mode, which is where you actually play the chords in.
I feel like I’m just tinkering with things to get them to work. They’ll be like, “I have this idea!” And then I get to do the audio stuff for it and mess around with it until it works.
I also got to work on the arpeggiator in Arcade. I used a JUCE tutorial for that in the beginning!
Do you basically work on any DSP topic that needs picking up?
Usually the product team has an idea and then they’re like, “Can you get this there?” Sometimes I’ve coded it 10 times, so it’s easy for me to do it right there. Sometimes I use Python and plot stuff out in there. Like, what should this clipper look like? Why is this filter blowing up?
It sounds pretty varied.
It’s really cool. Two years ago they actually had me work on some web UI code. And I was pretty upset about that at first. But a few months in, it started to feel very comfortable, and I’m grateful for it. So I added a whole new set of skills.
How has your role changed over the years? You are a Senior Software Engineer now but you started pre-COVID, right?
I touch more things in the code now. I understand a lot more about how to work in real-time software, how to optimize code, and how to fix certain types of bugs.
And I’ve worked with multi-threading a bit more, so I am feeling more comfortable with that these days.
I don’t know if there’s a single dev who feels all the way comfortable with threading! Even the people giving presentations are like, “We don’t really know…” [laughs]
It’s so funny.
We’re working on upcoming stuff and the architecture is very different from older products. The engineers that designed the architecture of this new project did a really great job with it. It’s much easier to work with. All the state is saved in one place, you can just go grab it. It’s safer and a lot easier to debug. So I’ve been learning about that kind of stuff while working here.
It sounds pretty collaborative there.
Yeah, it’s nice. They ask me to do videos for them sometimes too. That’s totally different from what I usually do on a day-to-day basis, it’s a lot of fun. Challenging in a different way.
Yeah! I saw that one. You guys do a lot of Web UI there, right?
Yeah. Arthur and Colin and Kevin all gave presentations about it [at ADC 2024]. Spencer too. Some of the JUCE graphics we do use at well, but there’s a web layer.
For DSP, you have your own libraries, I assume? Do you interact with JUCE dsp stuff at all?
I think we use some of JUCE’s audio buffer classes. But we don’t use any of the synthesis stuff. We use JUCE to get audio software to work across different platforms.
Actually, my manager had me take on a mentee recently. She’s worked on music for Charli XCX and everything, big stuff. But she wants to get into audio dsp, because she has VST ideas. So I told her to start with the JUCE tutorials. We just met yesterday and she did the sine wave tutorial, and she’s going to do a plugin too.
Nice! That reminds me, should we discuss being a woman in audio? It’s an important topic, but I didn’t want to frame the whole interview around it. Did you have role models when you were in the field?
When I first started doing this there were so few women. Even CS at UC Berkeley, there were so few. My parents have always told me I’m good at math things. So I was like, “I’ll do fine.”
At CCRMA, there weren’t that many women, but every time there was a woman I would look up to her, get inspired, and they’d help me out. Like Poppy Crum from Dolby helped me out with that internship there. And Tamara Smyth at USCD became my supervisor.
Really cool!
When I came to Output, there were only seven women here. It’s been five years now, and we have many more women in the company. It’s great!
I remember when I was TAing the audio production class at UCSD, a student came up to me and said, “Wow, it’s so cool to have a woman professor and a woman TA for a production class.” She was so used to seeing men in those roles. It made me grateful for the opportunity to contribute to female representation in the field.
But yeah, I feel like I’ve been really lucky. There’s always women who have done it before me that helped me out, brought me up and made it feel like it’s ok to try it too.
Featured Photo Credit: Tiff Chung. Jennifer was interviewed by Sudara.
Have comments on this interview? Ideas for who we should talk to next? Let us know on the JUCE forum.