Medford, Massachusetts, USA - January, 13, 1997 - We had the pleasure of visiting Dr. Joseph A. Paradiso and his very large (and complex) modular synthesizer. This modular system, which Joe started building over 20 years ago, includes two hand-built modular systems, three Moog keyboard synths, and 3 Casio synths, all integrated and patchable together to make one big modular system.

Dr. Joseph Paradiso spends most of his days at the MIT Media Lab. He joined the research staff there in 1994 to "explore novel applications of sensor technologies for opening new channels of computer-user interaction." During this period, he has designed many new interface devices based on electric field sensing, magnetics, acoustics, radar, and optics. These have included wireless violin bow trackers (used in performance by violinist Ani Kevaffian), the body-sensing "Spirit Chair" (used in performance by the magician duo Penn & Teller), gesture-sensing frames (for collaboration with The Artist Formerly Known As Prince), interactive piezoelectric balloons, musical radars, and scanning electric-field sensor arrays. As Technology Director for Things That Think, he identifies and pursues new areas of technical development for injection into TTT devices and projects. He also directs the design of advanced sensor and gestural interface devices for the Brain Opera in collaboration with composer and Media Lab Professor Tod Machover.

We follow Joe downstairs to his basement where we get our first look at the synth. As Joe turns up the volume on a patch that he put together the night before, we are immersed in a rich stereo blend of both analog and digital textures, loops fading in and out, virtually flying through the air. We asked him about the history of these modules.

synthmuseum.com: Which of these is the older modular system?

Joe Paradiso: That's the first one I made [the smaller rectangular unit with black knobs]. I did it when I was an undergraduate at Tufts during the mid-70's.... I started it pretty much my sophmore year and then I finished my senior year. It's a 36-module package with lots of inspirations. Some of the inspiration came by starting with standard PAiA kits and tweaking them around for better performance, others arose from going through the electronic music literature from the time, and most came from thinking about strange things I could do to control and generate unusual sounds. Some of them, like these VCF's, started out as PAiA's. They were cheap, [but] they were terrible... they were awful, so I went in an hacked them up, changed some of the circuits, adding things to it to make them more flexible, sound better and this thing just grew and grew until finally it ended. I said, "OK I'm done now. Gotta go to grad school. I'm going to forget about it." I've gone over it since and modified some of the electronics so it's stable and it all works.

[Joe Paradiso did not spend all of his time building synths and making electronic music. He received a B.S. in electrical engineering and physics summa cum laude from Tufts University in 1977, and in 1981 completed a Ph.D. in physics from MIT as a C.T. Compton Fellow in the Nobel Prize-winning group headed by Samuel C.C. Ting at the Laboratory for Nuclear Science. His dissertation research was based on an experiment measuring high-energy muon pair production at the European Center for Nuclear Research (CERN) in Geneva, Switzerland. From 1981 to 1984 he conducted post-doctoral research at the Swiss Federal Institute of Technology (ETH) in Zurich, where he developed precision drift chambers and fast electronics for the inner tracker of the L3 experiment at CERN/LEP.]

synthmuseum.com: And the rest of these modules?

JP: When I was in Zurich many years later, I had a lot of ideas left over from this [the first modular]. I had a great electronic lab. I was very busy the first year. I had no time. But the second year I had time. And in Zurich it takes a while to adapt and become part of life there. So I had to have something to do that I really wanted to do, aside from my work. I thought, OK, this is the time that I'll start making modules, and working a few of these ideas that I've had, so I made this. [points to the large box on the bottom shelf] I had a Casio VL-Tone, and I cut it apart, found the waveforms, made this thing. [Paradiso speaks more about the VL-Tone below.] I enjoyed it so much that I started making modules on some of my other ideas. So over a period of about a year, I cranked out most of the modules for these large units. That was in 1983. Then I came back from Zurich and brought them here. I went to the guy that ran the hardware store in Cambridge and had him build these cabinets. I just popped all the modules in and put power supplies in.

synthmuseum.com: What did you have the modules in before that?

JP: I just had them laying on the ground. If I did a piece in Zurich, for instance, I'd just have the modules laying on the floor or table and just wire them up. I'd build a little power supply for them and just let them go. It was a very busy part of life, but I don't think I'll build anymore modules. But that's it. The next thing I build, if I build on more thing for this, will be a good MIDI to CV interface. And that will control the whole thing.

synthmuseum.com: Now, is everything linear, 1v/octave?

JP: There's some non-linear stuff, but that's not tonal. It's just some control functions. All the stuff that maps the pitch, the oscillators and the filters, is linear. So that's easy. It can all be voiced. That's how the Minimoog and the other stuff can talk. It's all talking the same language. The guys from Aries, I met them, they told me never to go with an exponential keyboard. They were right.

synthmuseum.com: Tell us how you came about hooking up commercial synthesizers into this modular system?

JP: There are 2 parts to the philosophy. One was that I was buying synths to figure out how they worked and playing with them, together with building new modulars. And I want them all to be one system. I hate having a synth that's off by itself, and maybe they give you a plug for a filter, but that's not enough. so the Minimoog, I wanted to access all the oscillators and everything, totally separately, so I found all the points inside for the various control functions. Broke them up, put a jack in each one, made a [28] cable bundle, so the Minimoog is now totally abstracted. There are now separate control for the oscillators. I can Pulse-Width Modulate all of them independently, and make a complicated sound. I can control all the oscillators at once like the keyboard does. I have all the modulation waveforms coming out, the pitchbender... I can put in, take stuff out, stuff from the keyboard so the whole Minimoog is now abstracted to this panel. [points to upper left hand portion of modular system]

synthmuseum.com: And if you take the cables out, the Minimooog still works?

JP: As long as nothing's plugged into here, the Minimoog works just like a Minimoog always did.

synthmuseum.com: And the Moog Satellite?

JP: Same idea, although it's not as complicated a synth. It has some very strange features. The multimodal filter is very weird. The envelope generator is a little strange, and the waveforms makes it a little weird. But I have them all abstracted here so I can pull them out and change them. Then the Radio Shack [Concertmate MG-1], I did the same thing. It's a Micromoog basically. I have everything abstracted.....

[A modified MG-1]

(A heavily modified Casio VL-Tone)

...[for both the 101 and the SK-1] I figured out the whole keyboard scanning routine so I've completely mapped out what button and what key is what. The MIDI interface that I was going to build would have initially MIDI to CV, just a set of MIDI locations for these [buttons] so I can completely control all of these through MIDI, throw any switch or set any mode for the SK-1 or the others.

synthmuseum.com (still listening to the patch): This texture is quite amazing, it's so much richer of a mix than straight analog sounds.

JP: Yeah, well the digital stuff makes things very fat, too. Take, for instance, a lot of these waveforms. That thing that is being filtered in the background is coming from here. What it is is a binary rate multiplier. I'm getting high frequency signals coming from one of the VCOs and I'm just controlling the bits that are being used in the binary rate multiplier to control how it is dividing. The binary rate multipliers don't work to give you a nice even train of bits as you define the rate. They skip bits here and there and things like that in the patterns. It's a very gritty sound. You can still hear your tonality but it's gritty and it changes its grit according to how you mode it up.

synthmuseum.com: So you are digitally processing an analog-created bit stream?

JP: It's a square wave from an analog oscillator and then you just fatten it up by playing with the bitstream. And you don't necessarily need a computer to do it. You can just use simple chips or simple circuits and feed them parameters. One of my favorite things I use in alot of these songs is the pseudo-random sequencer. It is basically a sequencer, a ring counter like the old analog sequencers, but you can feed back the bits through exclusive-OR gates to the input to make pseudo-random chains, and this can be up to 256 stages long because there are 8 stages.

synthmuseum.com: These are processes and sounds that you just can't get from commercial synthesizers.

JP: You never see pseudo-random sequencers in synths, probably because the're too unpredictable. Although, they can be emulated digitally very easily.

synthmuseum.com: What else have you got going on in this patch?

JP: One of the strangest things I did that worked out fairly well was taking a waveform from some of these digital gates and making one of these little repeated trill-type sounds that come out of it. I tuned it up and threw it thru a phase lock loop so that's tracking it in an inaccurate fashion, kind of bouncing it around. and I ring modulated the two through a 4 quad multiplier ring modulator. So now when the phase lock loop is tracking the sound, you get all these beats as it comes up. You know when you ring modulate 2 sounds that aren't in tune, you have little side bands.

synthmuseum.com: I like the idea of creating sounds that no one has heard before. You seem to have that going on alot in this patch.

JP: The voice [sound] is strange. You don't hear that too often. That's the character of the synth. I use it a lot and it comes in a lot of the patches.

synthmuseum.com: Now, that is created by the Speech Synthesizer? How does that work?

JP: It's an old thing. It's a chip. Remember TI [Texas Instrument] did the Speak & Spell years ago, basic linear predictive coding, modeling the whole vocal tract. And then with very few bits they could actually make the thing talk, although it doesn't sound quite like a voice, but you can understand it. Then, after that, companies started making phoneme synthesizers, where they'd just have 64 phonemes, for instance, in a chip like this. [They would] make out a set of transition rules to go from one phoneme to another. And then your computer basically just feeds it bits, and [the phoneme synthesizer] knows how to put words into phonemes.

synthmuseum.com: So is each phoneme a waveform?

JP: It's a whole event. It's a triggered synthesizer event. In some cases it's a waveform, but it always has an attack that's specific. So going from one phoneme to another has an interpolation filter that makes it sound smooth. That's how, for instance, MacInTalk works. This [chip] is a very early version. But it's a simple 28-pin chip that you plug in. You just feed it bits and it talks. So instead of a computer, I feed it bits from my sequencer, my counter, from any of my other stuff. So there's no computer on this device. There are just parts of computers all over the place. It's like building your own ALU, in some sense, with flip-flops and gates. It's a lab course.

synthmuseum.com: I read an article recently talking about the Serge modular. Someone made the statement that "Digital is a subset of Analog."

JP: Serge was a big influence. When I wanted to do this, I was a sophmore at Tufts, and I called up all of the music departments around that would be into synthesized music. At MIT, I think I got ahold of somebody in Vercoe's group, if not Barry himself, and he didn't want to talk to me when I said analog synthesizers. "We don't do that here," that was already in 1974! But I got ahold of Serge's brother at Harvard, Ivan [Tcherepnin], and he invited me over. So I came down, and I saw the huge Buchla's in the distance, but he didn't go near those. He went over to the Serge, which his brother had designed. And he gave me a tour of the Serge, spent a good hour with me in there. And that probably was the event that changed my life in some sense, because I came out of that totally inspired. This is the way to build a synthesizer and it lead to this and eventually all this other stuff. Yeah, I'd love to spend more time with the Serge to see really how he did it. It's a wonderful set of ideas. The Serge is in a class by itself.

synthmuseum.com: People don't realize what synthesizer programming is until they work on a modular system. What do you see as some of the obvious benefits of a modular design.

JP: There's a tremendous amount of things you can do with an architecture like that. In some sense, you can think of this as being a set of control functions, a set of audio generators, a set of processing functions, like an effects rack built-in. And it's all integrated so anything can be combined with anything else, or driven by anything else, so it's kind of up to you how you want to put it together.

synthmuseum.com: Unlike most commercial synths out today, which are basically computers.

JP: With a computer, the difficulty is that you have data types that you have to worry about. You can't just plug a wire in and think it's going to work. You have a certain object and it has a certain format. It's going to plug into another object. You have to have your interfaces, basically appropriately put together so that they can blend arbitrarily. I guess if you built your structure properly you can get this kind of abstraction that we've got here. But your data types have to be consistent. Otherwise you can't plug anything into anything else. There are ways around that if you do that right, but at a certain point you have to deal with that digitally. Here you don't. Anything goes to anything. Any input can be driven by any source.

synthmuseum.com: Have you ever though of putting the ideas that you have developed through this modular, into designing synthesizers.

JP: If I were to design a synth now, it would definitely have to be digital. You can't build [analog modular systems] anymore. And you don't want to really. It's a pain in the neck. Whenever I move, wherever I go, I don't want to get rid of it, 'cause I know the instrument, I like it. But it's big. If my cat paws out a patch cord or if it falls out, it's got that fragility to it, and it takes up a lot of space. No one is ever going to really buy it except for a very few people. But the ideas here are good ones.

synthmuseum.com: What recordings has this system shown up in?

JP: It's appeared in, just recently, things that I did in the Media Lab [at MIT]. Tod [Machover] and I, we've worked together for a long time. I do all the technology and instrument design. But the other thing that I've done with him, too, is a little bit of sound design. We'll get together and think about sounds and I've given him some samples that come off of this [system] that we've used here and there. I've made a lot of pieces, not nearly as many as I'd like. [check out Joe's webpage for audio clips] Those [audio clips] all came from playing with this [modular system]. I have a few things I've worked on trying to compose music with it. It's been a long time, too long, It's been many years since I've really done anything. But I've got those [recordings] hanging around.

synthmuseum.com: I see you haven't lost sight of the purpose of all of this technology, making music.

JP: Well, you know, the other thing is, what's the object of doing this? For me, I guess, it was multi-fold. I had alot of ideas on how to do this technology and wondered what kinds of sounds it would make. And I did, and I learned. The other thing is, I did one instrument that matched what I had in my head. And I built that. And I'm going on to build other ones of other sorts. But, I haven't done enough music because I get caught up in all kinds [of thinking, such as] "I did this, but if I did this differently, it could be so much better. So I'm not going to do anything at all until I do this the way it should be done". And that's not a good idea necessarily. The object of the game in the end really is to make music. So I want to try and shift at some point. I'll never build another analog synth. (smiling) uh... I hope... mark my words. I may work with digital synths in software packages.

synthmuseum.com: After working with sensors and technology all day, creating music must be such a release.

JP: Oh, it is! Even putting a patch in is a release... it's a nice feeling. It's like sculpture.