Tupac and Vocal Synthesis
How did they do it?
The impressive visuals of “2pac’s resurrection” at Coachella 2012 towered over another admirable mystery…2pac’s voice?
Is he half dead? The voice just remained? NO!!! This is what we call Vocal/Speech Synthesis in the world of digital audio.
Now I do have to let you know… I do not know EXACTLY HOW they pulled this off. This technology grows exponentially! It’s everywhere. How about that synthesized voice over the phone you hate so much?????
#0, 0, 0, 0, 0, 0, 0, 0, 0!
Each Synthesist has his/her own technique. The techniques may vary, from old tricks of the trade, shifts due to technological advances, or simply personal creative processes.
All I can do, as an experienced Synthesist, is exercise my knowledge and proper judgment for inference in this case.
Have you ever heard really great vocal impressionists?
Whether it be Aries Spears, Jimmy Fallon, or Darrell Hammond, these vocal impressionists have the unique ability to re-create the qualities and physical characteristics of another’s vocal chords: inflections, pitch, timbre, and more. Other characteristics may include nasally, breathy, strained, muffled voices, etc.
Well… it wasn’t an impersonator. Actually, any one of us could have sung it.
Re-creating Tupac’s Voice
Each and every one of us has our own unique voice. My vocal chords produce a series of emphasized frequencies (without getting too technical) called formants. And yours do, too. And 2pac’s and everyone elses. Physical properties: formants, resonances, and other “stuff” all combine to create your voice.
And the beauty of these physical properties is, they are quantifiable (measurable and re-createable).
So let’s get back to 2pac. ?To figure out what properties make-up 2pac’s voice, we can feed some of his old recordings through a spectrogram. To look for frequency properties in a spectrogram, the search is on for bands of frequencies across the spectrum.
The video below shows you the look of a spectrogram, function, and behavior. It might take you off track due to its subject. This particular team are searching for “The frequency balance of a hit record”
I commend their efforts, but don’t be silly and try this. We are scientists. They don’t have a valid sample size (plus a lot more) to make truthful claims deeming this empirical evidence of “the frequency balance of a hit!”
Back on track…Maybe the analyzer only caught 2pac’s voice on “a specific day,” where he was particular harsh, nasally, or partied too hard the night before contributing to a groggy voice the next day. We all have our days. We don’t “exactly” sound the same each day, right?
So what do we do in the case of 2pac? Feed as many recordings into the spectrogram, and if time and money permits, even interviews! Whatever you can get from 2pac, get it! The objective: ESTABLISH A MEAN.
Establish and concretize the average of: formants, resonances, and the rest of 2pacs “voice”…. The physical properties.
Now that we know that each property is quantifiable, we plug 2pac’s numbers (very specific frequencies e.g., 548hz) into the master program.
Once our numbers, “2pac’s voice in numerical form”, are plugged into the master system (think auto-tune), YOU can start singing, IF you can mimic 2pac’s tendencies: The way he sings.
Put simply: the “stuff” that makes up a voice is now in a software program.
So you know how he sings right, that pitch bend thing??? We can all mimic that. Try it. But you don’t have to be in pitch to record in this master system! You’re in an auto-tune-like program. You just have to listen very intently and be able to re-produce the way he sings.
What They Probably Did
They probably hired someone that can mimic 2pac very well. He then sang through this master system, which had 2pac’s voice, and thus you can create 2pac yelling COACHELLA and anything else for that matter. After all, they are supposedly taking the hologram on tour. He’ll need more songs and shout-outs, in the year 2012.
Coachella yell is @ 0:33
3d hologram: Whether it’s creepy or not… it’s not my cup of tea, but I did find it interesting from the synthesists point of view.
I want to thank some good friends for helping me try and figure out what the company Digital Domain is not telling anyone: the robust SCIENCE involved in the creation of the 3d hologram.
Dragan Grubesic- Animator
Joseph Farrell- Aerospace Engineer
Pseudo-conclusion on the hologram itself: Angles, multiple 3d projections, reflective qualities of mesh background… the rest is the innovation of the company (basically we gave up guessing because it could be a lot of techniques used together.)
This one is dedicated to my boy Dragan Grubesic.
Check him out here: http://www.linkedin.com/in/dragangrubesic
Whether it’s Dragan, Joseph, or you… Be proud to be contributors of Google searches that start with “how did they make___!!!”