Anika Begay
Stars from Hollywood’s golden age are being reborn thanks to AI voice-cloning deals between celebrity estates, a sign that some of the “Wild West” concerns about unauthorized AI impersonation are being addressed by new business models.
ElevenLabs, an audio technology startup backed by venture capital firms including Andreessen Horowitz and Sequoia, has signed several deals with the estates of iconic actors for its IconicVoices tool that lets users have AI-generated voices read to them through an audiobook app. Stars include Burt Reynolds, Judy Garland, James Dean and Sir Laurence Olivier.
ElevenLabs, which launched in 2023, creates audio for books and newspaper articles, video game characters, film pre-production, social media, and advertising. The company already works with publishers including The New York Times and The Washington Post, and earlier this year was selected by Disney to join its accelerator program.
“It takes about 30 minutes of high-quality audio to create a professional voice clone,” said Sam Sklar, a member of ElevenLabs’ growth team, and the voices are generated from the celebrity’s catalog. Once created, it can be used to read text (articles, PDFs, ePubs, newsletters, or other text-based content). However, the voice and content cannot be exported, with all listening in a reading app.
For example, a user could listen to articles narrated by James Dean within the app, but would not be able to access the voices of any content not already in the app.
These kinds of agreements could help set the stage for a future where AI-generated voice content is less controversial and more of a controlled, curated playground. Google Play and Apple Books already use AI-generated voices to some extent, though there are high hurdles in recreating the rhythm, intonation, and emotion of the human voice.
The AI industry has been plagued by concerns about the use of celebrity voices, with OpenAI backtracking in May after actress Scarlett Johansson accused the company of copying her voice after she turned down licensing offers.
“We are very aware of the risks associated with synthetic media and take the safe use of our tools incredibly seriously,” Sklar said. Safeguards include active content moderation, enforcement with bans, and special provisions to safeguard against AI voice impact on the 2024 election.
Among the current generation of actors, there remains considerable anxiety about the use of AI in voice generation. Video game voice actors have raised concerns, and last year’s film and television strike had significant roots in anxieties about the use of AI. The use of iconic voices sold by properties is a niche market that potentially avoids these pitfalls, representing a new revenue stream from AI rather than a revenue stream lost to AI.
The use of celebrity lookalikes is an issue that predates AI, with Frito Lay using a Tom Waits lookalike in its ads in 1988 and Waits in 2007 after Waits himself had long refused advertising deals. AI presents an easier path to creating lookalikes, and recent lawsuits filed against AI startup Lovo alleging inappropriate and uncompensated use of voice actors in its AI voice generation remind us that the world of AI voice generation is likely to remain somewhat complicated and contentious. (Lovo denied the claims in the lawsuit and also pointed to a revenue-sharing model it offers actors for cloned voices.)
It’s difficult to assess the protections in some places without examining the specific language in IconicVoices’ contracts, said Steve Cohen, a partner at Pollock & Cohen who is representing voice actors in an unrelated lawsuit alleging voice cloning without permission.
ElevenLabs highlights how its IconicVoices tool obtains permissions and manages voice usage.
“Giving permission to use your voice is one of the foundations,” Cohen said. “I think the key factors are permission, compensation and control.”
Clearer new laws could also discourage people tempted to misappropriate a voice, “not for the hardened bad guys, but for the edge cases,” Cohen said. But quoting Bette Davis in “All About Eve,” he added, “‘Fasten your seat belts; it’s going to be a bumpy ride.'”
How realistic cloned voices sound is also an evolving issue. Many experts say that because AI doesn’t “know” what it’s saying, the quality of performance is limited. Sklar said that ElevenLabs’ latest level of voice quality is indistinguishable from real human speech. “ElevenLabs’ text-to-speech tools can understand the context of words,” he said.
AI is only as good as the models it is trained on, and actors’ voice datasets become part of the process.
“Neural models get their skills from mimicking/memorizing nuances and patterns in their training data,” says Nauman Dawalatabad, a postdoctoral associate at the MIT Computer Science and Artificial Intelligence Laboratory with extensive research in AI speech generation. “The quality and diversity of the training data significantly influences the model’s performance.”
The vocal delivery of movie stars could help AI imitate and learn by providing the kind of “high-quality voice datasets for training and fine-tuning large models” that Dawalatabad said are essential to the process. But he expressed reservations about whether “sounding human” is the right test for the AI voice field, as it could reinforce an antagonistic relationship between human and synthetic voices.
Voice actors remain divided over the technology, with some refusing to consider any deal but others saying the opportunities to clone their voices for faster, cheaper production on some forms of audiobooks can’t be ignored. “AI technology can help workflows. AI is not a new tool for voice actors, producers and editors, many of whom are using it to improve quality control in post-production,” Michele Cobb, executive director of the Audio Publishers Association, told Vscek last year.
Recent generative models have shown significant progress over previous iterations, making it increasingly difficult to distinguish fake and authentic voices by ear alone, according to Dawalatabad. AI voice licensing could ease the burden on voice actors, he added, without replacing them, since they “intercede in the process by focusing on providing correction or improvement on ineffable aspects like intonation, warmth, and emphasis, which still present challenges.”
