Last week, voters in New Hampshire received a robocall impersonating President Biden, urging them not to vote in the state’s primary election. Researchers believe that the deepfake call was likely created using technology from AI startup ElevenLabs. Teams of audio experts analyzed the call and found patterns consistent with ElevenLabs’ voice-cloning tools. The company, which recently achieved “unicorn” status with its valuation surpassing $1.1 billion, offers its AI tools for various purposes, including political speech. While concerns about the misuse of AI-generated content continue to grow, the incident raises questions about the preparedness of authorities and the tech industry as the 2024 election season approaches.
Background Information
Deepfake Biden Robocall in New Hampshire
Last week, voters in New Hampshire were targeted with a deepfake robocall impersonating President Biden. The call urged recipients not to vote in the state’s primary election. The origin of the call is still unknown, but experts believe that the technology used to create it came from ElevenLabs, a prominent voice-cloning startup based in Silicon Valley.
ElevenLabs and its AI tools
ElevenLabs is a leading AI voice-cloning company that primarily markets its tools for uses in audiobooks and video games. The company recently raised $80 million in funding, achieving a staggering valuation of $1.1 billion. With its advanced technology, ElevenLabs has gained recognition as one of the top providers of synthetic voice offerings in the market.
Other instances of suspected use of ElevenLabs technology
This is not the first time that ElevenLabs has been associated with the misuse of their tools. Last September, NewsGuard, a company that tracks online misinformation, claimed that TikTok accounts sharing conspiracy theories with AI-generated voices, including a clone of Barack Obama’s voice, were using ElevenLabs’ technology. While the majority of users on ElevenLabs’ platform create valuable content, instances of misuse have prompted the company to develop and release safeguards to prevent abuse.
Pindrop’s Analysis of the Robocall
Identification of ElevenLabs technology
Pindrop, a security company specializing in synthetic audio identification, conducted an analysis of the deepfake Biden robocall. Their research pointed to the high probability of ElevenLabs’ technology, or a system using similar components, being involved in the creation of the call. Pindrop’s team compared the audio patterns of the clip with over 120 different voice synthesis engines and found a clear match indicating the use of ElevenLabs’ technology.
Comparison with audio samples
To further validate their findings, Pindrop compared the suspect robocall audio sample with known samples created using ElevenLabs’ technology. The results reaffirmed their initial conclusion that the deepfake Biden robocall was likely made with tools from the AI startup. This analysis was crucial in establishing a strong link between the call and ElevenLabs’ technology.
Confirmation by UC Berkeley analysis
Hany Farid, a digital forensics specialist at the UC Berkeley School of Information, initially expressed skepticism about the involvement of ElevenLabs in the Biden robocall. However, after conducting their independent analysis of the audio sample, Farid’s team reached the same conclusion as Pindrop. Their high-confidence model affirmed that the audio was AI-generated and likely produced using ElevenLabs’ technology.
ElevenLabs and AI Voice Cloning
ElevenLabs as a market leader
ElevenLabs has cemented its position as a market leader in the field of AI voice cloning. With its high-quality synthetic voice offerings, the company has attracted substantial investments, valuing it at over $1.1 billion. Investors such as Andreessen Horowitz, Nat Friedman, and Mustafa Suleyman have recognized the potential and innovation of ElevenLabs’ technology.
Funding and prominent investors
The significant funding received by ElevenLabs has allowed the company to invest in research and development, enhancing its safeguards against misuse. Sequoia Capital and SV Angel, along with individual investors, have contributed to the growth of the company. These investments have positioned ElevenLabs as a frontrunner in the AI voice cloning industry.
Safeguards against misuse
Recognizing the potential for misuse of their technology, ElevenLabs has actively worked on developing safeguards. However, the widespread availability of AI voice cloning tools, including those similar to ElevenLabs’ technology, poses a challenge in adequately policing the use of these tools. The company continues to release safeguards to mitigate the risk of misuse while enabling legitimate use cases.
Concerns and Potential Misuse
Availability of AI voice cloning technology
The availability of AI voice cloning technology has opened up new business opportunities. Companies and individuals can now experiment with cloning voices, allowing for cheaper production of audiobooks and other audio content. However, this accessibility also increases the potential for malicious use, as the technology can be misused to spread false information or manipulate public opinion.
Business opportunities and malicious use
While the business opportunities presented by AI voice cloning are vast, the potential for malicious actors to exploit this technology cannot be ignored. The ability to create convincing deepfake audio can be used to deceive individuals, influence elections, or spread disinformation. The misuse of AI voice cloning poses a real problem that needs to be addressed.
Challenges in policing the technology
The broad availability and ease of use of AI voice cloning tools present challenges in effectively policing their use. Monitoring and preventing the misuse of AI-generated content becomes increasingly complex when anyone can access and utilize these tools. The responsible use of AI voice cloning technology requires the collaboration of industry leaders, authorities, and the public to develop effective regulations and monitoring mechanisms.
Implications for the 2024 Election
Underpreparedness of authorities and tech industry
The deepfake Biden robocall incident underscores the underpreparedness of authorities and the tech industry as the 2024 election season approaches. Ensuring a fair and transparent election process requires accurate verification of audio clips and prompt action against any AI-generated propaganda. Without reliable tools and rapid response capabilities, journalists, election officials, and others lack the means to effectively address the potential threats posed by deepfake audio.
Difficulty in verifying audio clips
Verifying the authenticity and provenance of audio clips, especially AI-generated ones, is a complex task. Without specialized expertise, it becomes increasingly challenging for individuals to differentiate between real and deepfake audio. The ability to quickly and accurately authenticate audio clips is crucial in preventing the spread of misinformation and ensuring public trust in the electoral process.
Impact of AI-generated propaganda
AI-generated propaganda, such as the deepfake Biden robocall, has the potential to significantly influence public opinion and sway election outcomes. The use of advanced technology to manipulate audio clips that mimic the voices of political figures can mislead voters and disrupt the democratic process. The impact of AI-generated propaganda on election integrity demands immediate attention and effective countermeasures.
Conclusion
Need for reliable tools and rapid response
The prevalence of deepfake audio, as demonstrated by the deepfake Biden robocall, highlights the urgent need for reliable tools and rapid response capabilities. The development of advanced technologies that can accurately detect and verify AI-generated audio is crucial in combating the spread of disinformation and ensuring the integrity of elections. Investing in these tools and providing access to journalists, election officials, and relevant authorities is paramount.
Journalist and election official limitations
The limitations faced by journalists and election officials in verifying audio clips highlight the need for accessible, user-friendly tools and resources. Collaboration between technology companies, academic institutions, and regulatory bodies can help bridge the gap between technical expertise and practical use. Empowering journalists and election officials with the necessary tools and knowledge is essential in combating the threats posed by deepfake audio in the upcoming elections.
