- Humans can generally trust what they see and hear — but that won't be the case for long.
- Advances in AI and CGI will soon make it possible for anyone to create photorealistic video and audio.
- Experts say it will transform information warfare, allowing the creation of sophisticated propaganda and misinformation.
- The tech's impact will be profound, turbocharging everything from fake news and hoaxes to revenge porn and DIY entertainment.
Hoaxes and trickery are almost as old as human history.
When the Roman Republic first conquered the Italian peninsula between 500-200 BC, it was known to send fake refugees into enemy cities to"[subvert] the enemy from within.""Pope Joan" was believed to be a woman who allegedly tricked her way into become pope in the Middle Ages by pretending to be a man — but the entire story is now viewed as fake, a fictional yarn spun centuries after her purported reign.
"Vortigern and Rowena," a play that debuted in 1798, was initially touted as a lost work of William Shakespeare— but was in fact a forgery created by William Henry Ireland. And in the 1980s, the Soviet Union attempted to damage the United States' reputation and sow discord among its allies by spreading the myth that American scientists had created AIDS in a military laboratory, in an "active measures" disinformation campaign called "Operation INFEKTION."
Some fringe historians even believe that almost 300 years of medieval history were a hoax— invented retrospectively by the Holy Roman Emperor Otto III for political purposes in 1,000 AD.
But humanity is now rapidly approaching the holy grail of hoaxes: Tools that will allow anyone to easily create fraudulent, photo-realistic video and audio.
Thanks to advances in artificial intelligence (AI) and computer-generated imagery (CGI) technology, over the coming decade it will become trivial to produce fake media of public figures and ordinary people saying and doing whatever hoaxers can dream of — something that will have immense and worrying implications for society.
In a previous feature, Business Insider explored how the tech will make it far more difficult to verify news media— boosting "fake news" and exacerbating mistrust in the mainstream media. But experts now say that its effects will be felt far more broadly than just journalism.
It will open up worrying new fronts in information warfare, as hostile governments weaponise the technology to sow falsehoods, propaganda, and mistrust in target populations. The tools will be a boon to malicious pranksters, giving them powerful new tools to bully and blackmail, and even produce synthetic "revenge porn" featuring their unwilling targets. And fraud schemes will become ever-more sophisticated and difficult to detect, creating uncertainty as to who is on the other end of any phone call or video-conference.
This may sound sensational, but it's not science fiction. This world is right around the corner — and humanity desperately needs to prepare itself.
The technology is basic — but not for long
Right now, the technology required to easily produce fake audio and video is in its infancy. It exists mainly in the form of tech demos, research projects, and apps that have yet to see a commercial release — but it hints at the world to come.
A few examples: In July, researchers at the University of Washington used AI to produce a fake video of President Barack Obama speaking, built by analysing tens of hours of footage of his past speeches. (The audio used also came from an old speech.)
The tech to do this live already exists. In 2016, "Face2face" researchers were able to take existing video footage of high-profile political figures including George W. Bush, Vladimir Putin, and Donald Trump, and make their facial expressions mimic those of a human actor, all in real time.
People are also working to spoof human speech. Voice-mimicking software called Lyrebird can take audio of someone speaking and use it to synthesise a digital version of that person's voice — something it showed off to disconcerting effect with demos of Hillary Clinton, Obama, and Trump promoting it. It's in development, and Adobe, the company behind Photoshop, is also developing similar tools under the name Project Voco.
The next generation of information warfare
In early August 2016, the US had an international crisis on its hands, and Americans were beginning to panic. As many as 10,000 armed police had surrounded the US Incirlik airbase in Turkey, and Twitter users were worrying that the situation could rapidly escalate — perhaps even with the nuclear weapons on the base falling into the hands of the demonstrators.
Except, it didn't really happen like this. As The Daily Beast reported, the reality was a peaceful protest of around 1,000 people. Russian state propaganda outlets Russia Today and Sputnik pushed the false narrative, aided by thousands of English-language tweets sent from accounts identified as bots controlled by the Russian government, Foreign Policy Research Institute fellow Clint Watts told the US Senate Intelligence Committee in 2017.
This is an example of Russia's longstanding policy of "active measures"— spreading misinformation for propaganda purposes or to help it achieve its strategic objectives. Gregory C. Allen, an adjunct fellow at the Center for a New American Security, argues that these efforts from Russia — and others like them — will receive a powerful shot in the arm from developments in CGI and AI.
"We have seen foreign governments be more than willing to rely on ... propaganda in the text real and in the fabricated imagery realm," he told Business Insider. "They have demonstrated their willingness to sprint as fast as they can in this exact direction, and making use of every tool that is available to them."
The future could see authoritarian states using forged media to help generate dissent in the populations of rival countries, much like what happened at Incirlik — and to discredit and damage political opposition at home.
Allen also discussed the national security implications of artificial intelligence in a recent paper, warning: "We will struggle to know what to trust. Using cryptography and secure communication channels, it may still be possible to, in some circumstances, prove the authenticity of evidence. But, the 'seeing is believing' aspect of evidence that dominates today — one where the human eye or ear is almost always good enough — will be compromised."
The tech is a bonanza for fraudsters
A popular technique employed by modern scammers is "CEO fraud"— an email sent to a company employee, masquerading as from the CEO or another executive, asking them to make a payment to an account or take another action.
These kinds of attacks will soon have a whole new line of attack: Voice.
Imagine your boss calls you up, and asks you to make a transaction, or send over a password or a confidential document. It's clearly her voice, she knows who you are, and you might even make some small talk. Today, no-one would think anything was amiss.
This is because, Allen says, you "are currently using voice as an authentication technology, but [you] don't think of it as an authentication technology because it's just a background of human life that you can trust."
But in five or so years, that trust may have evaporated — replaced by a mistrust of what you hear on the phone and even see with your own eyes in a video-conference: "In the very near future it's not going to be something that you can rely on. Likewise, a video forgery techniques get further along, the same will be true if you were to have a video chat with someone."
Old people and those are less tech-literate will be particularly vulnerable, Francis Tseng, a copublisher of The New Inquiry (who curates a project tracking how technology can distort reality) suggested: "Many people deal with their parents or grandparents falling prey to phone scams ... And an easy rule of thumb to tell them is 'don't give out private information to anyone you don't know!'. With these voice synthesis technologies, someone could easily forge a phone call from you or another relative."
It will turbo-charge fake news
We're already living in an era of "fake news." US President Donald Trump frequently lashes out online at the "phony" news media. Hoax news outlets have been created by Macedonian teenagers to make a quick buck from ad revenue, their stories spreading easily through platforms like Facebook. Public trust in the professional news media has fallen to an all-time low.
When anyone can throw together a video of a politician or celebrity saying whatever they want, it seems likely to engender further mistrust — and allow hoaxes to spread more easily than ever before.
And there's a flipside to this: It will also cast some doubts on even legitimate footage. If a politician or celebrity is caught saying or doing something untoward, there will be an increasing chance that the person could dismiss the video as being fabricated.
In October, Trump's presidential campaign was rocked by the "Access Hollywood" tape — audio of his discussing groping women in vulgar terms. What if he could have semi-credibly claimed the entire thing was just an AI-powered forgery?
- Read more: CGI and AI are going to turbocharge 'fake news' and make it far harder to tell what's real »
It will transform cyberbullying
This technology won't just be misused to pursue political and strategic objectives, or to defraud businesses: It will be a weapon for bullies, capable of inflicting arbitrary cruelty.
In the hands of children, it seems likely to be misused to hijack the image of victims', and to animate it for malicious purposes. A child's digital avatar might be made to confess their love for another, embarrassing them — or their voice could confess to a misdemeanour, landing them in trouble with school authorities.
Justin Thies, who helped develop Face2face, predicted it would "lift cyberbullying to a whole new level."
A spokesperson for child protection charity NSPCC acknowledged the danger: "Emerging technologies, such as AI and CGI, pose both potential risks and opportunities to young people and we must make sure they do not leave children and young people exposed to danger and harassment online.
"We know that cyber-bullying can be particularly devastating to young people as it doesn't stop in the playground and follows them home so they feel they cannot escape."
It will create a new category of sexual crimes
The episode indicates there is likely to be significant interest in on-demand pornography produced using these technologies in the years ahead, regardless of whether the subjects of these CGI films give permission.
"Revenge porn" websites already exist dedicated to cataloguing and sharing the intimate photos and videos of non-celebrities, and it seems likely that media-editing technology will be used to produce material featuring "ordinary" people, as well as the rich and famous — bringing with it the widespread risk of shame and blackmail.
A whole new world of entertainment awaits
Not every use case of this tech will be negative, however. The internet is already home to a vibrant remix culture — just look at the Reddit community "Photoshop Battles" — and photorealistic video-editing tools may well spark a huge wave of DIY creativity.
"There could be a lot of interesting IP cases if amateur filmmakers start synthesizing films using the likenesses of celebrities and start profiting off that. I can imagine a whole culture of bootleg films produced in this way," Tseng said.
The tech that powers face-modifying filters in apps like Snapchat is "primitive compared to the Hollywood CGI or today, but it's actually significantly more advanced than the Hollywood CGI of the Eighties," Allen said. "So what we're seeing is the state-of-the-art capabilities slowly come down in price and availability such that amateurs have access to ultimately what are rather impressive capabilities."
The tech likely to be used by the established entertainment industry as well as amateurs, Tseng suggested: "We've also seen movies adapt their scripts for certain markets (e.g. the 'Red Dawn' remake changing the villains from China to North Korea). There is already a practice of filming scenes to be slightly different for different markets but this technology could lead to it on a much larger scale, where even individuals experience a version of a film totally personalized for them."
Just look at "Star Wars: Rogue One" for an example of how this tech will be employed by Hollywood studios in years to come. Peter Cushing reprised his role as Grand Moff Tarkin — even though he had been dead for 22 years. His image was reconstructed using CGI overlaid on a real actor.
This is all right around the corner
This is all currently theoretical. But it won't be long until it becomes a reality.
"I think we are one to two years away from these sorts of forgeries, especially in audio where progress is a little bit easier," Allen said. "One to two years away from forgeries being able to fool the untrained ear and somewhere between five to 10 years away from them being able to evade certain types of forensic analysis."
So how do we prepare? Journalists and organisations will have to rely increasingly on cryptography to "sign" media, so it can be verified when required. Big platforms like Facebook will have a roll to play in policing for fraudulent material, Face2face's Justus Thies argues: "Social-media companies as well as the classical media companies have the responsibility to develop and setup fraud detection systems to prevent spreading / shearing of misinformation." And it will force ordinary people to be far more skeptical about the media they consume.
In some cases, "it may be possible to come up with a video format that simply rejects editing," Allen suggested. "But this will still be a suboptimal solution compared to what we have now ... in the best case scenario, this results in there [being] trained experts who can discern the most likely version of the truth, and that is just so far away from where we are today which is amateurs can rely upon their own eyes to discern the truth."
We don't realise just how lucky we've been
These advances mean that humanity is rapidly approaching the end of a unique period in human history. We "live in an amazing time where the tech for documenting the truth is significantly more advanced than the tech for fabricating the truth. This was not always the case. If you think back to the invention of the printing press, and early newspapers, it was just as easy to lie in a newspaper as it was to tell the truth," Allen said.
"And with the invention of the photograph and the phonograph, or recorded audio, we now live in a new technological equilibrium where — provided you have the right instruments there — you can prove something occurred ... we thought that was a permanent technological outcome, and it is now clear that is a temporary technological outcome. And that we cannot rely on this technological balance of truth favouring truth forever."