Deepfake detectors struggle to tell real from fake using real world data

It could be a weird pixel in the corner, or a slight facial change, but that could be all a deepfake detector needs to tell if an image or video is real or fake. 

At least, that's the idea, but a new study by Australian and South Korean researchers has found that might not be the case. 

When tested in the real world, deepfake detectors struggled to tell the difference between what was real and what was fake, getting it right just two thirds of the time.

"In our evaluation, the current generation of deepfake detectors are not up to the mark for detecting real world deepfakes," Shahroz Tariq, one of the authors on the new paper and a deepfake researcher at CSIRO, said.

The study, which has been accepted for presentation at the IEEE European Symposium on Security and Privacy 2025, highlights an arms race between those generating new types of deepfakes and those trying to detect them, with real-world consequences.   

A cat-and-mouse game

Deepfake detectors are designed to deduce if an image, video, audio clip or other type of media has been manipulated, or fabricated entirely. 

This could be a swapped face on a video, an image of a completely made up person, or a faked video of a news event.

Detectors are trained on whole databases of fake images, audio and video to create a type of fact-finding neural network. 

In the same way that AI can learn what a banana or a car is from countless pictures and videos being fed into it, being trained on these large databases teaches deepfake detectors what a deepfake looks like.

Lea Frermann, a computer scientist at the University of Melbourne who specialises in misinformation, notes these are not necessarily the same clues that a human would pick up on.

"If you change something that seems completely inconsequential to a human, like five random pixels in an image, it can completely derail the model."

Those making deepfakes and those trying to detect them are constantly trying to outpace the other. 

"It's a bit of a cat and mouse game," said Dr Frermann.

"The underlying technologies are very similar, and whenever the generators get better and the deep fakes get more convincing, they also get harder to detect."

And unfortunately, being a neural network, these detectors have limits. Something as simple as compressing the video or image in a certain way can fool some detectors. 

Detector training

While some databases of deepfakes are used to make the detectors, other databases are used to put the detector through its paces, analysing real and fake results. 

Two common databases are used to test the models — CelebDF and the Deepfake Detection Challenge.

CelebDF is made up of mostly YouTube-style, well-lit videos of celebrity deepfakes, while Deepfake Detection Challenge (DFDC) has a wider variety of videos, including those that are compressed, grainy or not well lit. 

According to Dr Tariq, the well-lit shots with common celebrity faces are easier for the detectors to spot than those in DFDC. 

In 2020, a coalition of companies including Facebook, Microsoft, and Amazon paid US$1,000,000 in prize money to the most accurate deepfake detection model using the DFDC dataset. 

The winning detector scored just 65 per cent.

In the current study, the team looked again at the newest detectors and how they worked on the DFDC. 

The top scorer could correctly identify the real or fake video 86 per cent of the time. This was lower than the easier to identify CelebDF, but higher than the 2020 results. 

"From the benchmark data set perspective, we can say that we are getting better," Dr Tariq said. 

But the issue is that these datasets are now years out of date. 

When the team took a collection of newer deepfake videos from the internet — what they call "in the wild" — the percentage dropped from 86 to 69 per cent.   

But, because deepfake generation moves so fast, and the team only collected images up until 2024, the number might be even lower if the same analysis was done today.

"Detectors which work on past data sets do not necessarily generalise to the next generation of fake content, which is a big issue," Dr Frermann said.

While politicians and researchers have highlighted the potential for election misinformation and deepfake porn creation, Dr Tariq said the issue had also become a big problem in security, especially with features such as facial verification.  

Humans working with machines

Unfortunately, solutions for this problem aren't easy. 

In the paper, the team suggest a new framework that other researchers can use to create new and better deepfake detectors. 

Dr Tariq also said building specialised detectors, which only focus on one type of deepfake, could be helpful.

"We tried one detector in this paper which was specifically trained on celebrity images, and it was able to do really well on celebrity deepfakes," he said. 

"But if you tried to use that same detector on images of all sorts of people then it doesn't perform well."

Dr Frermann agreed that current detectors were particularly bad at "generalising" to other deepfake types, but she thought this may not solve the arms race issue. 

"Technology always changes," she said. 

Instead, she suggested more non-technical solutions might also be required. 

The most important one for her is ensuring people know that deepfakes are out there, and understanding the subtle ways they may be able to be spotted. 

Can you spot a deepfake?

Researchers at Northwestern University have also created a website called Detect Fakes where punters can test how good they are at spotting real from fake media. 

The ABC NEWS Verify team has also created a whole quiz that will really put you to the test.

One benefit that humans have that deepfake detectors don't is the context surrounding the image. 

Dr Frermann says this can be extremely helpful in working out what's real and what's fake. 

"We know the platform where we found this video, we might know who posted the video," she said. 

"There's a lot of metadata and context that we could use to make a more informed decision."

Another potential way to fix this issue is ensuring that effective AI regulation is put in place. 

But according to Dr Frermann, this is extremely hard to get right. 

"'I'm not aware of any place where this has been done satisfactorily," she said. 

In the meantime, the cat-and-mouse game continues.

"Knowing if something is real or fake is becoming a big problem, and that's why we need to do more research and develop better tools," Dr Tariq said.