Vision Pouya
فارسیFA DeutschDE EnglishEN
Follow
Article

When AI becomes the guarantor of lies

Tools built to separate fake images from real ones have, in documented cases, done the opposite: certified the fake as real.

Three video frames of city scenes with smoke, fire and missile trails on a dark background; the frames look like short-video app clips labelled "live israel"
Frames from the fabricated videos of the missile attack on Tel Aviv — the ones Grok certified as real, citing Reuters and CNN reports that did not exist.

In March 2026, fully fabricated videos of an Iranian missile attack on Tel Aviv spread across X, shared millions of times. Grok, the platform's AI assistant, confirmed them as real — and invented the receipts to prove it, citing Reuters and CNN, neither of which had published any such report. Euronews documented the case on 6 March.

Users ask Grok on X whether the video is real; Grok confirms it
Users asked Grok whether the video was real, and Grok confirmed it; Euronews refuted the confirmation with documentation on 6 March

According to the International AI Safety Report 2026, around 68 percent of today's deepfakes are nearly impossible to detect, and in controlled tests, humans performed close to random guessing. The automated tool was supposed to be the answer — but the same year's data shows the tools share the problem, and sometimes make it worse.

Three cases, one pattern

The same month, in the US midterms, a party committee released a video of a rival candidate saying things he never said. The "AI-generated" label was there — in tiny type, for a few seconds. CNN called it the first political deepfake to recreate a candidate realistically for an entire clip.

2 months later, an image of Italian Prime Minister Giorgia Meloni in a fabricated situation spread online; Reuters and the Associated Press confirmed it was AI-generated — but by then the image had been circulating for a long time without any warning. Three cases from three different worlds — war, elections, a political figure — share one trait: at the exact point where AI was supposed to help, telling fake from real failed.

Why the tools get it wrong

The technical core: language models do not test truth; they produce the most probable continuation of text. Asked how it verifies images, Grok itself answered: pattern recognition from training data and comparison across sources — not a laboratory instrument. The output is only as sound as the input it was trained on; when the input is skewed, so is the verdict.

The second reason is an arms race between generation and detection. Every sharper detector becomes the training material for the next generation of fakes, which are tuned to slip under that exact method; researchers call it the AI cat-and-mouse game. No fixed detection tool stays effective forever.

The third reason is how uneven the tools are. In a test where 15 completely real images were given to several detection systems, one tool advertising "industry-leading accuracy" flagged 6 of them as fake — a 40 percent error rate — while two other tools identified all 15 correctly. Wrongly rejecting real images is just as common; no shared standard exists between the tools.

The false claims date to the period of the Iran war, which began on 28 February, per AP. A NewsGuard review found that chatbots asked about those claims repeated the false claim in more than 50 percent of their answers — some of them, including a European model, did so consistently.

Grok's verdictThe record
Verdict"real — citing Reuters and CNN"Neither outlet had published such a report; Euronews documented the fabrication on 6 March

The fix: provenance, not detection

The industry's practical answer is a change of direction. Instead of examining every suspicious image after publication — which the speed of fabricated content no longer allows — authenticity is registered from the moment of creation: cryptographic watermarks, verified metadata, standards like C2PA that trace an image's origin from camera to publication. The burden of proof shifts from "prove it is not fake" to "prove from the start where it came from".

The EU's AI Act has required labelling of synthetic content since 2024 — but it obliges the deployers, and the networks where the clips actually circulate are not always the deployers.

As long as detection rests on statistical guessing rather than traceable proof, the pattern will repeat. AI today is not a neutral judge but a participant in the same game it was supposed to referee: it produces deepfakes, sometimes confirms them, sometimes rejects real images. The real solution is not smarter detection tools — it is changing where proof begins.

The most important name that was in no headline

Mohammadparsa Amini, 17, killed in Fardis
Mohammadparsa Amini; memorial photo published by the Iran Human Rights Center
Mohammadparsa Amini in the evening light
Mohammadparsa in the evening light; from his personal album, published by Javidan Iran
Mohammadparsa Amini on the balcony at night
One of his nights in Fardis; from the Javidan Iran memorial gallery

According to Iran International, Mohammadparsa Amini was killed on 18 Dey 1404 — 8 January 2026 — in Fardis, near Karaj, by a gunshot from security forces to the waist. He was 17. Videos shared on social media showed moments from his life, his funeral and his memorial service. One of his famous lines, preserved by Javidan Iran: "I will do something you can be proud of." In the days when the reports ran on the war and its machine verifiers, his name appeared in none of them.

Read on

For the day the network goes dark, the Toosheh survival guide was written. The war's chronology is in the anatomy of the 2026 war. The full inventory of this war's fabricated material, with a source for each item, is in the deepfake case files, and the Prisonbreak network shows the other side running the identical operation. To run the same test on the language of war reporting, the war headline toolkit sets out its questions.

Related posts