em360tech image

A fake, AI-generated image appearing to show a large explosion near the Pentagon led to a brief shutter in the US stock market after being shared on Twitter. 

The image, which purported to show a pummel of smoke near the headquarters of the Department of Defense on Monday, quickly gained traction online after a string of verified accounts reposted the image, including several accounts masquerading as media outlets. 

A now-suspended verified account called “Bloomberg Feed”, was one of the first to re-share the image on the platform, spurring other outlets to share the AI hoax.

Within minutes, the Indian television network Republic TV began sharing the post with its 2.8 million followers, showing viewers the fake image on its air and citing reports from the Russian outlet RT. It later retracted its report. 

The US Department of Defence and Arlington Fire Department were quick to confirm that the image was falsified. 

“[We] are aware of a social media report circulating online about an explosion near the Pentagon. There is NO explosion or incident taking place at or near the Pentagon reservation, and there is no immediate danger or hazards to the public.,” the department wrote on Twitter. 

Despite the announcement, the hoax caused the US stock market to briefly dip as a result of reputable media outlets sharing the post. 

The false reports were first shared by the Twitter account Delaone at 10:06 am ET. By 10:10 the market had fallen 0.26 per cent before bouncing back.  

AI-generated deceit 

While it has so far not been confirmed whether the image was created using AI, Nick Waters from the news verification group Belingcat was quick to point out that the image had been manipulated using AI.

Waters noted that there were no firsthand witnesses to corroborate the event and that the building itself looked different from the Pentagon, with warped sections and an odd-looking floating lamp protruding from the pavement. 

There are many generative AI image generators including OpenAI’s DALLE-E 2, Midjourney, and stable diffusion which allow users to quickly create realistic images with minimal effort. 

Like text generators such as ChatGPT, these tools are trained on large volumes of data and real images from the internet and their own interpretation of data to create images. 

In recent months, some users have exploited these tools to create deceptive photos of public figures. One such example is the “deep fake” of the pope wearing a Balenciaga coat that went viral in March. 

Other examples are more sinister. Following the collapse of the centralized cryptocurrency exchange FTX, an AI-generated video of FTX co-founder Sam Bankman-Fried circulated shortly after and attempted to direct FTX users to a malicious site under the promise of a “giveaway.”

Since it can sometimes be difficult to distinguish AI-generated content from real content, experts are concerned that these AI that these tools could also be used to spread misinformation online. 

Just last week, the Irish paper Irish Times was forced to apologise after it accidentally published an opinion piece generated by an AI chatbot. 

A verification failure 

While AI may be to blame for the creation of the hoax image, the incident also once again highlights verification challenges for Twitter since the platform overhauled its subscription service Twitter Blue. 

Up until last month, the platform gave out blue check badges to users whose identity the platform had officially verified by reputable sources. 

Twitter previously had over 300,000 verified users under its legacy blue tick system, many of them journalists, musicians, politicians and public figures. 

But CEO Elon Musk has remained heavily critical of Twitter’s legacy system since acquiring the social media platform last October, describing Twitter’s verification as a “lords and peasants” system in a tweet last year. 

Under the new Twitter Blue program rolled out last month, Twitter now allows individuals can pay $8 a month for a blue check mark, allowing anyone willing to pay to create an account and impersonate public figures and organisations.

While this is against Twitter’s terms of service, it has not stopped people from creating such accounts – as seen with the “Bloomberg Feed” handle that spread this latest AI hoax.