image

by Dara Morgan

Why Did Meta Disable Its Deepfake AI Tool?

I have told you before: less Instagram might be a good New Year’s resolution. Of course, that was partially a joke. The recent news, however, is considerably less funny.

On July 7, Meta introduced Muse Image, its latest generative AI model, promising a delightful new era in which we could restore family photographs, redesign our living rooms and finally see what we would look like as clay figures. Very useful. Very creative. Very Silicon Valley (as in "we are making the world a better place").

Unfortunately, the launch also included a feature that allowed people to reference public Instagram accounts in AI prompts and generate new photorealistic images using their content. In other words, strangers could type your username into Meta AI and place your face into a scene you had never entered, approved or possibly wanted to explain to your employer.

Within three days, Meta removed the feature after an enormous backlash. The company admitted that it had “missed the mark”, which is corporate language for everyone immediately understood why this was a terrible idea.

Give me more details

Muse Image is Meta’s first image-generation model developed by Meta Superintelligence Labs. It is integrated into Meta AI and also powers creative tools across Instagram and WhatsApp, with broader expansion planned for Facebook, Messenger and advertising products.

The model itself can generate images from text, edit existing photographs, combine several visual references and make targeted changes directly on an image. Meta presented examples including removing unwanted people from photographs, changing hairstyles, restoring old pictures and placing users in imaginary locations.

So far, fairly standard AI-generator territory. The problem was one particular function: users could @-mention a public Instagram account in their prompt, allowing Muse Image to reference that account’s publicly available content while creating a new image.

The account owner didn't need to approve the request. Public adult accounts were effectively included by default unless their owners changed their settings, and users weren't necessarily alerted when someone referenced their material.

Meta technically provided controls. Practically, it placed the responsibility on millions of users to discover a setting, understand a newly introduced risk and opt out before someone decided to generate an AI image of them.

A textbook example of consent, provided you define consent as “we assumed yes until you found the correct menu”.

How would the tool have worked?

A user could open Meta AI and write a prompt containing the handle of a public Instagram account. Muse Image could then use photographs, videos and other public content associated with that profile as references for the generated result.

The model was built to combine multiple visual sources, understand complex instructions and produce polished, photorealistic images. This meant the feature wasn't limited to placing a badly cropped face onto a cartoon body. It could potentially create convincing images that appeared to show a recognisable person in a completely invented setting.

That is why the word deepfake appeared so quickly. The term generally describes synthetic or manipulated media that convincingly depicts a real person doing or saying something that didn't happen.

Not every silly AI portrait is malicious, of course. There is a difference between turning your friend into a Renaissance duke and manufacturing an image designed to damage someone’s reputation. The technology, however, doesn't always understand social nuance. More importantly, neither do all the people using it.

Before Meta removed the function, public-account holders could block this type of reuse by switching their account to private or turning off the relevant content-reuse controls under Instagram’s Sharing and reuse settings. The existence of an opt-out did little to calm the situation because critics argued that the use of someone’s identity should require active permission, rather than a race to locate a toggle.

What was the reaction?

Swift, loud and unusually united.

Ordinary users began sharing instructions explaining how to disable AI reuse. Privacy and online-safety organisations warned that the function could make harassment, impersonation, identity fraud, sexualised deepfakes and sextortion easier.

Haley McNamara of the National Center on Sexual Exploitation described the feature as an obvious tool for scammers and argued that users shouldn't have to “jump through hoops” to protect their likeness.

SAG-AFTRA, the union representing actors and other media professionals, also objected. For performers, creators and influencers, a recognisable face isn't simply personal information. It is part of their professional identity, reputation and income.

The union warned its members about non-consensual digital replicas and initially urged them to alter their Instagram settings. After Meta withdrew the function, SAG-AFTRA praised the decision, saying that encouraging the creation of unapproved digital replicas was unwise and that discontinuing it was the responsible choice.

The central criticism wasn't merely that the technology could be misused. Almost any editing tool can be misused. The greater issue was the product design: Meta had connected a powerful image generator to an enormous library of identifiable people and treated access as the default.

On July 10, Meta updated its original announcement. The company said its intention had been to offer a useful creative tool while giving people control, but acknowledged that the feature had “missed the mark”. The ability to reference public accounts by mentioning them was disabled.

Muse Image, however, remains available.

Are we now safe from deepfakes?

No. We are safe from this one particularly convenient method of making them.

Meta has removed the function that allowed users to reference arbitrary public Instagram accounts through an @-mention. This closes an obvious shortcut and means your public Instagram profile should no longer work as an instant character library inside that specific Meta AI workflow.

The broader image-generation and editing tools remain active. Users can still create and alter images with Muse Image, including photographs they already possess or upload themselves. Other generative AI platforms can also produce synthetic images, and increasingly accessible tools allow people to create convincing replicas from relatively small collections of photographs.

Research into publicly available deepfake models has found tens of thousands of downloadable model variants, with women making up the overwhelming majority of identifiable targets. Many were explicitly associated with the creation of non-consensual intimate imagery.

Detection isn't a perfect solution either. AI images may contain labels, metadata or invisible watermarks, but such signals can be removed, lost through screenshots or ignored when content spreads between platforms. Meanwhile, visual clues such as strange fingers and haunted-looking teeth are becoming less reliable as image models improve.

So, what can you realistically do?

Keep sensitive photographs off fully public profiles where possible. Review Instagram’s Sharing and reuse settings, even though this particular feature has been withdrawn. Be cautious when accepting messages, requests or urgent claims supported only by screenshots, audio or video. When a suspicious image appears, look for the earliest available source rather than trusting the repost that arrived with seventeen exclamation marks.

Most importantly, don't treat realistic media as automatic proof. We have entered an era in which seeing is still useful, but believing now requires a little administrative work.

Meta’s speedy reversal is good news. It also reveals how the next stage of the AI debate will probably unfold: companies will introduce increasingly powerful tools, the public will discover the most alarming possible use within approximately twelve minutes, and “creative innovation” will once again meet the ancient concept of asking permission.

Perhaps less Instagram isn't such a bad resolution after all.