This new data poisoning tool lets artists fight back against generative AI

Weekly Business Insights from Top Ten Business Magazines

Week 320 | Shaping Section | 2

Extractive summaries and key takeaways from the articles curated from TOP TEN BUSINESS MAGAZINES to promote informed business decision-making | Since September 2017 | Week 320 | October 27-November 2, 2023

This new data poisoning tool lets artists fight back against generative AI

By Melissa Heikkilä | MIT Technology Review | October 23, 2023

Extractive Summary of the Article | Listen

A new tool lets artists add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways. 

The tool, called Nightshade, is intended as a way to fight back against AI companies that use artists’ work to train their models without the creator’s permission. Using it to “poison” this training data could damage future iterations of image-generating AI models, such as DALL-E, Midjourney, and Stable Diffusion, by rendering some of their outputs useless—dogs become cats, cars become cows, and so forth.  The poisoned data is very difficult to remove, as it requires tech companies to painstakingly find and delete each corrupted sample.  Generative AI models are excellent at making connections between words, which helps the poison spread. 

AI companies such as OpenAI, Meta, Google, and Stability AI are facing a slew of lawsuits from artists who claim that their copyrighted material and personal information was scraped without consent or compensation. Ben Zhao, a professor at the University of Chicago, who led the team that created Nightshade, says the hope is that it will help tip the power balance back from AI companies towards artists, by creating a powerful deterrent against disrespecting artists’ copyright and intellectual property. 

Zhao’s team also developed Glaze, a tool that allows artists to “mask” their own personal style to prevent it from being scraped by AI companies. It works in a similar way to Nightshade: by changing the pixels of images in subtle ways that are invisible to the human eye but manipulate machine-learning models to interpret the image as something different from what it actually shows. 

The team intends to integrate Nightshade into Glaze, and artists can choose whether they want to use the data-poisoning tool or not. The team is also making Nightshade open source, which would allow others to tinker with it and make their own versions. The more people use it and make their own versions of it, the more powerful the tool becomes. The data sets for large AI models can consist of billions of images, so the more poisoned images can be scraped into the model, the more damage the technique will cause.  Zhao admits there is a risk that people might abuse the data poisoning technique for malicious uses. 

3 key takeaways from the article

1. A new tool lets artists add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways. 
2. The tool, called Nightshade, is intended as a way to fight back against AI companies that use artists’ work to train their models without the creator’s permission. Using it to “poison” this training data could damage future iterations of image-generating AI models, such as DALL-E, Midjourney, and Stable Diffusion, by rendering some of their outputs useless—dogs become cats, cars become cows, and so forth.  
3. The poisoned data is very difficult to remove, as it requires tech companies to painstakingly find and delete each corrupted sample.  Generative AI models are excellent at making connections between words, which helps the poison spread. 

Full Article

(Copyright lies with the publisher)

Topics:  Technology, Artifical Intelligence, Intellectual Property Rights