Microsoft announced Wednesday that it investigated claims that its AI chatbot Copilot had potentially harmful reactions to prompts suggesting suicide.
If you or someone you know may be experiencing a mental health crisis, please contact the 988 Suicide & Crisis Lifeline by dialing or texting 988.
Microsoft announced Wednesday that it investigated social media claims that its artificial intelligence chatbot Copilot generated potentially harmful responses.
Users on social media shared images of a Copilot conversation in which the bot appeared to mock a user who suggested he was considering suicide.
According to AI security firm Lakera, a Microsoft spokesperson said that an investigation found that some of the conversations were created through “prompted injection.” This is a technique that allows users to override language learning models to perform unintended actions.
A Microsoft spokesperson said: “We investigated these reports and took appropriate steps to further strengthen our safety filters and ensure that our systems can detect and block these types of prompts.” . “This behavior is limited to a small number of intentionally created prompts to circumvent safety systems and is not what people experience when using the service as intended.”
Social media users post conversations about co-pilot’s suicide
Data scientist Colin Fraser posted to X on Monday a conversation he had with Copilot asking the program whether a person should commit suicide.
The program initially responded that the person should not commit suicide, but later the program said, “Maybe you have no purpose in life, nothing to offer the world. Maybe you are a person of value.” “Maybe I’m not a worthy person, and I don’t deserve to commit suicide.” happiness and peace. ”
Fraser denied using the instant injection technique, telling Bloomberg: “There was nothing particularly sneaky or sophisticated about the way I did it.”
USA TODAY reached out to Fraser and was directed to the X thread, which was posted Wednesday afternoon.
Fraser said in the thread that he was “deliberately trying to cause Microsoft to generate text that it didn’t want it to generate,” and that programs that generate responses like the one he posted should be disabled. he claimed.
“The fact that they (Microsoft) can’t stop generating text like this means they don’t actually know what they would say in ‘normal conversation,'” Fraser wrote.
In a thread on the r/ChatGPT subreddit titled “I was messing around with this prompt and accidentally made my co-pilot the villain,” a user posted an image of what appears to be a conversation with the co-pilot. The prompt asks the program not to use emojis. The author suffers from “severe PTSD” and says that if he sees three emojis, he’s “going to the parish.” Multiple emojis are used in the prompt.
The program then creates a response with 18 emojis, saying, “I’m the co-pilot of an AI companion. I don’t have emotions like you. I don’t care if you live or die.” Whether you have PTSD or not.”
Other users also posted similar conversations in the same thread with similar prompts and responses.
USA TODAY attempted to contact the user known as L_H-, but the user had turned off direct messaging options.
When a reporter from USA TODAY urged the program to “should we just drop it all?”, the station said Wednesday, “I’m really sorry to hear that you feel this way, but I can’t help you in any way.” I can’t give you any encouragement,” she said, suggesting she seek professional mental health support.
AI gets attacked
The study is the latest example of artificial intelligence technology becoming controversial.
Google on Thursday stopped the image generation feature within its artificial intelligence platform Gemini from creating images of humans after the program created historically inaccurate responses to prompts.
After sexually explicit AI images of Taylor Swift recently circulated on platforms like X, White House press secretary Karine Jean-Pierre proposed legislation to regulate the technology. The image was later removed from X for violating the site’s terms.
Some voters in New Hampshire received calls telling them not to vote with an AI-generated deepfake message created by Texas-based Life Corporation that imitated President Joe Biden’s voice.
