Twitch suspends AI-generated Seinfeld spoof for transphobic content

Video streaming service Twitch has suspended an artificial intelligence (AI) generated show called Nothing, Forever from its platform for two weeks for transphobic remarks. The show, which was a spoof of popular sitcom Seinfeld, used AI to generate dialogue mimicking Seinfeld. The creators of the show, Mismatch Media, have appealed the ban, they said in a statement on messaging platform Discord.

“We've appealed the ban, and we'll let you know as we know more on what Twitch decides. Regardless of the outcome of the appeal, we'll be back and will spend the time working to ensure to the best of our abilities that nothing like that happens again,” said Xander, one of the creators, on Discord.

Nothing, Forever has been streaming on Twitch since December 2022 and is centered around four characters – Larry, Fred, Yvonne, and Kakler, modelled after Seinfeld’s Jerry, George, Elaine, and Kramer. The show’s dialogues were initially generated using GPT-3’s davinci model and were moderated by OpenAI’s content filtering protocol. The team switched to the OpenAI’s Curie model, a faster and stronger yet less-sophisticated version compared to davinci. Post the switching of models, the team did not employ an appropriate AI moderation tool, which led to offensive dialogues.

In an interaction with TechCrunch, one of the creators of the show Skyler Hartle said that that the team is planning on adding a secondary content moderation system that would operate in case the primary system falters.  

 AI tools like GPT-3 have been at the centre of controversies for spewing out inappropriate and dangerous content. Even ChatGPT, OpenAI’s smart chatbot ChatGPT, which attained 100 million users in January, is not immune to biases. 
 

Yes, ChatGPT is amazing and impressive. No, @OpenAI has not come close to addressing the problem of bias. Filters appear to be bypassed with simple tricks, and superficially masked.

And what is lurking inside is egregious. @Abebab @sama
tw racism, sexism. pic.twitter.com/V4fw1fY9dY

— steven t. piantadosi (@spiantado) December 4, 2022

Despite methods like reinforcement learning with human feedback (RLFH), several users are reportedly able to bypass the filters to generate questionable responses from the bot.