AI-powered Bing made errors in first demo too, claims researcher

US-based artificial intelligence (AI) researcher Dmitri Brereton has claims that Microsoft’s new AI-powered Bing search engine made several errors during its demo last week and generated made-up information.

Unveiled on February 7 by chief executive Satya Nadella, the new Bing uses Generative Pre-trained Transformer 3 (GPT-3) and promises ChatGPT-like experiences to users within the search. 

During the unveiling of AI-powered Bing, Yusuf Mehdi, consumer chief marketing officer, also gave a demo of how it will carry out searches and answer user queries. 

“It made several mistakes during the demo, much worse than Google’s Bard mistake,” said Brereton in a Twitter post. 

He pointed out that the AI-powered Bing gave up information about the “Bissell Pet Hair Eraser Handheld Vacuum” and cited sources, which doesn’t include anything about “limited suction power”, “short cord length”, or it being “noisy and may scare some pets”. 

Further, Brereton claims that Bing AI erred again while summarising a financial document and got most of the numbers wrong. “According to Bing, Gap Inc. reported an operating margin of 5.9%, even though 5.9% doesn’t appear anywhere in that document,” said Brereton. 

“It’s important to note that last week we announced a preview of this new experience. We're expecting that the system may make mistakes during this preview period, and user feedback is critical to help identify where things aren't working well so we can learn and help the models get better. We are committed to improving the quality of this experience over time and to make it a helpful and inclusive tool for everyone," said a Microsoft spokesperson. 

Brereton has also published a blog post summarising all the errors made by the new Bing during the demo.  

“Bing AI did a great job of creating media hype, but their product is no better than Google’s Bard. At least as far as we can tell from the limited information we have about both,” said Brereton in the blog post. 

AI-powered Bing isn’t the only transformer-based search and chat platform that has made errors in its demo. Last week, Google unveiled its AI chatbot Bard, based on Language Model for Dialogue Applications (LaMDA), which made an error during the demo. Bard claimed in the demo that the James Webb Space Telescope was the first to take pictures of a planet outside of Earth’s solar system, which was immediately flagged as inaccurate by space experts. 

Google parent Alphabet’s shares fell 7.7% last week wiping $100 billion off its market value after the error by Bard was detected and reported by Reuters.