Are AI-based Search Engines ready for Prime Time

Sam Altman-led OpenAI became the latest entrant into the artificial intelligence (AI)-powered search engine market with the launch of SearchGPT. Still in the prototype stage, it combines AI models with web information to provide fast, timely answers with clear and relevant sources. The company wants to integrate its features into ChatGPT in the future.

The AI-powered search engine market is highly competitive, with major players like Google, Microsoft, and Meta already making significant strides, alongside innovative startups like Perplexity AI.

Despite its potential, the market faces several challenges. The question remains: will AI-powered search make a significant impact or fade into obscurity after a short blip?

Plagiarism and tussle with publishers

Perplexity AI, widely promoted as the worthy replacement for Google Search, has been under the scanner for plagiarising content from news channels and paywalled websites. Conde Naste which owns publications like The New Yorker, Vogue, and Wired, has sent a cease-and-desist to the AI search company asking it to stop using its content in the search results, a report by The Information said.

This is not the first time Perplexity AI has faced an accusation of unethical web scrapping. In June, Forbes said that the company allegedly plagiarised one of its articles in its beta Pages feature. A report by Wired in June found that the AI company had violated the Robots Exclusion Protocol (REP) which is a set of rules that is used by websites to control how crawlers access their content. 

Aravind Srinivas, CEO and co-founder, responding to the plagiarism accusations denied that violating REP. In a Fast Company interview, he stated that REP is not a legal framework and called for a new setup between publishers and AI-driven sites. Perplexity AI on Tuesday launched a revenue-sharing model for publishers with several media outlets and content platforms including Fortune, Time, Entrepreneur, The Texas Tribune, Der Spiegel and Wordpress.com under its ‘Publishers Program.’ 

“We have always cared about giving attribution to content and have designed our product from the beginning to clearly cite its source materials, which most chatbots are unable to do reliably and prominently even today. Following the feedback from Forbes, our team worked overnight to ship an update so we present sources on Pages in the same format as the answer engine,” Perplexity AI told TechCircle in an email response to queries. 

The company is valued at over a billion dollars after it raised over $62 million in its fourth round of funding in April. Its backers include billionaire Stanley Druckenmiller, Y Combinator CEO Garry Tan, Amazon founder Jeff Bezos, and Nvidia Corp. Perplexity AI prefers to refer to itself as an ‘answer engine’ which is “in the business of finding information for people who seek it and presenting it to them in a way that makes sense”.

The platform processes millions of questions in a day but current events or news accounts for just 3% of the queries, the spokesperson said. According to recent company updates, AI model prompts now reference news outlets. Third-party providers used by the search engine will also be mandated to follow REP guidelines, ensuring they do not access restricted content from news publisher sites.

Alleged unethical scrapping and plagiarism are hardly unique problems faced by just Perplexity AI. The Japan Newspaper Publishers and Editors Association on July 17 released a statement rapping organisations such as Google, Microsoft, and Perplexity for copyright violation and “freeriding on the labor of news outlets”. 

Hallucinating search engine and enshittification

Last week, Microsoft’s Bing introduced generative AI in its search results, which is currently available to a limited number of users. Microsoft follows Alphabet-owned Google’s footsteps which launched generative AI capabilities in its Search feature under the AI Overview banner earlier this year. This feature aggregates information to save users’ time. Google is also planning to integrate ads into AI Overview. Notably, Google Search is one of the major growth drivers for the company generating significant revenue—$48.5 billion out of the total $85 billion.

Despite its potential, Google's AI Overview has faced issues. Shortly after launch, it produced bizarre and inaccurate information, such as advising users to add glue to pizza. Following user backlash, Google temporarily disabled AI Overviews for certain searches to address these errors.

“AI processes vast amounts of data, identifies patterns, and synthesizes information. An AI search engine crawls the internet, reads, and assimilates content from millions of websites. When a user submits a query, the AI connects the dots and presents synthesised information directly, rather than pointing to specific resources,” said Rishi Agrawal, co-founder and CEO, Teamlease Regtech.

“But since such AI systems are synthesising information from various sources it may include false and misleading information. Unlike traditional search engines that references specific links, AI synthesises and presents this information directly, without revealing its sources. This shift makes it easier for fake information to influence AI training and outcomes.”

"Enshittification", coined by Cory Doctorow in 2023, describes the degradation of online platforms due to the over-promotion of ads and sponsored content. Kashyap Kompella, CEO, RPA2AI Research extends this phenomenon to include AI-generated content. “We're already dealing with clickbait and SEO-driven content, but AI-generated content in search results compounds the problem. This synthetic content is twice removed from originality, degrading search quality. To address this, search engines should exclude AI-generated content from their indexes and results until they find a broad solution to this content enshittification."

Not all is lost, yet

As consumer habits continue to evolve, it is unfair to completely write off AI-based search engines, especially when it is still at a nascent stage, believe experts. Kompella said that despite the shortcomings, the climate is ripe for disrupting the search experience. 

“These tools offer a good user experience by finding specific information quickly. Traditionally, we research topics by consulting multiple credible sources and synthesising the information ourselves. Now, tools like Perplexity AI, Bing AI, and Google AI Overview propose eliminating that step by doing the synthesis on our behalf. This is an intriguing proposition and can be highly valuable for certain needs,” he said.

Search is not a single monolith, says Jayanth Kolla, founder of Convergence Catalyst. “Search is now segmented by user intent across different platforms. For example, Amazon leads product searches, Google Maps dominates local searches, and large language models (LLMs) (such as Perplexity AI and ChatGPT) are used for information-based searches. This diversification is gradually eroding Google's market share, even though it still holds an elephant share.” He further says such platforms are still in the early stages and must deal with these challenges before positioning themselves as profitable business models.