In today’s rapidly advancing digital business landscape, the roles of search and AI-powered search are shifting. While traditional search experiences often struggle with challenges like irrelevant results and inconsistent filtering, AI-powered search offers promising capabilities to better understand user intent and retrieve conceptually relevant information. At the same time, the emergence of generative AI models is presenting new possibilities to create original content.
This blog compares and contrasts features and use cases of AI-powered search versus generative AI, while also highlighting the practical considerations around cost and implementation. Understanding the strengths and limitations of these technologies is crucial for organizations looking to leverage the power of AI to improve access to information while promoting content creation.
The focus of this blog is on Enterprise search in the digital workplace and AI in relation to this.
In organisations, the traditional search experiences are rarely truly enterprise-wide, and even when labelled as an enterprise search, the search results will be limited to one or a few core data sources. These search experiences are typically keyword driven, and in some cases, users are provided with some ability to narrow down search results with filters (often called refiners).
The most common challenges for end-users of traditional search experiences include:
Rather than using the term "AI Search", we use the term AI-powered search, and like a normal search, an AI-powered search focuses on finding and presenting information based on a given query. As an aside, AI-powered search is sometimes referred to as retrieval-based AI.
Unlike a normal search however, it offers a number of helpful capabilities, including:
AI-powered search involves using special index searches - known as a vector search - to locate content that is conceptually similar and relevant. When searching for "where do I send an apple for repair", an AI-powered search might also provide you with results for your laptop warranty.
In summary Search AI algorithms analyze and interpret the input, assess the intent, match it conceptually with available data, and generate a list of results for the user that are relevant in the context.
The elephant in the room however, is time and cost. While AI-powered searches can be hugely powerful, they take time to "construct" and they are costly to run.
The analysis done by Barnacle Labs detailed in this blog by John-David Wuarin is an excellent run-down of how costly AI-powered searches can be. The analysis was based on "a small example problem, a dataset of 1M documents that are each on average 44 chunks long — each chunk is about 1000 token" (as per the referenced blog).
I won't go into the detail (and some of the detail is frankly above my head) and I also won't argue whether the research is completely accurate. Having read this article and a few others, it is however clear that we are in this general ball park, and awareness of cost and time is paramount, if you are looking to invest into this (and to set the right expectations to your stakeholders or customers).
Focusing just the cost analysis for Azure, using the Specter2 model, embedding 1 million documents would take 75 days and would somewhere between $450 and $950 (I encourage you to study the overview table in the linked blog).
Now, consider this in a scaled production environment where:
In conclusion:
As of today - in 2024 - the cost and time (and don't forget the energy consumption) involved in a fully AI-enabled search experience across all enterprise documentation is often prohibitive and impractical. The most practical and pragmatic approaches - when it comes to AI-powered search - are one of two options (or the two combined):
Generative AI, on the other hand, focuses on creating new content or generating original ideas based on a given input (also known as a prompt). Unlike Search AI, which relies on existing data, Generative AI uses Large Language Models (LLMs) to generate responses. It can be used to create text, code, images, voice, music, or even entire virtual environments.
Generative AI models are trained on large datasets to learn patterns and generate coherent and realistic outputs. They can be used for various creative purposes such as generating artwork, writing stories, or composing music.
|
By 2026, more than 80% of enterprises will have used generative AI APIs and models and/or deployed GenAI-enabled applications in production environments, up from less than 5% in 2023. Source: Gartner (https://www.gartner.com/en/articles/generative-ai-can-democratize-access-to-knowledge-and-skills) |
While both AI-Power Search and Generative AI are subfields of artificial intelligence, they have distinct differences in terms of their goals, methodologies, and applications.