In today’s rapidly advancing digital business landscape, the roles of search and AI-powered search are shifting. While traditional search experiences often struggle with challenges like irrelevant results and inconsistent filtering, AI-powered search offers promising capabilities to better understand user intent and retrieve conceptually relevant information. At the same time, the emergence of generative AI models is presenting new possibilities to create original content.
This blog compares and contrasts features and use cases of AI-powered search versus generative AI, while also highlighting the practical considerations around cost and implementation. Understanding the strengths and limitations of these technologies is crucial for organizations looking to leverage the power of AI to improve access to information while promoting content creation.
Search is a broad topic
Search is both an experience and a technology. Below are examples of different types of search experiences and technologies.
-
Public search — Publicly accessible sources such as Microsoft Bing, Google and Yahoo.
-
Website search — Search across a single or group of websites.
-
Digital commerce search and product discovery — Augmentation of digital commerce solutions to serve product discovery.
-
Enterprise search — Enterprise-wide, intranet and localized search to serve workers within a digital workplace.
-
Application search — Third-party or custom-made applications with built-in searches scoped to the application.
-
Device search — Localized to device and networked storage.
And since 2023, "AI Search" is increasingly finding itself into vision statements and requirement statements.
In the context of enterprise search (i.e. searching within an organization, whether small or large), when discussing "AI Search" we have found that it helps when everyone involved first agrees on "what is it?", and secondly, to be very clear on it's upsides and downsides.
The focus of this blog is on Enterprise search in the digital workplace and AI in relation to this.
Traditional (enterprise) search
In organisations, the traditional search experiences are rarely truly enterprise-wide, and even when labelled as an enterprise search, the search results will be limited to one or a few core data sources. These search experiences are typically keyword driven, and in some cases, users are provided with some ability to narrow down search results with filters (often called refiners).
The most common challenges for end-users of traditional search experiences include:
- Too many or too few search results.
- Quality of results is questionable.
- The refiners are not consistent or widely used due to lack of (consistent) content tagging.
- There are different search systems and experiences for different data sources and platforms.
Understanding AI-powered search for enterprise data
Rather than using the term "AI Search", we use the term AI-powered search, and like a normal search, an AI-powered search focuses on finding and presenting information based on a given query. As an aside, AI-powered search is sometimes referred to as retrieval-based AI.
Unlike a normal search however, it offers a number of helpful capabilities, including:
- It will attempt to understand your intent based on your query and for instance the context of the query (e.g. to understand whether you mean the fruit or the tech when you reference "apple")
- It will deal with typos and spelling mistakes in a similar way to how a human might (e.g. "Were do i send an aple for repar" which is perfectly understandable for a human, but causes problems for normal search queries due to the typos/misspelling.)
- It will use the whole (and improved understanding of the) query to identify content based on the overall "intent" rather than just a breakdown of the query into words with "AND" or "OR" between them.
AI-powered search involves using special index searches - known as a vector search - to locate content that is conceptually similar and relevant. When searching for "where do I send an apple for repair", an AI-powered search might also provide you with results for your laptop warranty.
In summary Search AI algorithms analyze and interpret the input, assess the intent, match it conceptually with available data, and generate a list of results for the user that are relevant in the context.
The elephant in the room however, is time and cost. While AI-powered searches can be hugely powerful, they take time to "construct" and they are costly to run.
The analysis done by Barnacle Labs detailed in this blog by John-David Wuarin is an excellent run-down of how costly AI-powered searches can be. The analysis was based on "a small example problem, a dataset of 1M documents that are each on average 44 chunks long — each chunk is about 1000 token" (as per the referenced blog).
I won't go into the detail (and some of the detail is frankly above my head) and I also won't argue whether the research is completely accurate. Having read this article and a few others, it is however clear that we are in this general ball park, and awareness of cost and time is paramount, if you are looking to invest into this (and to set the right expectations to your stakeholders or customers).
Focusing just the cost analysis for Azure, using the Specter2 model, embedding 1 million documents would take 75 days and would somewhere between $450 and $950 (I encourage you to study the overview table in the linked blog).
Now, consider this in a scaled production environment where:
- "everything" is available to search. This will typically include copies of documents, old versions, redundant versions, incorrect versions, draft versions, etc.
10 million documents would not be unusual at all - and often it will be a lot more.
- you have a modest expectation of refreshing your vector index monthly, and a weekly incremental refresh (and we estimate here a 5% volume for the increment, based on growth and changes), yielding a total of 12.2 (10 + 52/12 * 10 * 0.05) million documents monthly, costing between approx $5,500 and $11,600 per month.
- we assume that the lag of 75 days (for the 1 million documents) can somehow be managed and user-expectations can be set.
In conclusion:
As of today - in 2024 - the cost and time (and don't forget the energy consumption) involved in a fully AI-enabled search experience across all enterprise documentation is often prohibitive and impractical. The most practical and pragmatic approaches - when it comes to AI-powered search - are one of two options (or the two combined):
- Using AI to understand the semantics / intent of the user's search query which will also catch common spelling mistakes and handle grammar variations (including singular vs plural)
- Using AI to search for relevant results in a vectorised subset of all the data. Typically this is a set of curated / validated data, amounting to a fraction of the total number of documents in an organization.
Examining Generative AI
Generative AI, on the other hand, focuses on creating new content or generating original ideas based on a given input (also known as a prompt). Unlike Search AI, which relies on existing data, Generative AI uses Large Language Models (LLMs) to generate responses. It can be used to create text, code, images, voice, music, or even entire virtual environments.
Generative AI models are trained on large datasets to learn patterns and generate coherent and realistic outputs. They can be used for various creative purposes such as generating artwork, writing stories, or composing music.
Key differences between AI-Powered Search and Generative AI
While both AI-Power Search and Generative AI are subfields of artificial intelligence, they have distinct differences in terms of their goals, methodologies, and applications.
- AI-Powered Search focuses on retrieving and presenting existing information, whereas Generative AI aims to create new content based on given input.
- AI-Powered Search relies on pre-existing data and algorithms to match queries with relevant results, while Generative AI uses LLMs to generate original outputs.
Understanding these key differences is essential for leveraging the strengths of each approach and applying the right AI techniques in different contexts.
View all articles by this author
