Generative AI (GenAI) has proliferated across the corporate world since the launch of OpenAI’s ChatGPT-3.5 in fall 2022. ChatGPT utilizes a chatbot interface to facilitate user access to the underlying GPT (generative pretraining transformer), a large language model (LLM) now in its fourth generation. This interface enables the public to interact with the LLM through natural-language conversations. 

GenAI’s projected impact on global economic growth over the next ten years ranges from $2.6 trillion (Chui et al., 2023) to $7 trillion dollars (Goldman Sachs, 2023). The emergence of GenAI chatbots is revolutionizing various industries. For example, in the banking industry, JPMorgan has a suite of internal AI chatbots that are used for day-to-day tasks and it utilizes AI models to help investment portfolio managers spot investment opportunities.¹ Goldman Sach aims to use AI to cut down on rote tasks, like writing financial documents, by 95%.¹ Morgan Stanley has rolled out similar AI-powered tools for everyday productivity, including Debrief, which specializes in taking notes during client meetings. Meanwhile, Wells Fargo has a customer service AI chatbot for its retail customers.¹

Despite the plethora of new AI-driven products and business models, however, most companies struggle to understand the full scope of AI’s potential or use it to implement company-wide solutions. According to a recent study by Boston Consulting Group²:

• Seventy-four percent of companies have yet to show tangible value from using AI.
• Only 4% of companies have developed cutting-edge AI capabilities across functions and consistently generate significant value. 
• An additional 22% have implemented an AI strategy, built advanced capabilities, and are beginning to realize substantial gains. 

How can an organization get started with GenAI?

Search and retrieval is an area where GenAI technology can deliver immediate impact. Implementing GenAI-powered search and retrieval capabilities across all business verticals is a potent way to put GenAI into action. These capabilities can be made accessible to employees through chat-based tools that allow them to easily query policy documents, conduct quick Q&As using the organization’s latest sales data, or have meaningful conversations involving all manner of institutional knowledge. And these functions are only the beginning. Advanced GenAI search and retrieval capabilities can also assist in document generation, report generation, code generation, recommendation systems, and much more.

Building GenAI tools for search and retrieval is an effective way to demonstrate the value of the technology. Further, doing so will help establish the technological data and AI infrastructure required for a broader-scale, long-term digital transformation. 

To understand the foundations of search and retrieval technology, we first need to start with the basics of natural language processing (NLP).

Natural Language Processing Models

From the emergence of the earliest statistical models to the birth of the first large language models (LLMs),³ the evolution of models that process (and understand) language has been a story of constant invention and improvement. Below, we explore the technological advancements driving this chatbot evolution, look at the key differences between basic conversational agents and GenAI chatbots, and discuss how businesses can harness the potential of NLP models to improve productivity and decision-making. 

Basic Chatbots: Chatbots are computer programs designed to mimic human conversation. First conceived at MIT in 1966, chatbots have evolved from their humble beginnings—when they offered simple scripted responses using pattern matching—to become complex conversational agents powered by GenAI.  According to AI pioneer Andre Nguyen, research into early NLP machine translation and speech recognition capabilities funded by the US Department of Defense and intelligence agencies over the last few decades paved the way for new AI subdomains transcending just translation.

Conversational agents: Advancements in ML led to the rise of conversational agents in the early 2010s. Conversational agents use advanced NLP and ML capabilities to understand natural language more accurately than basic chatbots. Conversational agents that serve individuals, as opposed to teams, departments, or companies, are known as virtual digital assistants. Common examples include IBM Watson, Amazon’s Alexa, Google Now, Microsoft’s Cortana, and Apple’s Siri.⁴

Conversational agents—which are also referred to as traditional AI-based chatbots or rule-based chatbots—can learn from past interactions, understand voice commands, and perform tasks.  They require meticulous step-by-step training. The process begins with the creation of a knowledge base with large datasets to detect patterns, create rules, define prompts. Then the bot is trained to recognize these prompts. 

Conversational agents have been successfully deployed in the customer service industry. Businesses have quickly implemented these chatbots to help reduce costs, improve efficiency, and provide around-the-clock customer service support through the generation of responses to customer inquiries. Conversational agents that serve individuals, as opposed to teams, departments, or companies, are known as virtual digital assistants. Common examples include IBM Watson, Amazon’s Alexa, Google Now, Microsoft’s Cortana, and Apple’s Siri.⁴

The downside? The chatbot is only as good as its training data set and instructions. Chatbots are hand-crafted, which is a highly time-consuming process. It can take years to train a chatbot in a particular area of expertise. They are limited to the knowledge base provided to the model and can only regurgitate the information on which they have been trained. In order to keep chatbots current and relevant, their training data must be constantly updated, which requires significant resources. If a chatbot encounters a question or scenario outside its pre-defined parameters, it can only say, “I do not know.”⁴ 

Conversational chatbots have found extensive use in the healthcare industry, where they assist health professionals by providing patients with personalized summaries tailored to the user search medical term based on medical keywords. Based on the query entered into the chatbot, the user is informed of the recent literature and developments in personalized medicine. Popular examples in this realm include VDMS (Virtual Diabetes Management System), built to assist both diabetes patients and the general community with diabetes education and management; SHIHbot, built by Facebook to answer a wide variety of sexual health questions about HIV/AIDS; and Forsky, which analyzes patients’ eating habits and supports users in improving their dietary health.⁴

GenAI chatbots: Unlike traditional chatbots, ChatGPT does not rely on preprogrammed responses or rules, and GenAI-powered chats are not limited only to text. GenAI chatbots can provide a multi-modal experience to the user by generating new content in formats such as audio, video, and images. They do not just regurgitate information but instead deliver new creative content via a conversational interface that understands human language and semantics. The advanced capabilities of GenAI chatbots have attracted a continuous stream of new users, generating widespread interest in GenAI-enabled applications and laying the foundation for broader-scale and long-term transformation.

With planning and due diligence, GenAI implementation can work for most businesses. Integration with legacy systems presents challenges, however, both in terms of functionality and cost implications. Software, hardware, staffing, training, and research all take time and can incur a significant upfront cost that might not be fully recouped for months or years. Once GenAI chatbots are implemented in the workplace, user over-reliance on the search tech can also lead to security concerns and other issues.

GenAI chatbots are based on complex neural-network architectures, advanced natural language processing (NLP) techniques, and deep learning (DL) algorithms. GenAI applications like ChatGPT learn from a variety of existing training data—including text, images, speech, structured data, 3D signals, and even videos—to generate novel outputs in various formats (Gozalo-Brizuela & Garrido-Merchan, 2023; Kietzmann & Pitt, 2020; OpenAI, 2022). Following ChatGPT’s release, major tech giants quickly grasped the potential of GenAI technology and have been racing to develop their own GenAI chatbots and LLMs. For example, Google launched Bard, with the underlying language model evolving from the LaMDA family to PaLM and Gemini; Microsoft introduced Copilot, initially known as New Bing Chat and based on GPT-4; and Meta introduced LlaMA, now in its second generation (Singh, 2023).

Unlike conversational agents that could only regurgitate information, the new breed of GenAI chatbots can leverage vast knowledge bases to understand context; provide dynamic, real-time solutions for complex user queries; and generate human-like responses. What fueled this revolutionary shift?

Transformers, a type of advanced artificial intelligence (AI) deep learning architecture, are the bedrock of foundational models and are responsible for their context awareness and language capabilities. The Google Brain team originally introduced the transformer in the paper, “Attention Is All You Need” by Vaswani et al. in 2017. Since then, transformer-based models have become state-of-the-art for many tasks in industry and academia.⁵ 

Search and retrieval, intrinsic to every facet of business, can be a good first project for organizations struggling to incorporate AI into their organizational practices and business processes. Chatbots fueled by GenAI can fundamentally change how an organization interacts with its data. These chatbots can unlock insights, spark new ideas, inform better decision-making, and improve employee productivity by reducing the time spent sorting through copious amounts of data.

In this post, we focused on the evolution of chatbots from the scripted applications of the past— which handled basic customer service inquiries but struggled to understand human natural language—to the intelligent machines of today that leverage GenAI. In the next post, we will delve more deeply into the role of transformers—the T in ChatGPT. We will continue learning more about GenAI chatbots and the role of transformers in future posts in this blog series.

Embracing GenAI in business means being open to radical change, questioning existing business processes without fear of disrupting the status quo, being dauntless in throwing out the rulebook, and starting anew to achieve better business outcomes. This means it will benefit the trailblazers, the innovators, and the curious the most, those on the lookout for technological developments that lie around the corner. AI will not replace the role of humans in critical functions, but those that are not able to consider AI technologies as a partner and unable to collaborate with AI practitioners.

References:

1. https://fortune.com/2025/01/17/goldman-sachs-ceo-david-solomon-ai-tasks-ipo-prospectus-s1-filing-sec/
2. https://www.bcg.com/press/24october2024-ai-adoption-in-2024-74-of-companies-struggle-to-achieve-and-scale-value
3. https://learning.oreilly.com/library/view/building-llms-for/9798324731472/index_split_009.html
4. https://www.techtarget.com/searchcustomerexperience/infographic/The-evolution-of-chatbots-and-generative-AI
5. https://learning.oreilly.com/library/view/quick-start-guide/9780135346570/ch01.xhtml#ch01lev1sec1