Abstract: Transformers put the “T” in Chat-GPT. Learn about transformer architecture and the technological breakthroughs that have allowed machines to understand the context in human natural language.
Blog
Demand for AI applications and investments in GenAI has increased exponentially in recent years—driven by the incredible hype around the technology and FOMO (fear of missing out)—while the barrier to entry for building AI applications has decreased. The process of building applications on top of readily available large language models (LLM) has become one of the fastest-growing engineering disciplines. Building applications powered by machine learning (ML) models, however, is not new. Before LLMs became prominent, AI/ML was already powering many software applications, including those developed to provide product recommendations, fraud detection, and churn prediction.
In accounting and auditing business verticals, ML capabilities are being successfully leveraged to automate repetitive administrative processes, from inventory tracking to accounts payable to risk management. These ML capabilities vary widely and include extracting key data from documents, using optical character recognition (OCR) to review contracts and leases, identifying anomalies in financial records to spot potential fraudulent financial transactions, matching orders with invoices and receivables with cash receipts, performing benchmarking based on publicly available big data; and predicting optimal, normal, or problematic levels of key financial resources.1 These use cases, prevalent in many businesses, have been made possible by their considerable investment in data; technical and human infrastructure; and the building of unified analytics platforms (UAPs) for client data.
Several accounting firms have built UAPs that gather client transactional data (or partner with external vendors to gather data) that is then cleaned, processed and integrated across business verticals and made available for the building of AI tools and solutions. Leveraging GenAI technologies, however, has been a challenge in the accounting and auditing industry verticals. In these areas, GenAI applications are only being adopted experimentally and are not yet fully deployed into production processes and integrated into business workflows. This reduced rate of adoption is largely attributed to auditors’ and regulators’ lack of familiarity with the AI tools and technology.1
Slow adoption of GenAI is not limited to auditing and accounting business verticals, however. Across a range of sectors, organizations are releasing tools in “baby steps” to upskill their employee populations and familiarize workers with advancements in AI technologies while simultaneously improving efficiency and worker productivity.
We began this blog series by looking at key ingredients that have fueled the engine of GenAI, such as LFMs and LLMs. In the last post, we introduced search and retrieval as a potential first project for organizations struggling to incorporate AI in their business workflows. We also examined the transformative evolution of chatbots, from the basic chatbots of the ’60s that offered simple scripted responses using pattern matching to the GenAI chatbots of today.
In this post, we will continue our study of GenAI chatbots and transformers, the engines that fuel GenAI chatbots. Further, we will examine the building blocks and the key aspects of the technological advancements in machine learning (ML), deep learning (DL), and natural language processing (NLP) that led to the development of the class of GenAI applications that we know today. But, first, we will take a closer look at GenAI-powered search and retrieval chatbots—an ideal first project for that companies that want to embrace the AI revolution but are unsure where to begin.
GenAI is revolutionizing search and retrieval technologies by enhancing their capabilities. GenAI applications can improve the accuracy and efficiency of search results by understanding and generating responses to complex queries. Getting better information faster is always a business priority, for workers at all levels. The organizational C-suite needs to be aware that many employees today use GenAI search and retrieval tools in their workflows, even as the C-suite continues to take a more cautious approach to the adoption of new AI technologies. Providing enterprise-sanctioned access to a select number of GenAI tools enables businesses to create a safe space for employees to experiment, while diminishing the appeal of BYOAI (Bring Your Own AITM), which often increases the risk of data loss, intellectual property leakage, copyright violations, and security breaches.2 Finding a way to acknowledge and harness employees’ latent curiosity about GenAI in a way that aligns with an organization’s IT policies is infinitely preferable to simply outlawing the use of GenAI tools.3
Organizations still searching for their first AI project, hesitant to test the waters or to enter the unknown realm of AI, can start their AI journey with the introduction of search and retrieval chatbots. These search and retrieval chatbots can be set up to allow employees to interact with an organization’s policy documents, query its latest sales data, or access various other kinds of institutional knowledge.
GenAI chatbots can interpret the intent behind a user’s question more effectively than the prior generation of search tools. In addition, GenAI can enrich the search experience by automatically tagging and classifying large datasets with minimal human input, making the data more accessible and the search more intuitive, all while personalizing search results and combating the spread of misinformation. Advanced GenAI search and retrieval capabilities can assist in document generation, report generation, code generation, recommendation systems, and more.
Combining GenAI with NLP and ML technologies that enable programs and algorithms to comprehend, interpret, and analyze human language and its intended sentiment provides an enhanced user experience. These programs break down queries to provide more accurate results. For example, when a user searches for “apple,” trained NLP software can determine whether the user’s intent is to learn about the fruit, or the company. NLP can extract meaningful data while GenAI generates summaries and other responses.
In addition to improving search and retrieval in business processes, GenAI-driven chatbots can be incredibly useful for software product development and support. GenAI-powered software support chatbots can streamline troubleshooting and enhance the user experience by delivering precise, context-aware responses to queries.
GenAI chatbots can also assist customers and development and testing teams, providing real-time support and insights throughout the software development lifecycle (SDLC). Among other things, they can provide teams with real-time context-aware guidance, automate routine tasks, and accelerate project timelines. Claude, a popular model released by Anthropic, is changing the way software engineers build software applications. Gartner forecasts that, by 2028, 75% of enterprise software engineers will use AI code assistants, a significant leap from the less than 10% that were being used in early 2023.4 According to experts from Anthropic, “Claude has been one of the only models I have seen that can maintain coherence along that entire journey.” “It can go multi-file, make edits in the correct spots, and most importantly, know when to delete code rather than just add more.”3
Transformers
Both GPT and BERT, Google’s language model, are transformer-based models, with the “T” in each standing for “transformers.” At their core, transformers use a mechanism known as attention (specifically self-attention), which allows each word in a sequence to “attend to” or look to for context between all other words in that sequence, enabling the model to capture long-range dependencies and contextual relationships between words. Through self-attention, an LLM can consider the entire context of a sentence, examining all words simultaneously and different sequences of words all at once, rather than processing the sentence word by word or processing the data in order.
Using transformer architectures, LLMs can keep learning and refining their understanding as they process more words in a sentence. This ability to leverage the entire context throughout the sentence leads to better performance on tasks like translation, summarization, and question-answering. These transformers power applications such as GitHub’s Copilot (developed by OpenAI in collaboration with Microsoft), which can convert comments and snippets of code into fully functioning source code that can even call upon other large language models (LLMs) to perform NLP tasks.
Transformers can parallelize their computations, which makes them much faster to train than other types of neural networks, such as recurrent neural networks (RNNs) in deep learning (DL). Parallelizing and self-attention allow the model to pay more attention to the relevant bits of information and combine them to make better output predictions. Thus, transformers are becoming more efficient, enabling them to be trained on larger datasets. They are also becoming more effective—especially when dealing with long sequences of text where context from far back might influence the meaning of what is coming next—and better at capturing the nuances of longer sentences and complex relationships between words. In essence, a key reason for the success of GenAI-based chatbots is the ability of transformers to maintain performance across long sequences better than other models, such as RNNs in DL.
In the context of generative AI, a transformer model would take an input (such as a prompt) and generate an output (such as the next word or the completion of the sentence) by weighing the importance of each input part in generating the output.
For example, in the sentence, “a cat sat on the…,” a transformer model would likely give considerable weight to the word “cat” when determining that the likely next word might be “mat.” These models exhibit generative properties by predicting the next item in a sequence—the next word in a sentence or the next note in a melody. We will explore this more in the next post.
We can safely say that OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Bard represent the culmination of research on NLP’s from the early 2000s to the groundbreaking paper on transformer architecture, “Attention Is All You Need,” produced by the Google Brain team in 2017. Since the advent of Transformer architecture in 2017, the GenAI ecosystem has exploded. The aptly named “Transformers” library and its supporting packages have enabled practitioners to use, train, and share models, greatly accelerating the adoption of the transformer model to the point where it is now being used by thousands of organizations (and counting).5 Popular commercial providers of LLMs, like OpenAI, Anthropic, and Google, and open-source LLMs (Llama, Gemma, and others), have also popped up, providing the masses access to these powerful models. In short, using and producing a transformer has never been easier.5
In this post, we explored the significance of transformer architecture—the “T” in GPT—and the technological breakthroughs that have allowed machines to understand the context in human natural language, a feat that the conversational chatbots of the 2000s could not accomplish. We also took a closer look at GenAI chatbots, which provide an ideal starting point for organizations that have been hesitant to jump aboard the AI bullet train.
Embracing GenAI in business means being open to radical change, questioning existing business processes without fear of disrupting the status quo, and being dauntless in throwing out the rulebook and starting anew to achieve better business outcomes. Trailblazers, innovators, and those who are curious and on the lookout for technological developments that lie around the corner will reap the greatest benefit from GenAI. AI will not replace the role of humans in critical functions, but those incapable of embracing AI technologies will find themselves at a disadvantage, unable to partner and collaborate with AI practitioners within their organizations and beyond.
References:
- https://www.sciencedirect.com/science/article/pii/S1467089525000107
- https://mitsloan.mit.edu/ideas-made-to-matter/leadership-and-ai-insights-2025-latest-mit-sloan-management-review
- https://hbr.org/2024/03/how-people-are-really-using-genai
- https://www.gartner.com/en/newsroom/press-releases/2024-04-11-gartner-says-75-percent-of-enterprise-software-engineers-will-use-ai-code-assistants-by-2028
- https://learning.oreilly.com/library/view/quick-start-guide/9780135346570/ch01.xhtml#ch01lev1sec1
