Saeed Elnaj, a veteran healthcare CIO, shares lessons learned from two years of implementing enterprise GenAI products about what works and what doesn’t.

There are already indications that Generative AI (GenAI) will have a significant impact on business, the economy, and society at large. There certainly may be some excessive hype around claims that the technology will be “bigger than the Internet.” What is clear, however, is that GenAI offers tremendous opportunities along with some serious challenges. Legal considerations, privacy and security concerns, toxicity, and hallucinations top the list.

GenAI is in its early phases but evolving with rapid innovation and large investments. A diverse group of companies are competing for primacy, from startups to well-established tech giants. According to Gartner’s 2024 technology hype cycle, GenAI is just over the peak of inflated expectations, which had driven some healthy skepticism around it. 

The question that IT leaders are trying to answer is what we should do until the technology matures, delivering serious productivity gains and disappearing into the background. Executives want to know when to invest, in what, and how. In this blog, I’ll share my experiences and lessons learned from implementing GenAI proofs of concepts and projects.

Applying a use case lens

I’ve found a helpful approach to exploring, evaluating, selecting, and implementing GenAI is examining these opportunities through a use case lens. These fall into three broad categories: horizontal, vertical, and specialized.

  • Horizontal use cases: These GenAI opportunities are applicable to most, if not all, industries and enterprises. In fact, such GenAI capabilities are already implemented in many applications including office productivity tools and enterprise software. These horizontal use cases have the potential to significantly improve productivity. They provide suggestions, take notes, and summarize documents and meetings, for example.

Within the horizontal category, the code generation use case is already yielding good results. Software developers reported working 55% faster using GenAI tools, according to a GitHub study.

With the proper change management and GenAI guardrails in place for security and legal ramifications, these horizonal cases should be evaluated, implemented, and ingrained within enterprises across industries and geographies.

  • Vertical use cases: Here we have the opportunity to implement GenAI to transform specific business processes or offer new customer experiences within an industry vertical. This is where IT executives need to work alongside business executives to identify and experiment to determine which will deliver meaningful business outcomes and value.

The Molecule to Market (M2M) value stream to create and launch new drugs is a prime target for GenAI-driven innovation. Bringing a new drug to market takes 10 to 15 years, costs $1 to 2 billion, and has a 90% failure rate. GenAI will play a critical role in transforming M2M, as I’ve observed in one proof of concept. 

In the earliest days of GenAI, I used a simple evaluation spreadsheet that included the following questions to evaluate vertical use cases:

  • Is there enough well-curated, readily ingested, and ready-to-consume data? Use cases with significant data that are not sensitive and do not cause privacy risks are the first to consider.
  • What is the expected business value or measurable business outcome to be achieved by implementing the use case? The bigger the impact, the better.
  • What risk, legal, and compliance issues could the use case trigger that would delay implementation? Use cases with lower risks jump to the top of the list.
  • What is the complexity of the use case? Those that require data ingestion from multiple data sources and in multiple data formats that might require different large language models (LLMs) are difficult to handle especially when data ownership resides within different lines of business. These use cases tend to require more resources and greater coordination and therefore may make more sense for an enterprise that has gained experience with multiple GenAI use case implementations.
  • Do we have the right resources — human and financial — to dedicate to the proof of concept or project?
  • Specialized use cases: These opportunities are specific to the value stream of an enterprise. For example, within the overarching M2M value stream process, each biopharma company will have a specific set of data and ways to identify molecule targets for drug discovery. Some of these companies are already using their proprietary and unique set of genomics data combined with longitudinal electronic health record data to link gene mutations to diseases and drug discovery. Some biopharma companies, for example, are using  NVIDIA’s BioNeMo GenAI platform to accelerate the building and training GenAI models using proprietary data for drug discovery.
Of course, there can be exceptions to the above use case considerations. There may be opportunities to implement GenAI that are very complex and whose business value is hard to quantify that still bear pursuing. For instance, we are seeing GenAI transforming customer experiences by enabling users to engage with data via chatbots. In some cases, it’s hard to nail down the business value of such implementations. Yet, such projects have been successfully implemented for competitive, customer retention, and new customer acquisition reasons, as the blog SpiceWorks noted.

 

Related article:

Generative AI: 4 Questions CIOs Should Ask

By Anil Cheriyan

 

4 Lessons From My GenAI Projects

I’ve learned an enormous amount about what works and what doesn’t during my time working on GenAI proofs of concept and projects over the last two years. But the following lessons stand out:

  • Many enterprises are implementing horizontal GenAI use cases, but true differentiation and transformation come from vertical and specialized use cases. Such use cases require teaming up with the business executives to identify, select, prioritize, and implement them, applying the above-mentioned rubric.

  • Taking an iterative approach to GenAI governance is best. As a relatively new technology, GenAI requires IT to build specific governance and enterprise processes around it. However, slowing down the implementation process to figure out the full enterprise GenAI governance process is not ideal. It is hard, time consuming, and resource intensive — and the technology is still evolving. Building such governance iteratively with each use case implementation, employing large language model operations (LLMOps) and Foundation Model Ops (FMOps) practices, is a prudent approach. 

  • There is tremendous work required to understand the user experience and the questions that the users are trying to answer. One of the critical architectural frameworks for GenAI implementations is Retrieval Augmented Generation (RAG), which requires the ingestion of proprietary and specialized data with the LLM model to increase accuracy, reduce hallucination, and provide up-to-date information. Many vendors claim that you could use their LLM, point it to your data and — voila — you’ve implemented a proof of concept! The reality is far from this.

    The user interface  should be built to cater to the user experience; the LLM ought to be able to understand the user context and provide highly accurate answers in the expected format. Enterprises that are most successful will build a GenAI “conveyor belt” whereby new vertical use cases with specialized data can be implemented with the help of reusable components (processes, procedures, software tools, scripts). This approach can help jumpstart a project. Critical to the “conveyor blet” strategy is having LLMOps that streamlines the development and deployment life cycle of GanAI use cases, starting with data preparation, data management, and model training. Automated tools and standardized workflows reduce manual tasks and optimize resource utilization. The goal is to continuously improve the development and deployment life cycle.

  • Prompt engineering is a critical success factor. Guiding the LLM to answer the specific question accurately requires providing well-defined context. Prompt engineering is already emerging as an important field in GenAI, and we found it crucial to our efforts. Skilled professionals that are experienced in both business domain and prompt engineering techniques are essential to the success of GenAI projects.

It’s still early days. However, GenAI technology cannot and should not be overlooked. It will bring about transformations in business processes and in businesses themselves. New business will emerge, old ones will mature, and others will decline or even disappear as GenAI’s transformative powers take hold. We IT executives will play a lead role in determining whether our organizations fall on that spectrum.

 

Roles We Recruit


 

Read our weekly e-newsletter packed with career advice and resources for the strategic technology leader, and information about active searches.

The Heller Report

Add a Comment

The Impact of the Tech C-Suite Expansion

Oct 23, 2024

My CIO Career: Arthur Phidd, CIO at Reeds Jewelers, on the Transformation Principles that Work Across Industries

Oct 9, 2024