These data leaders agree that GenAI is a massively impactful technology, one that will change every aspect of life as the internet did. The advantage will go to those who are not limited by the constraints of how we use technology and data to affect how we work, learn, and collaborate today.
The Gartner Data & Analytics Summit 2024 gathered a stunning group of market and technology leaders into a common conversation. The Summit’s agenda had set the stage for discussing generative AI (GenAI): Is it enterprise-ready? Are organizations GenAI-ready? What is the path forward, with an emphasis on preparation and taking the first step?
To collect individual perspectives from vendor executives. RTInsights asked four questions to assess the current state of GenAI adoption as they are experiencing it and catch a glimpse of what they are seeing on the near horizon within their customer base.
- What is the impact of market demand for GenAI on your business?
- What are your customers’ top requirements for meeting their own GenAI objectives?
- Do you have customers seeking to train publicly available LLMs on their own data? What is their biggest challenge?
- What are the real-time or data-in-motion aspects of GenAI implementations, and how are they being addressed?
Their answers are a snapshot of a market that embraced a new technology and is now seeking to reinvent customer interactions, increase worker productivity, and accelerate invention and innovation. While enterprises are still very bullish on GenAI and moving forward with expansions, there are some hurdles to overcome. Already, new technologies and best practices, such as Retrieval-Augmented Generation (RAG) and prioritizing data governance, are emerging to handle them.
Reading through executives of data-focused software providers is like looking at the GenAI phenomenon through a prism. Their answers are shaped by the challenges they address, but taken together, they give us a comprehensive view of this significant moment when business and technology are rapidly and impactfully shifting.
The benefits balance the manageable risks
Dr. Ingo Mierswa, SVP of Product Development, Altair
There was never a bigger and more disruptive breakthrough in the data science and machine learning market than Generative AI. It is so disruptive because I and any other data scientist out there will never go back to performing our work how we used to.
There is the obvious reason that we can finally tackle use cases that require complex, human-like outputs. So far, the best we could offer is a “class” or a “number” as model output. But now we can not only calculate the probability that a client is going to churn. But we can also generate the email which will make her change her mind.
Generative AI allows us better than any other technological advancement to tap into the so far mostly unused data potential which is unstructured. Maybe as much as 99% of all our data is unstructured in nature. It comes in the form of web pages, documents, images, audio, and video streams. And, like humans who tend to prefer using data in these formats, so does Generative AI truly shine when making sense of these data sources.
This is the reason why we see a massive amount of chatbots on proprietary documents as well as emerging technologies such as Retrieval-Augmented Generation (RAG). Next to [minimizing] hallucination, RAG became so important: it can still use the world model and the generative capabilities of Gen AI to provide content. But it can also be used to feed more recent data into the model as context, so to say in real-time. This avoids the need to retrain those large models frequently just to keep them up to date.
Executive accountability for AI policies
Matt McLarty, CTO, Boomi
Every company is trying to figure out their place in the AI economy. How will AI affect their business models? Their staffing? Their bottom line? The topic is very much on the minds of the C suite, so it’s not surprising that companies are appointing Chief AI Officers to set policy and governance around the adoption of AI within the enterprise. I worry that this role could create some confusion and ambiguity since the impact of AI is so wide-ranging.
It’s vital that organizations establish AI policies, but that could be handled by CIOs and CISOs. Perhaps the best way to handle executive direction around AI is to extend the role of the CDO (either Chief Data Officer or Chief Digital Officer) since their mandate will typically align with the upside of AI. Regardless of executive accountability, the most successful operating model will be one that enables strong communications across all affected business units and functions and one that enables the integration of existing data and software capabilities with new AI technologies.
Flexibility is a top customer requirement
Kjell Carlsson, Head of AI Strategy, Domino Data Lab
…The key to getting value from GenAI is to use it in tandem with traditional ML models. It turns out there is no such thing as a pure GenAI application. It is invariably faster, cheaper, more accurate, and often more secure to create an integrated pipeline that decomposes the problem into the tasks that use predictive models and LLMs for what they are each best at. Because of this, the demand for GenAI really grows the need for platforms that can support all types of AI and ML.
[Customers want] flexibility to leverage both proprietary and open-source LLMs, to be able to use LLMs offered as a managed service, and to host LLMs themselves. Next, they want the ability to leverage all of the methods needed to implement GenAI pipelines, e.g., leverage retrieval augmented generation (RAG), the ability to fine-tune models, orchestrate complex GenAI pipelines, etc. Further, they want the ability to audit, govern, and manage the costs associated with GenAI. For example, they want to be able to ensure that sensitive data is not leaking to third parties (e.g., the ability to audit prompts), to ensure that their LLMs are not providing inappropriate outputs (e.g., by logging results), and limit cost (e.g., by monitoring and limiting the amounts spent calling third party hosted LLMs).
Bad data delivers bad GenAI
Brendan Grady, GM, Analytics Business Unit, Qlik
The top requirement to meet any of these objectives remains the same: good data. The best AI outcomes start with the best data. Enterprises are confronted with a massive and ever-increasing volume of data, and ensuring those insights are accurate, governed, and of high quality is more critical than ever. To do this, a modern data fabric is essential.
[Qlik’s 2023 Generative AI Benchmark Report provides a quantitative look into the state of the market. For example, only 56% of organizations with a GenAI tool or project have a formalized AI strategy.]
Qlik reported that 68% of organizations plan to leverage public or open-source LLMs, augmenting them with their own proprietary data. The biggest challenge with this method is many organizations’ data houses aren’t up to snuff. According to the same report, a staggering 73% of organizations say their data fabric needs upgrades, and at present only supports generative AI somewhat well to somewhat poorly or very poorly.
Organizations must invest in their data fabric, data mesh, and real-time data analytic pipelines to reap the most benefits from their generative AI implementations.
A clear view of GenAI benefits and challenges ahead
Don Murray, President and Co-founder, Safe Software
GenAI has captured the imagination of many of our customers, and so they look to us to use AI to help them in three ways.
- Use AI to improve their productivity when creating and updating integration workflows.
- Use AI in their workflows to gain insights that otherwise would be very difficult or not economically feasible.
- Use AI to improve the administration tasks of our products so administrators can better anticipate where resource constraints are going to impact serving customers.
Their biggest challenge seems to be not getting locked into one solution, and so we see clients looking for approaches where they could pivot if they decide to move from OpenAI, for example. Security is another big challenge, and we are seeing our clients evaluating solutions that they host and have control of the data. Organizations are very concerned about their data being used to train public models and then being leaked through clever prompt engineering. Again, lots of learning as organizations strive to determine what are real concerns and what are imagined.
Risks and powerful results on the GenAI path
Paul Venditti, Advisory Industry Consultant, IoT, SAS
When considering our customers’ top requirements for meeting their own GenAI [GAI] objectives, one fundamental principle is “Do No Harm.” While the potential for disruption and value generation through GAI is undeniably exciting, it also brings forth a host of serious risks. Among these risks are:
- Legal / IP Risks: GAI technologies do not infringe upon existing legal frameworks or intellectual property rights. Mitigating legal risks is paramount to safeguarding their operations and reputation.
- Brand Reputation: GAI solutions that enhance a brand image rather than risking negative publicity or backlash due to unethical or harmful applications of the technology.
- Mitigating Harm from LLM Hallucinations: Particularly concerning are the potential consequences stemming from LLM hallucinations, which could lead to actions harmful to employees or customers. This is especially true in industrial applications.
Many efforts we encounter are focused on “experimenting” with public LLMs in tasks that carry less risk, such as summarization of private, trusted data. Training publicly available LLMs on their own data…[and] tailoring LLMs to specific use cases, the expenses involved in fine-tuning can be prohibitive.
An approach for protecting data when using public LLMs
Shinji Kim, Founder and CEO, Select Star
[GenAI] is an opportunity and/or a threat for many of our customers. It is becoming one of the most sought-out table stake features in the market. This is fueling our product development direction to be faster and our internal operation to be more efficient. On the customer side, we’re seeing an increased interest in training their employees to be more data-driven.
As far as we’ve seen, many customers have been OK with us using GPT or public LLMs with their data. This is primarily because we use only metadata and metadata analysis rather than the data values itself. We’d face more challenges if we were to try using their data as-is.
RAG for real-time data feeds into GenAI
Madhukar Kumar, CMO, SingleStore
A large language model doesn’t know about your distributed corporate data. If you ask it, “What is Europa?” it will tell you it’s Jupiter’s moon. Whereas in your data set, it could be a project or an important product feature. To include your corporate data, you would use retrieval-augmented generation (RAG). To answer the question using RAG, you’re looking at your exact corpus of structured and unstructured data. You search. And then you come up with context. You tell the LLM the user is asking for Europa. By the way, here’s the context. Now, go answer the question. So that’s where it all comes together.
So, let’s take the customer support use case where a bot is responding to a user directly about an issue that they’re facing. Now, in your production system, there is an ongoing incident with outages, but there’s a bug fix. If your bot doesn’t know about that resolution, it will keep feeding data to the customer based on previous information, which leads to a worse experience versus if, in a few milliseconds, that bot also knew about the existing incident. So now it’s more about the freshness of data– the freshest data happens in real time and in milliseconds versus batch jobs, which can take three to 12 hours or even days.
GenAI makes great AI applications even greater
Yali Sassoon, Co-founder and Chief Product Officer, Snowplow
We know the impact [of GenAI] is enormous, but we’re only just starting to tease out the possibilities.
[AI enables] behavioral data applications that help drive growth for [our customers’ business – things like personalization engines, fraud detection systems, next best action, dynamic paywalls, and pricing, for example. The scope of the applications that can be built on behavioral data is much greater with generative AI.
To take just one example, companies can look not just at personalizing a customer’s experience by picking the best-suited experience from a list that best matches this customer at this moment in time: they can now create that experience dynamically from scratch…Can we use GenAI to help our customers better understand and activate behavioral data?
Open source LLMs like Llama are particularly popular because they can be run by companies themselves, which is important from a compliance perspective; however, many use hosted, proprietary solutions like OpenAI because of the great user experience, which means it is incredibly fast to prototype with.
Learning how to fine tune LLMs with your own data is really critical for most use cases, and this is where a lot of our client’s (and our own) R&D are focused on at the moment.
A company’s customer behavioral data is often their largest yet most underutilized data asset. Companies can only optimize and “keep up” with the real-time aspects of GenAI if they are defining, tracking, and utilizing their real-time customer behavioral data. Companies need to prioritize getting their hands around this data to make it AI-ready, and that starts with getting it into their data warehouse or lake environments in a high-quality state. Only then can it be utilized for real-time or data-in-motion GenAI use cases downstream. GenAI is showing what data can deliver, but companies need to first understand the gold mine of data (customer behavioral data) they already have access to but are currently not leveraging.
The big challenge: Knowing when the output of GenAI is right
Matthew Fuller, Co-founder and VP, Product, Starburst
[These requirements] are essential for GenAI workloads.
- Scalability: Customers require platforms that can scale efficiently to handle the vast amounts of data processed by GenAI models.
- Data Integration: The ability to integrate and federate data from diverse sources is crucial for training and operationalizing GenAI models.
- Performance: High-performance analytics capabilities are essential for near-real-time GenAI applications.
- Security and Governance: As GenAI involves processing sensitive and critical data, robust security and governance features are paramount.
- Flexibility: Customers seek flexible platforms that support various GenAI frameworks and models, allowing them to choose the best tools for their specific needs.
…We have customers interested in training publicly available LLMs on their proprietary data to tailor models to their specific business contexts and requirements. The biggest challenge they face is ensuring data quality and relevance. Training LLMs require substantial, well-curated datasets, and customers often struggle with preparing their data, ensuring it’s representative, bias-free, and adequately annotated, which is critical for the success of the trained models.
The real-time aspects are crucial for applications like predictive maintenance, fraud detection, and customer service enhancements. Addressing these requires:
- High-performant Data Processing: Implementing efficient data processing pipelines that can handle high-velocity data streams.
- Scalable Infrastructure: Infrastructure that can dynamically scale to accommodate varying data volumes and processing demands.
- Integration with Streaming Platforms: Seamless integration with streaming data platforms to enable continuous data ingestion and processing.
GenAI ushers in a new renaissance
Cindi Howson, Chief Data Strategy Officer, Thoughtspot
We see organizations who like to use the unreliability of GenAI as an excuse for a wait-and-see approach. Because of the fear of hallucinations, some people are saying no, we’ll wait. Quite frankly, we shouldn’t be calling it hallucinations–it’s really just predictive analytics making a mistake, predicting the wrong next word.
I think that’s the wrong approach–the generative AI era is a wave of creative destruction, but also of business opportunity. We’ve lived through the internet, but GenAI will be bigger, faster, and more furious than the internet disruption.
I think we are at a time when some organizations are doing really well by reimagining the way of working with customers, onboarding employees, and asking questions. We need to bring our imagination rather than just sub-optimizing one process. We need to go back and reimagine the engagement model with all our stakeholders, from customers to employees to suppliers. We can view this as a rebirth like the Renaissance was a reawakening or rebirth.
See also: Gartner Keynote: Be AI-ready to Empower Change
Consensus around recommendations
These data leaders agree that GenAI is a massively impactful technology, one that will change every aspect of life as the internet did. The disruption creates opportunities to re-imagine how we define challenges and how we address them. The advantage goes to those who are not limited by the constraints of how we use technology and data to affect how we work, heal, learn, and collaborate today.
The Gartner analysts, too, had a common theme at the 2024 Gartner Data & Analytics Summit: ensure the foundation of your corporate data is in the best possible condition, for GenAI, indeed for any large AI project, will stumble on the cracks.
Contributors to this article were united in identifying requirements for GenAI around having good quality data, well-governed data, scalable and high-performant platforms, and well-defined guardrails to ensure GenAI does not expose organizations to legal and business risk–and does no harm.
