IBM’s Matt Roberts discusses the factors driving the demand for integration, common obstacles to success, and how the right integration technologies complemented with AI can help.
As companies digitally transform and react to the ever-changing business landscape, integration is increasingly important. However, many companies find their efforts hindered due to the technical complexity of integrating data and applications running on disparate platforms. Additionally, there is a skills gap or shortage limiting how fast integration projects can be completed.
RTInsights recently sat down with Matt Roberts, Distinguished Engineer and CTO of IBM Integration, to discuss the factors driving the demand for integration, some of the common obstacles to success, and how the right integration technologies complemented with AI help companies realize their business integration goals. Here is a summary of our conversation.
RTInsights: What business changes have occurred that make integration so important today?
Roberts: There has been a combination of world events and industry trends over the last three years or more that have had a significant effect on the importance of integration to our clients. Things like the global health pandemic and the enforced working-from-home practices have accelerated the digital transformation for many of our customers.
And a key facet of that digital transformation effort is about integrating different systems together. Typically, that allows clients to bring together a responsive and agile solution for their end users.
And at the same time, there is growing complexity in clients’ application estate. So it’s quite common to have up to a thousand different types of applications across many of the large enterprises that we work with. Talking with the US healthcare provider a few weeks back, they were saying that they actually onboard something like 500 new applications to their ecosystem each year as a result of acquisitions and organic business growth.
Each of those things can’t operate in its own kind of silo. It has to be joined to the larger whole, and that’s obviously where the integration capabilities come in. Integration is able to provide that single view and that consistent access to the data across different environments.
RTInsights: What are some of the technical obstacles businesses encounter when trying to address integration issues?
Roberts: The key challenge our clients are looking to address is how to respond with speed and agility to the increasing pace of integration needs driven by their business domains. Typical projects are now generally expected to complete in weeks and months rather than months and years.
The integration teams within any enterprise need to be able to deliver much more quickly than previously to keep up with the pace of demand effectively. And at the same time, there’s a skills challenge being driven by things like the great resignation. That means that many teams are having to deliver on increasingly complex integration scenarios without the depth of skilled experts and SMEs that they could previously depend on.
Alongside that challenge of how to successfully build those new integrations in an agile way is the equally critical conversation about how you operate and manage them on an ongoing basis after they’ve been deployed. So, clients are looking to invest their limited financial and staffing resources in the most efficient methods possible.
That often brings in a conversation about should they be deploying those environments themselves, managing them themselves, or should they be looking to adopt some kind of software-as-a-service style model or a hosted offering, where they can delegate some of that responsibility onto the vendor.
RTInsights: What is a hybrid environment?
Roberts: In the simplest terms, a hybrid environment is a deployment that spans a mixture of two or more kinds of different deployment styles. Typically, that’s something like some footprint running on-premises, coupled with a private cloud or a public cloud environment. Our clients are often coming from a position where lots of their existing deployment is on-premises, and then they are building up new IT estate in public cloud environments.
There are probably three main patterns of hybrid cloud environments, starting with something where you have segregated deployments in each location. So, you might have one application running on-premises, one running in the cloud, and not too much interaction between them.
Then there are some interconnected patterns where the two or more different environments need to exchange data to achieve a particular business scenario.
And then the third main pattern is around trying to provide a single, logical function where part of that footprint is running on-premises, and part of it might be running in a public cloud environment. The use of public cloud, in this case, is typically to provide higher resilience or better scalability or to deal with peaks in demand.
Often, our clients begin to build up that hybrid footprint using some new project as a way to try a new deployment approach. So, perhaps a retailer is looking to build a new mobile application to engage with its customers. They want to take that new application and prove out running it, perhaps in containers in a public cloud platform such as AWS or Azure. Running the front-end application in the cloud is no problem, but maybe the product catalog and the order processing systems all still exist in the on-prem network.
That is one of those key cases where, even though we have a kind of hybrid deployment, the reliable and secure connection between the two environments is really important. They need that reliable and tight integration to take the order and fulfill it.
RTInsights: What types of technology can help build hybrid environments?
Roberts: The key one here is portability. It is a major enabler for delivering hybrid environments. That includes things like being able to use the same tools, the same artifacts, and processes, as well, whether you’re running on-premises or in a public or private cloud. It is dramatically more efficient if you can use the same things rather than having to go and completely re-implement whatever kind of processes you need in place across the different environments. That need for seamless portability exists at multiple levels throughout the technology stack.
So whether it’s something like IBM MQ or DataPower, you can deploy the same application artifacts, whether that integration platform is running as software in a virtual machine and a container environment, in a physical appliance, or even on specialized hardware like a mainframe. In that way, there’s an application-level portability and also a platform level portability that says, “Can we do the deployment of all those bits of software in a consistent way regardless of what the environment is?”
And the best example the industry has today of that kind of platform for software portability are container-based platforms like OpenShift and Kubernetes. They enable you to build your software and applications so you can easily run them on your own laptop, a self-managed cluster in your own data center, or a managed cluster in a public cloud.
From an IBM perspective, the foundation of all of IBM software today is OpenShift, which is IBM’s branded, supported, Kubernetes distribution. That’s our hybrid cloud platform. And having that as our hybrid platform enables us to run in a wide range of environments, anywhere, including on a mainframe and Power hardware, VMware, bare metal servers, and all the way up to virtual machines on public clouds like AWS, Azure, IBM, and Google.
RTInsights: What do hybrid and AI mean for integration?
Roberts: Hybrid has always been a key scenario for our integration offerings. The whole premise of integration is about joining together disparate systems – and to be successful in doing that, you have to be able to run in whatever environment the client has chosen to deploy their business endpoints and their data sources.
We have a long history in our integration portfolio with products like MQ, App Connect, and DataPower, supporting a wide range of different deployment environments. We can support whatever the client’s deployment choices are. The advent of containers and OpenShift extended the ease and flexibility of our hybrid deployment patterns even further.
On the AI side, AI is actually one of the key routes that we’re aiming to help clients address that increasing skills gap. We’re passionate about applying the power of AI inside of our integration portfolio to help automate and optimize the tasks that our clients are carrying out to deliver their integration projects.
We have multiple examples where we’ve embedded AI capabilities inside the offerings to make our clients’ lives easier. We’re not expecting all of our customers to suddenly become data scientists to make use of this AI. It’s almost the complete opposite. We want to go and hide the AI inside the machine and present that advanced capability through simple, intuitive user experiences that makes it easier for the user to get more done more quickly without having to be a deep expert in the area.
To give a few examples, we have AI-driven capabilities that automatically build and run API test suites based purely on the external specification of the API. It understands the structure and knows how to make those sorts of indications.
In another area, in App Connect, we have intelligent field mapping capabilities that recommend the correct mapping from source and target fields. So, when you’re creating an integration, there’s less hard work for the user to do to understand how to convert this kind of source to a target object. And in that case, the solution also learns from your responses so that the next time around, if you’re mapping similar fields, it will give you a more accurate, better set of recommendations based on what you or your colleagues potentially have set up in previous projects.
And then the last example is around data mapping that uses machine learning to create complex mapping expressions to convert from one style of data to another. Rather than requiring users to build those mapping expressions themselves in semi-code language, all it asks is for the user to provide a handful of input fields, such as Matthew Roberts and Ginny Rometty. Users then specify the output they’re expecting, Roberts, M, or Rometty, G, for example. The embedded intelligence in the products does the smart bit in the middle to work out what would have been written as a code transformation between those two things.
RTInsights: Where or how does IBM fit in?
Roberts: IBM’s focus is to be a hybrid cloud and an AI company to help clients solve their business problems. That applies across all the different segments of IBM. It particularly applies in the integration space. And we’re at the epicenter of that effort to make our clients successful because integration is so important to them in their various areas of business.
There are some actions that IBM takes at the cross-company, cross-enterprise level. That includes things like the acquisition of Red Hat, for example, that enabled us to bring in OpenShift as the foundation of our hybrid cloud platform.
We’re also taking leading-edge work, which is done by our IBM Research teams. We’re leveraging their advancements in raw technology and Artificial Intelligence and bringing those developments into the functions and features that we provide as part of the products.
Another kind of cross-IBM perspective is that this isn’t just about the platform, products, and software but also about having skilled consultants and advisors that are able to advise customers on how they apply these things.
So, while our integration portfolio is specifically in the software business, we partner very closely with areas like IBM Consulting and IBM Technology in order to help customers be successful as they deploy these sorts of projects.
RTInsights: Can you give some use case examples of the benefits businesses achieve with IBM integration?
Roberts: One is the Digital Ajman, which is a government agency in one of the United Arab Emirates responsible for digital transformation. They use the IBM Cloud Pak for Integration as the foundation of that digital transformation project. Their goal as an organization is to make it much easier for citizens to interact with the government in a wide range of scenarios and things like digital payments or planning and approvals process for building new homes, for example.
They’ve been focusing on streamlining and simplifying those government processes and going paperless. And through a combination of those various activities with the Cloud Pak for Integration, they’ve actually been able to achieve savings of more than $500,000 US dollars a year, as well as an environmental benefit of saving more than 200 trees worth of paper every year from the reduction in paper.
Another example is a US-based healthcare company called CVS Health. They took a step back and were looking at how they needed to deliver integration within their business and what they needed to do to best support the business goals and agility.
They identified that being successful at hybrid cloud integration is a key enabler for their business success. And having made that observation, they then focused on how they take Cloud Pak Integration as a suite of integration products and use them to help deliver on that goal.
They did a very holistic job. They looked at their patterns, practices, and skills, as well as the raw software. And then, by the end of the journey that they went on, they were actually able to build integrations for around one-third the cost of when they started.
That meant they could do things for a third of the price or in a third of the time, or if you prefer, do three times as many things in the same length of time. That really enabled them to be much more responsive to their business.
A final word
Modern applications and processes require a high level of integration capabilities. The diverse elements that must be brought together in hybrid environments makes the process complex. In many cases, businesses do not have the time, budget, technical expertise, or sheer volume of data specialists to carry out the work.
One key to success is portability, which allows the use of the same tools, the same artifacts, and processes, whether you’re running on-premises or in a public or private cloud. Another critical element is AI-enabled integration. Applying the power of AI inside an integration solution helps automate and optimize common integration tasks.
These are areas where IBM can help. It supports software portability via container-based platforms like Red Hat OpenShift and Kubernetes. And its IBM Cloud Pak for Integration is a complete hybrid integration platform that includes a mix of traditional and modern integration styles and makes extensive use of AI and automation.
