To win new business, companies are leveraging digital innovation to try innovative new offerings, new business models, and partnership ecosystems.
We are in a battle for the customer… as we always have been. But, the new state of the art “weaponry” to win this battle is “Digital Transformation.” What is
It involves a change in leadership, different thinking, the encouragement of innovation and new business models, incorporating digitization of assets and an increased use of technology to improve the experience of your organization’s employees, customers, suppliers, partners and stakeholders.”
This definition hits on many important points most around how a business needs to change, but it closes with the critical aspect that differentiates digital transformation – an
See also: Tata’s Hariharan,
For example, airlines advertise their flights and try to provide the best air travel option for a traveler. Is this all the client was looking for? Maybe. But, they might also be looking for transportation to/from the airport, a place to stay, a car rental, and depending on the type of travel – vacation activities, restaurants, etc. Delivering on the client perspective and not just the company offering is the digital transformation differentiator.
To win the client’s business, companies are looking for rapid time to market to try innovative new offerings, new business models, and creating partnership ecosystems. Traditional business planning and long-term delivery cycles are not going to cut it if the business is to be successful. Partner on-boarding cannot take months for each individual partner to develop a working customer-centric ecosystem.
And, trying a new idea cannot require significant time and expense just to see if the offering is desirable. A far more agile method to enable innovation is required. Yet, we cannot expose our core business assets without proper security and controls to ensure reliability and availability. We must support rapid innovation while maintaining service levels for our core business functions.
Business APIs and Microservices
For the last several years companies have been using Business APIs and Microservices to speed time to market and expand market reach.
Let’s start with defining these terms:
- Business API – is a simple to understand programmable interface focused on business recognizable assets – a product, an order, a customer, etc. Business APIs are targeted toward the consumer of the API, not the provider application(s). APIs do not contain business logic, they are an interface. APIs are focused on self-service consumption of the asset, simplicity, security, analytics, and speed to deliver.
- Microservices – a microservice contains business logic (i.e. code). Using microservices implies a micro component architectural style for creating applications. Instead of creating one large monolithic application that contains all the application parts in a single “package,” the purpose of microservices is to break the application into component parts, each providing a specific capability that together
implement the functionality desired in the application. This approach increases agility, scalability, and improves DevOps.
APIs and Microservices are frequently used together which has caused some confusion. Many have come to believe they are one and the same, but they are not. Please see “Clearing up misconceptions about APIs and Microservices” for more information on the positioning.
APIs provide a level of abstraction from the implementation. Using an API allows IT to change the back-end system implementations without affecting the consumer (assuming the API interface remains compatible). A common scenario is to rearchitect monolithic applications using microservices and have the APIs which previously accessed the monolithic application or new APIs now access the microservice application.
The new microservice application provides greater agility on the back-end and supports the ability to move appropriate components to the cloud. Enhancing agility and transition to cloud have been the two primary drivers for microservice initiatives.
The modern view of Enterprise Systems of Record (SoR) is that they are composed of some microservice applications, some monolithic applications, some are on the cloud, some are on-premise, and some are provided by others (i.e. SaaS). The combination of APIs used by your systems of engagement (SoE) and partners accessing your modern SoR is the target architectural model for most enterprises today.
With APIs and Microservices we provide improved agility. But what about innovation?
Innovation Tier
Most businesses are hesitant to add business logic to their critical core systems without significant effort to ensure they will not adversely affect the existing functionality, qualities of service, and security. Rightly so! Yet, part of the definition of digital transformation deals with encouraging innovation. We need a mechanism to support innovation without the time, cost, and risk associated with modifying core systems before proof that the innovation is a good idea.
I propose a third business driver for microservice: innovation.
As shown in the figure above, we have our existing SoR accessed via APIs from the SoE. Inserting the innovation tier allows a place to add new business logic without changing the back-end SoR systems. New microservices are developed for the innovation tier which access the existing SoR to obtain information and then provide the new business logic for the innovative offering we are testing. APIs are provided to consumers to access the new business logic to test our offering.
If our new offering is not successful, we simply throw away the new API and Microservice. Our investment was minimal, and we learned that the offering is not worth further investment. This is called “failing fast” – which is a good thing.
If, however, we find the offering is successful we can now take the appropriate care to implement the innovation in our core systems while ensuring the existing qualities of services are not affected. By investing small efforts rapidly in the innovation tier we can test the market without the major time and expense that would previously have been required.
So now we have a way to implement innovation rapidly, however accessing the new innovative offering is still a bit cumbersome. We need to create or change an API to invoke the new business logic, deploy the API, have someone consume it and deploy their update before we can test the new idea. Perhaps there is a better way?
Introducing a Service Mesh
The key benefit of breaking a monolithic application into microservice components is that each microservice is fully decoupled from the others. It can be maintained, or even replaced independently from the rest of the application. This is what enables the greater agility, scalability, and resilience benefits that microservices architectures promise. However, few microservice components are completely isolated. Most have a role that involves communication with other microservices in the application. This is where fragility can easily creep back into the application, reducing its agility and resilience.
The industry has begun to standardize on container orchestration platforms such as Kubernetes to administer microservice components. Kubernetes provides basic functionality, for enabling discovery of other microservices at runtime and dynamic load balancing across replicated containers, but that is pretty much as far as it goes.
It still is necessary to provide some of these inter-microservice communication capabilities in a way that is independent of the individual microservice components. This is where the service mesh (e.g. Istio) comes in. The mesh intercepts all intercommunication between microservices in a given scope (or application boundary). It can then introduce things such as routing, resilience and security patterns completely independently of the microservices themselves. Effectively, it extends the number of inter-connectivity patterns available to Kubernetes.
Now let’s apply this to our prior scenario with the innovation tier. Instead of creating a new API with all the on-boarding and time required to deploy a new application, we can use an existing API (if there is one) that is related to the area of innovation. The API can pass the request to Istio for routing rather than directly calling the new microservice or old SoR function. Istio can choose where to route the request.
This allows for more dynamic routing which can provide additional real time data on how our innovation project is working. For example, if we want to try multiple different innovation options, Istio can route different percentages of the traffic to multiple different implementations (A/B testing). Or, we can implement the innovation for a small percentage of the traffic (canary testing) and then increase the usage as we see positive results or turn it off in the case of negative reaction. Each of these scenarios is implemented without changing the API or the business application microservices.
In addition, Istio can also provide customer differentiation in testing new business models. API Management solutions use a “plan” to offer different amounts of API calls to consumers. A gold consumer may use the API more invocations per time unit than a silver or bronze consumer. A consumer signs up for a plan and the API gateway enforces the metric. This is also frequently used in direct monetization models to charge for higher usage levels. All of this is good functionality, but the concept of “plan” is only known to the API Management solution. Once the API invokes the microservices or other back-end systems these applications do not know anything about the plan of the consumer. What if we could pass the “plan” information onward – beyond the API Gateway? Now a gold customer might not only get more API calls, but also have priority on throughput, performance, or additional functionality provided by the called application. For a digital business we need to treat our best customers even better! Istio provides this capability for microservice clusters. In conjunction with API Management as an ingress, Istio can provide the prioritization outside the business application without changing the application itself and without over-deployment of resources. Higher priority customers can have their requests prioritized and/or routed to enhanced business functions.
Earlier we also spoke about Digital transformation providing a customer-centric view. As a customer I find it frustrating when I must manually bridge internal company boundaries. A prime example is my insurance company where I have home, auto, and an umbrella insurance policy with the same company yet need to deal with the company as three separate entities, accessing and dealing with each policy separately.
Digital businesses do not have these boundaries, rather they use the data from their other LoBs and outside entities to make for a better experience for their customers, not letting the internal boundaries show through. Istio can assist in this scenario by routing requests to various microservices that supply different functionality for different LoBs.
However, in this scenario APIs need to help Istio pull all this information together. While Istio does wonderful things for microservices within its scope of control, it does not handle anything outside that scope. Some of these resources may be within an Istio service mesh, but many may be in either other Istio clusters or in other systems such as other clouds, z/OS, SaaS applications, etc.
To pull together information from these sources, Istio needs to invoke APIs to bridge between environments to gather the comprehensive customer view. Using integration between API management and Istio to provide appropriate security context to ensure privacy and data security is critical.
Systemizing Innovation
Now for the biggest challenge – people! We need to change behavior. I have seen that when people are trying to accomplish something their first course of action is to use the method that they have successfully used before. The fact that there may be newer better methods is not an immediate consideration. So, if we implement all the IT changes that I have previously described and do not change the human behavior we will not succeed.
Our first step is to make it easy for innovators to take advantage of the new methodology and harder to go back to the prior time-consuming process. After enabling the IT environment for innovation, we need to execute education and communication tasks for our target innovator audiences. Set up lunch and learn sessions and demo how to introduce a new innovative scenario and provide a sandbox where the audience can try it. Provide metrics on the time it takes to use the new approach versus the historical options. And, put further roadblocks in the way of the historical options. Anyone involved in the previous process for approvals, provisioning, etc. for innovative projects should redirect requestors to the new methodology.
Digital transformation success requires innovation. But, it is unlikely that every innovative idea should move forward to become a company offering. We want to encourage more innovation attempts – successful and not successful. The ability to innovate with rapid low-cost low-risk scenarios is a competitive differentiator. Creating a systemized approach to innovation where time and cost is minimal accelerates our digital transformation.
