API platforms powered by in-memory computing are the fastest and most effective way to drive business performance and deliver highly-engaging customer experiences.
As companies attempt to leverage their enterprise-wide data to power real-time business processes that enhance customer experiences and drive higher revenues and improve business insight, they often encounter two major challenges. First, the relevant data is spread across multiple, siloed datastores, so they need an architecture that allows them to aggregate and process data from multiple, disparate data sources in real time. Second, they must be able to easily scale out the solution while maintaining real-time performance as they continue to add data sources and roll out new use cases for the aggregated data. Increasingly, API platforms play a key role in driving such digital transformation.
A key obstacle companies often face in achieving these goals is the inherent limitations of the existing APIs used to access the data sources. These APIs may have limited functionality and the API calls may be expensive, resulting in high costs, a limited ability to access the data in real-time, and difficulty scaling the solution.
See also: The Architectural Imperative for AI-Powered eCommerce
However, modern API platforms (or “smart operational datastores” or “digital integration hubs”) powered by in-memory computing can enable businesses to aggregate data from multiple cloud-based and on-premises systems and query the aggregated data in real-time. By doing so, the API platform can:
- Decouple frontend and backend systems, enabling much easier changes to the applications and data sources
- Minimize the limitations of siloed data, allowing businesses to analyze information across the entire business in real-time
- Enable previously impossible real-time business processes, such as creating real-time 360-degree customer views
- Power previously unavailable application functionality and increase operational flexibility while lowering costs
- Reduce API calls to operational and analytical systems and SaaS applications, potentially reducing costs and complexity
API platforms and their in-memory computing-powered architecture
API platforms are built on a high-performance data cache that can aggregate a subset of data from multiple cloud-based source systems, including data lakes, SaaS applications, operational databases, and streaming data sources, as well as multiple on-premises analytical and operational datastores. The aggregated information in the high-performance data cache can be accessed in real-time by numerous business applications using one of a variety of common native or custom-developed APIs, independent of any API call or functionality limitations of the source data systems. The consuming applications can range from customer-facing websites to back-office systems to applications that power omnichannel customer experiences. A change data capture layer synchronizes data between the API platform cache and the source data systems, so whenever relevant source data changes in any of the data siloes, the data cache will be updated such that current data is always available in the cache.
The most popular and cost-effective solution for building the high-performance data access cache of the API platform is an in-memory computing platform that includes an in-memory data grid (IMDG). The IMDG is deployed on a cluster of commodity servers that pools the available CPUs and RAM. The IMDG automatically distributes data and compute across the cluster, which can be deployed on-premises, in a public or private cloud, or in a hybrid environment.
With the required source data aggregated in the IMDG, the in-memory computing platform processes data queries using massively parallel processing (MPP) across the distributed, in-memory cluster. The combination of caching the data in memory and using MPP improves performance by up to 1,000x compared to solutions built on disk-based data stores, enabling real-time performance.
The IMDG should support the most popular APIs, including key-value, SQL, JAVA, C++, .NET, JDBC/ODBC, REST, PHP, MapReduce, Scala, Groovy, and Node.js. These APIs may enable a business to access and query the data in ways that were not available using the APIs of the source data systems, providing new functionality. The IMDG should also support the creation of custom APIs.
The IMDG also solves the challenge of scaling the size of the data cache while maintaining real-time performance. The IMDG makes it easy to increase the compute power and RAM of the in-memory computing cluster by adding new nodes. The platform automatically detects any additional nodes and redistributes data to ensure optimal use of the cluster CPU and RAM. This enables a business to add new data sources and roll out new applications that rely heavily on API calls without impacting the performance of the source data systems, the other applications that are consuming the data in the cache, or the IMDG.
With the relevant data cached in the API platform and compute distributed across the in-memory data grid, a wide range of business systems that drive real-time business processes – consumer-facing applications such as customer portals and customer support, back-office systems such as CRM and data analytics, and mobile and IoT applications – can access the data. These systems can use this data to create real-time, 360-degree customer views that would be impossible to achieve without the high performance, data aggregation powered by the API platform.
Use cases for API platforms
API platforms are already being used in production environments in banking & financial services, retail and eCommerce, communications, media & entertainment, manufacturing, and healthcare, and more. A couple of examples can easily demonstrate how powerful an API platform can be.
Financial institutions usually offer several services, including core banking, credit cards, car loans, mortgages, and wealth management. Because of the evolution of these services, the data for each is often stored or generated in siloed systems, which today can include on-premises operational datastores, data warehouses, data lakes, SaaS applications, and streaming data sources. Because of this, creating a real-time 360-degree view of a customer across all these services would be extremely difficult and expensive. However, an API platform can span all these source systems, aggregate current and historical information for relevant customers, and create a comprehensive, current view of the data. These views can then be used to present personalized upsell and cross-sell opportunities across the firm’s entire product line via a customer’s preferred touchpoints, such as a mobile app, website, bank teller, or ATM.
eCommerce businesses can take advantage of an API platform to offer far more personalized and relevant online recommendations. For example, a retailer can easily aggregate data from a customer’s account profile, current reward program status, and past purchases and returns, along with the past behavior of similar customers, current product inventories, and pricing and shipping information. This can power highly customized offers for their customers that exactly match their buying patterns.
Multi-channel retailers can use an API platform to improve and tightly integrate the in-store and online customer experience. For example, an application could present in-store employees or online customers with customer account data, existing stock quantities and locations, stock delivery schedules for out-of-stock items, recommendations for alternative products, and more. This can enable a personal and far more profitable response to every customer engagement.
Conclusion
Many businesses now consider data-driven, real-time business processes to be essential for driving business performance and delivering highly engaging customer experiences. An API platform powered by in-memory computing is the fastest and most effective – and may be the only – way to achieve these goals. The good news is that there are now many resources and service providers that can help companies get up to speed on specific strategies and requirements for implementing an API platform that meets the current and future performance and scale requirements for their business.
