As customer expectations around instant gratification grow, fintechs increasingly need to respond in real-time. In this interview, we explore what the challenges are in meeting such market demands, and the role a unified real-time data platform can play in doing so.
Fintech, like many industries these days, operates in real time. Whether for fighting fraud or making optimal trading and investment decisions, companies need the ability to act on events as they are happening. However, the challenge is putting each event into context. That requires breaking down data and system silos that have existed for years.
RTInsights recently sat down with Lalit Ahuja, CTO at GridGain, to talk about the shortcomings of traditional approaches that try to derive actionable insights from disparate systems and the need for a unified real-time data platform to address those issues.
Here is a summary of our conversation.
RTInsights: What is a unified real-time data platform?
Ahuja: Unlike other data platforms, a unified real-time data platform combines data at rest and data in motion with the execution of complex computational workloads on that data, thus enabling holistic and contextual processing of information for timely and accurate outcomes.
For example, you can enrich information coming in a streaming event with the appropriate historical context and then execute some intelligent logic on that enriched data, including a mathematical model or an AI/ML model, to make a decision or do predictive analysis – all in one platform and in real time.
RTInsights: Why is this unified data platform relevant now, more than ever before?
Ahuja: There are really three main reasons for that: the amount of data being generated today, the culmination of technological advances that allow us to access and process more data than ever, and the increasing interdependence of such global data.
First, with the growth in the number of devices and sensors and the explosion in social media, millions of new events are being generated globally every second.
Second, advances in AI, compute hardware capacity, cloud computing, and 5G network bandwidth have also improved our ability to solve more complex problems by processing more data faster.
The third factor is the interdependencies of businesses across geographical boundaries. In the past, data was processed in silos, but this is no longer the case. Businesses today realize they must be able to gain insight across these interdependencies and process information across these silos and geographic boundaries practically immediately. For example, the tsunami off the coast of Japan impacted financial markets, supply chains, and consumer services all over the world. When China’s GDP numbers come out, an investment manager in London, New York, or Singapore may want to immediately rebalance your portfolio. If they spend two extra seconds processing that information, then guess what? You lose out on the opportunity, and somebody else whose investment firm reacted faster wins.
This type of impact due to the speed at which one can process global economic, political, social or environmental events is happening across industries, which is why enterprises want to process information faster and faster and are increasingly interested in unified data platforms.
RTInsights: What kinds of real-world problems does such a platform address?
Ahuja: I gave you the example of GDP and financial markets. Banks are required to maintain a certain minimum balance in terms of liquidity, especially since the 2008 financial market meltdown. This impacts their ability to take risks in terms of hedging and buying on margins. Banks have to calculate these risks continuously because the market is constantly changing. If the New York Stock Exchange is closing, Shanghai, Hong Kong, and Mumbai will be open. Other asset classes – currencies, commodities, etc. add to this complexity. So, the inherent risk in your portfolio is not static; it’s dynamic, and it’s changing by the millisecond. Asset managers and institutions have to be able to continuously roll up the risk at an enterprise level and figure out how much of a margin they have to maintain, what hedging strategies to employ, and what liquidity they have available to invest.
Another example is in the energy sector, where utilities are managing their grids with millions of sensors that are constantly feeding data, which must be correlated with data from other sensors and historical trends to predict whether the grid is in good health or not. If it’s not in good health, you have to react ahead of time to get it back to a healthy state because preemptive resolution is obviously preferred over fixing things after an outage has happened. A unified real-time data platform can enable utilities to better manage their energy grid, reduce costs, minimize unforeseen outages, and maximize revenue.
RTInsights: How are these problems different today compared to similar problems before?
Ahuja: The problems really are not different. What is different is the consumer’s expectation around instant gratification and the ability of enterprises to cater to that real-time demand.
For example, credit card fraud has always occurred, but with the increase in electronic transactions, payments, etc., the opportunity for fraud has also grown exponentially. What has really changed here are our expectations of how the fraud problem gets addressed. Banks are now going beyond saying, “Use any of your preferred electronic payment options, and if there’s fraud, we will pay you back.” Instead, they say, “Use these electronic payment options worry-free because there will be no fraud – we’ll catch it and notify you before it happens.” Consumers, armed with mobile devices and apps for all of their preferred retailers or entertainment venues, expect a certain level of personalization every time they walk into such a location. It’s not that consumers did not like to be treated well before, but now they are saying, “You have so much of my personal information, so use it intelligently.” This expectation has put the burden on vendors and service providers to process the consumer’s location, trends and preferences to practically preempt their thought processes and give them a very personalized and focused interaction.
RTInsights: The capabilities or features you mentioned are available from the big hyperscalers. How do you differentiate yourself?
Ahuja: Hyperscalers have siloed solutions for multiple problems: storing your data in the cloud, processing data in the cloud, processing an event stream, running a mathematical model, or processing events in a transactional system. If you want to solve complex multidimensional problems, you have to glue these things together by yourself. This is no different than acquiring individual point solutions to build these multidimensional data processing capabilities. You still end up with silos. That’s not the unified way!
And as one can imagine, every time you move data from one silo to another, you introduce network latency, data integrity loss, and data security risks. Furthermore, if you have an issue, you have to figure out where the problem lies across the whole data ecosystem.
A single platform gives you a unified solution that minimizes data movement and data latency and allows you to solve that multidimensional problem far more efficiently. And a unified platform that can be deployed in the cloud also gets you the scale you need and 24/7 availability across zones and regions, giving you the best of both worlds.
RTInsights: Has anything changed with the explosive growth in AI and the accessibility of AI to the masses?
Ahuja: Twenty years ago, AI was used for simulation, training, and designing. Today, companies are using it for risk analysis, fraud detection, grid operations, inventory balancing, and more.
But AI is only as good as the data that feeds it. If you have stale data, the results and decisions based on your AI models will be inaccurate and outdated as well. Let me go back to my risk analysis example. If I have the value of an asset from 10 seconds ago, and the real value has already changed five more times, then my calculated perceived risk profile is different from what it actually ought to be. And this means I may end up picking the wrong hedging strategies or go overboard with the liquidity I maintain to balance the perceived risk I carry.
Basically, for AI to be effective, all the relevant information, not just a subset of it, needs to be available in real time. So, the growth in AI has dramatically increased the need for accessing and processing all the latest data across the board in real-time.
RTInsights: Any thoughts on how the data landscape will evolve over the next 12 months or so?
Ahuja: Our customers come to us with all the point solutions they procured over the last three to five years and say, ”These solutions are great for individual siloed problems across our data ecosystem, but I need to make my entire data ecosystem more efficient holistically.” They know they need a unified solution.
As more enterprises make that the cornerstone of their data strategy, enterprises and big cloud providers with data solutions will try to bring efficient and optimized data processing solutions to market, and this may mean that they first consolidate their offerings and integrate them at a native level to make them unified. Another outcome of this general strategy would be that best-of-breed solutions across the individual silos get absorbed into bigger data ecosystems by bigger players, leading to consolidation in the data processing space via technology partnerships or mergers and acquisitions.
