Customers expect fast apps and personalized experiences. Here’s how a NoSQL database, machine learning, and other tools can help.
Application intelligence is a given in today’s digital world. App consumers have come to expect that Netflix will fill the post-binge void with insightful viewing recommendations, Facebook will never run out of friend suggestions, banks will alert them to unusual credit card activity, and Pandora will always know the perfect song to play next.
But while users may have the luxury of taking application intelligence for granted, developers do not. Finding creative new ways to anticipate user wants and needs is more than a fun diversion; it’s a competitive necessity for retaining customers—and relevance.
Fortunately, the availability of cheap compute capacity, data processing tools, and learning frameworks is making it easier than ever to incorporate intelligence into applications. And savvy developers are increasingly looking to capitalize on the point of decision, when the user is still warm and ready to enhance their experience. For example, a traveler who just booked an airline ticket is much more likely to convert on a car rental discount offered in real-time as compared to one that comes in hours or days later via a cold email, phone call or browser ad.
But there’s a catch. Not only do your app users demand intelligence that delights and inspires in exchange for their loyalties, they also demand high performance. So high, in fact, that even a fraction of a second of additional latency has been shown to negatively impact engagement. This means that analytic operations on live data need to be embedded inline without disrupting responsiveness. That’s no easy feat with user experience expectations hovering at less than 100 milliseconds response time.
With the stakes so high—and no room for error—what can you do to ensure that you’re squeezing every last drop of performance out of your smart application? To begin, you need a database optimized for real-time analytics. Here are four tips developers can use to find a database that will get their applications off to a blazing fast start!
1: Application data: Go beyond relational databases to NoSQL
If your app deals with unstructured data such as texts, JSON, events, actions, posts, video, email, etc., there’s no compelling reason to endure the high latencies that typically accompany the cross-table joins and heavy queries of relational databases. NoSQL databases, due to their lightweight data models and ability to scale across multiple servers, can achieve much higher performance than traditional relational databases in the fast-moving landscape of real-time analytics. A step further than NoSQL is in-memory NoSQL, where the blazing fast speeds of memory make real-time truly real.
2. The more data structures, the better
NoSQL offers more flexibility than relational databases, but not all NoSQL is created equal. When it comes to analytics, true performance boosts occur when your app can take advantage of data structures that have been purpose-built to support the objective at hand, whether it’s time-series analysis, spatial analysis or machine learning (to name a few).
Sets, sorted sets, hashes, lists, strings, bitmap, and hyperloglog are all examples of data structures designed to more elegantly store variably structured data, as well as perform complex analytics on the data via built-in operations. The more data structures and built-in operations you have at your disposal for current and future intelligence needs, the more network and computing overhead you can eliminate, while also radically simplifying application development.
A step further than data structures is the capability to incorporate any custom functionality into your database – so that it can adapt to your data and performance needs rather than the other way around.
3. Use built-in in-database analytics to the largest extent possible
Leading media and entertainment companies already do this — instead of waiting for post-processing of data to inform intelligent behavior, they process and analyze data in real-time, as the user is interacting with their application.
Doing this inline is only possible with real-time built-in analytics, such as sorted sets for keeping track of ranks or bids, using statistical estimators for streaming data, or combining such native analytics to build recommendation engines that utilize the users’ actions and similar users’ behavior to extrapolate what offers to make or next steps to present. Techniques like these ensure that apps are simultaneously intelligent and high performance.
4. Embrace techniques like real-time collaborative filtering or machine learning
Intelligent apps are often intelligent because they employ sophisticated algorithms or automated learning techniques to glean a user’s intent and anticipate what would delight them. Sophisticated algorithms might include learning techniques that classify the user based on his or her choices and then determine the right action on the fly. In these cases, it’s important to use a database that natively supports machine learning, so the most up to date models can be used in real-time.
Why is real-time better than batch? Real-time allows for more flexibility than a batch model that responds to new trends every two months — it gives your application the ability to stay in lock-step with the latest trends, seasonal changes and mindset shifts. To go back to the travel booking example, if the customer has already declined car rental and hotel booking package options, maybe he or she needs to know about top attractions around the place they are visiting. Or, if you can glean that they’re a business traveler based on their itinerary, maybe discounts like an airport spa or lounge visit might be a more relevant offer.
For less-sophisticated uses, you could use open source modules to both train and execute simple feed-forward neural networks. For larger, more sophisticated models, machine learning frameworks like Apache Spark are often used to train models. However, the challenge of serving these models in production at extremely low latencies and with the capability to update them in real-time, is best addressed by modules, which can store and update machine learning models in real-time with very few resources.
Intelligent, fast apps
This year, as the race to provide the best user experience intensifies, application developers will need to remain laser focused on intelligence. But improved application intelligence doesn’t have to come at the cost of performance. With the right database running your apps, it’s possible to have the best of both worlds.
