Analytics queries that used to take 12 hours can now be done within seconds using an in-memory database.
Name of Organization: MyThings
Industry: Online advertising
Location: London, England, UK
Business Opportunity or Challenge Encountered:
Websites get a lot of traffic, but only four percent of visits result in actual consumer purchases. MyThings, an ad-tech company, seeks to help clients increase this number by employing real-time data to deliver personalized advertising. The company calls its approach “dynamic personalized retargeting,” which is essentially data-driven advertising that personalizes content in banners in real-time.
MyThings runs ad retargeting campaigns on desktop, mobile and Facebook, personalizing more than five billion impressions a month for the largest e-commerce brands in 30 markets, including Adidas, Walmart, ToysRUs, Very.co.uk, Littlewoods, Zalando, Orange, Best Buy, and Microsoft. The company accomplishes this through predictive artificial intelligence algorithms, capable of analyzing hundreds of parameters from six different onsite and offsite data sources to calculate an up-to-the-second user value. The company also seeks to provide its advertising clients enhanced visibility into audience viewing patterns.
MyThings is integrated with 16 ad exchanges – including The Facebook Exchange and leading mobile exchanges – in addition to having direct media partnerships with hundreds of top publishers and premium media networks.
With the rapid growth in clients and ad-traffic volume, the company needed to beef up its back-end infrastructure to support the real-time delivery of targeted advertising for its client ecommerce sites. Reducing latency in data transmission was also a key goal.
How This Business Opportunity or Challenge Was Met:
To achieve more robust real-time ad content delivery and tracking within its back-end infrastructure, myThings implemented in-memory database technology from Exasol. Within in-memory tech, data and code are managed and processed within the random access memory of computers. Conversely, when data is stored on disks, there is greater latency as it is located and brought over to RAM as it was called by applications. This disk-to-RAM transfer process is accomplished in milliseconds, of course, but when multiplied by the hundreds of gigabytes it begins to add up to a great deal of latency.
This data latency was slowing down the company’s services, according to Ofer Ohana, business intelligence team leader at myThings, told Exasol, “We had tremendous data growth that pushed our previous solution to the limit. Due to the size and complexity of our analysis, the solution would run for 12 hours or more and sometimes never finish.”
The Exasol in-memory database system employs a shared-nothing architecture to ensure greater resilience, and runs on commodity x86 servers, supporting thousands of concurrent users. The solution includes an intelligent cost-based query optimizer, which is based on self-learning algorithms and manages what data is kept in-memory. This automates workload management at the system level.
MyThings’ system inventory grew rapidly, from four servers with a total of 288 GB of main memory, to a cluster of six Dell servers with a total of 4.5 TB of memory. As a result, the company can run complex analysis workloads on more than 20 TB of raw data. The company is also investing in the ability to process 100 TB of raw data while still performing fine granular analysis.
Measurable/Quantifiable and “Soft” Benefits From This Initiative:
The time it takes for myThings to build statistical models offline to support ad retargeting has now been reduced to a few seconds. In addition, ad-hoc queries on billions of rows of data now take milliseconds, Ohana told RTInsights.
With a faster back-end analytics and more responsive infrastructure, myThings identifies relevant data more rapidly and thus improves its competitiveness and the performance delivered to its ecommerce site clients.
In conjunction with complex data science methodologies, the clients can analyze and determine audiences’ interests and affinities – increasing the quality and relevance of advertising generated in real-time. Furthermore, myThings can now significantly reduce query times for each consumer, with basket analysis, performance monitoring and reporting now available in real-time.
The in-memory database implementation has also lowered the amount of hardware and systems myThings needed to devote to its services, as well as take advantage of higher thresholds in terms of numbers of users. There is also greater scalability to support larger workloads.
(Sources: Exasol, myThings)
Related:
E–book: How to move to a fast data architecture
How in-memory data grid turbocharge analytics
What’s behind the move to in-memory tech
