The future of recommender systems is real-time machine learning. Here’s everything you need to know about what that means for enterprise applications.
The future of machine learning is real-time and, when it comes to recommender systems, companies took several steps to help lay the foundation for that this year.
Over the summer, ByteDance, the company behind Tik Tok, released a paper explaining its Monolith recommendation system which is considered a breakthrough in real-time online training, and made it available for businesses to access and license.
Around the same time Slack, the collaboration tool, released its Recommend API, a unified end-to-end machine learning infrastructure to generate recommendations. Along with the API, it provided a detailed overview of how Slack uses it in-house.
Ahead of Tecton’s recent apply(recsys) event last month, RTInsights discussed with Gaetan Castelein (GC), Tecton’s VP of Marketing, some of the latest trends, apply(recsys), as well as what Tecton has planned for 2023.
Here are some highlights from our conversation:
How companies such as Google and Amazon use machine learning today
GC: Machine learning has been around for a while. And the cloud native companies, companies like Google, Amazon, Facebook, use machine learning all the time. And they use it to power the application.
These are systems which are running in production, they make very high-scale recommendation volumes, they do them in real time. And they can use those recommendations to build new customer-facing applications and things that our business processes.
For example, Amazon uses real time machine learning for things like fraud detection, and they use it to make product recommendations. If you shop on Amazon, it would say, “Hey, we think you would like this other thing,” and that’s delivered by machine learning.
If you look at Google, the ad recommendation engine is delivered by machine learning.
Why the Future of Machine Learning is Real-Time for Most Companies
GC: We believe that similarly, the majority of enterprises are eventually going to deploy machine learning in real time. Why? Because unless you do that, you really will be missing out on most of the value of machine learning.
If machine learning is running in batch mode, and just being used to power dashboards for human decision making, you’re still down to the speed and the scale of a human. And that’s really missing most of the point. The point is we should be able to automate simple decisions and do it in real time and at a scale which humans couldn’t match.
And once you do that, once you run machine learning in production in real time, you open up all sorts of exciting new use cases.
Related: To Batch or To Stream–That Is the Question of the Day
Examples of Real-Time Machine Learning
GC: To give you a few examples, if you are Uber and you want to do surge pricing, well, you couldn’t do surge pricing if you had humans doing it, because you couldn’t have a human in every single geo making pricing decisions. That wouldn’t be practical. But a machine can do it.
Or if you do Uber Eats and you order a meal, getting your ETA is delivered by machine learning. You couldn’t have a human estimating delivery times for every single meal. That will be too expensive, but the machine can do it.
Related: Uber Deploys Exactly-Once Processing System for Ads
So these are just a few examples of how real-time machine learning can power new services.
How Machine Learning is Used in Recommender Systems
GC: When it comes to recommendation systems, recommendations are a fairly common usage of machine learning.
A few examples of where they could be used:
Imagine you’re watching Netflix and Netflix recommends other shows that you may like, or if you are listening to Spotify, Spotify may recommend some songs that you may like- Discover Weekly may create a playlist just for you based on your listening preferences.
These are two good examples of recommendation systems.
Other common ones, like [what] I mentioned, Amazon shopping, but really any online retailer benefits from recommending products that you may like, so it’s a very common thing. But they’re very complicated systems.
Why Recommenders are Complicated
GC: Most recommendations are actually based on one entity. And one decision point, such as, “Is this transaction fraud or not? Should I increase the price for surge pricing or not?”
In the case of recommendation systems, you have a matrix of users and products, and you need to match those two. And that multiplies the complexity by the order of magnitude of both of those entities.
So if you have 1000 users and 1000 products out, you need to make a million recommendations on which product to match to a user. That gets a lot more expensive than to make a decision on one or two entities.
Challenges with Architecting Recommendation Systems
GC: So recommender systems are complicated, they’re resource intensive and people really struggle with it. And we see this all the time with our customers. They are asking us “How do I architect this? What do I do?”
And there’s a layered approach to how you adopt recommendation systems. You can start off by just doing batch recommendations. This could be, I’m going to predict which song you are going to like, based on your historical listening preferences.
It could be like, we will look at a song that you listened to over the past two weeks, and based on that, we will predict which songs you will like. But when you are making these recommendations, they become a lot better if you can make them using real time data.
For example, if you can notice, “Oh, well, the listener seems to be in the mood for a happy song. Let’s suggest more happy songs?” Or, “The listener is in the mood for a holiday song, let’s suggest more of those songs.”
That is when things get much more complicated- when we move from batch to real-time data to make these recommendations.
Those are the use cases and the context for this event. It’s really helping people get started with recommender systems and batch, but then also, develop intelligence and move them to real time.
Overview of apply(recsys)
Keynotes
GC: We have two amazing [keynote] speakers – Katrina Ni from Slack and Youlong Cheng from Bytedance, which is the company behind Tik Tok.
Both of these speakers published recently. [In] the case of Katrina, it was a blog on Slack’s Recommend API. This is an API behind which Slack’s recommender system sits and makes these recommendations broadly available to all of the engineers within an organization.
So for engineers that believe that they can make their application better by using recommendations, they can tap into this API. [Watch her keynote replay here]
And then we have Youlong and he’s going to talk about Monolith, the recommender system that’s developed by Bytedance. It’s used on Tik Tok and we all know how good Tik Tok’s recommendation engine is, recommending the next videos you may want to see. And so he’s going to talk about what they do at Tik Tok. [Watch his keynote replay here]
Panel discussion: Lessons Learned – The Journey to Operationalizing Recommender Systems
GC: The panel is intended to bridge the gap from the newbies to the experts. We’ll have these expert level talks and most of the attendees are not at that level so the panel is going to be more people who are just now getting into recommender systems or have done a recommender system over the past couple of years, but are not yet at that level of very advanced use cases, like Tik Tok and Slack.
And they’ll talk about their experience and lessons learned and recommendations for the audience. [Watch the replay here]
Mike Del Balso: Crawl, walk, run – a practical introduction to applied recommender systems
GC: And then we have a session by Mike, Tecton’s founder and CEO. He’s actually kicking it off which we do every time. And he is also going to talk about this journey called “Crawl, walk, run,” the journey to get into recommendation systems.” [Watch the replay here]
Workshop: Choosing Feast or Tecton for Your RecSys Architecture
GC: We also have a workshop that is given by Danny Chiao, a Feast expert. Feast is a leading open source feature store, which is heavily backed by Tecton.
And with by Jake Noble, who is Tecton software engineer, He has a lot of experience with recommender systems, including working on YouTube’s recommendation engine. He is now involved with a lot of our recommender system customers.
So they will talk about using Tecton and Feast to implement that data architecture, the data infrastructure for recommender systems, and they’re going to do it using hands-on examples. It’s really a workshop where people will get to code and follow along and so on.
What to Expect from Tecton in 2023
GC: From a product standpoint, there are three key things that we are working on. One is expanding the availability of Tecton.
Today, Tecton [which is a SaaS product] is on AWS. We are going to make it available on Google Cloud and Microsoft Azure, because we get a lot of customer demands to be on those clouds so that’s a priority for us.
A second priority is to make Tecton more easily accessible. And the motivation there is this move to real-time machine learning. It’s still in its early stages. A lot of people are playing around with it or are starting to do it.
They are still finding their way around what needs to happen to do real-time machine learning. A lot of them end up going to Feast because Feast is open source and easy to download and access. For Tecton, you do still have to speak with us and it is more involved to get access to the product and onboard people.
And then the third big one is making the Tecton engine better, such as being able to support more ideas and use cases, [and] supporting more demanding customers into the right scale and latency.
Those are the big three things from a product standpoint. [Also,] expanding the company by hiring more people in the field to support our customers, expanding the engineering team.
