Revolutionise your content delivery with AI-based recommendations

In this post, we discuss the advantages of a recommendation system for video streaming services and explore its positive impact on brand and content discoverability. Additionally, we gain insights into a new AI-powered recommendation engine by Magine Pro which will soon be released to our partners.

OTT streaming platforms offer an extensive range of content to their users. However, people often find it challenging to choose from numerous options, and may become quickly overwhelmed, resulting in poor choices or no decision at all. 

According to consumer research at Netflix [1], a user typically loses interest in selecting content after browsing for just 60 to 90 seconds and reviewing 10 to 20 titles. If the user cannot find anything of interest, there is a substantial risk of them discontinuing the service.

Recommendation systems provide a solution by offering a personalised experience to users that make them feel as if the streaming service was created exclusively for them, which enhances organic engagement.

 

Where we started

During our last Hack Days at Magine Pro, we challenged ourselves to create a recommendation service for the platform. We brought together a cross-functional team of engineers from the Backend, Infrastructure, and Media teams to tackle this problem from different domain perspectives.

The aim was to create an end-to-end real-time recommendation system for a streaming service. Modern recommendation machine learning algorithms use datasets of users, items, and their interactions as inputs. This data has to be streamed, cleaned, transformed, and inserted into the model for the training.

Using modular programming, a technique of breaking down a product into smaller, self-contained units that can be developed and tested independently, we built separate components and went through all stages of the recommendation system creation process.

 

 

As a first proof of concept, we brought together the following services:

  • Mux Analytics is used to examine clickstream data and get insights into video engagement and has an integration with AWS, our main cloud provider
  • AWS Kinesis Data Streams provides the ability to stream large amounts of data in real-time
  • AWS Lambda can be used to perform extract, transform, and load (ETL) jobs on the protobuf-encoded data we get from Mux
  • AWS Personalize is a self-contained AI service that takes input data according to the schema and generates various types of recommendations

 

That worked like a charm! As a next step, our team tried to reduce the costs since using MUX/AWS integration and AWS Personalize was expensive. We also wanted to experiment with different recommendation engines to find the best suitable for our needs.

 

Minimising the costs

The team started to think about how we can make this feature less expensive for Magine Pro and our partners.

Here are the steps we performed to decrease the costs:

  • Replaced the Mux Analytics module with the Apache Kafka, a distributed streaming platform, running in our AWS infrastructure setup
  • Created a separate backend service for the ETL jobs, which listened to the Kafka topics, modified and enriched them with additional information, and provided the resulting data further
  • Integrated Gorse, another recommendation engine, into our system, utilising Redis as our in-memory data store and a fully managed Postgres relational database, both running on AWS

 

 

The modularity of the system gives us flexibility in many ways. For example, we can introduce yet another recommendation engine and compare them to decide which one provides the best results. No need to rebuild the entire setup; only one component has to be modified.

 

Key concepts

Under the hood, Magine Pro uses a recommendation engine that automatically chooses the best machine learning technique for the most accurate results. The model processes real-time clickstream data from the platform, capturing diverse user behaviours. The following recommendation types are supported:

  • Most popular
  • Latest releases
  • Because you watched X
  • More like X
  • Top picks for you

Magine Pro’s recommendation system includes a vast selection of diverse recommendation types which can be customised to align with your business requirements.

 

Recommendation algorithms

There are many algorithms that can be used in recommendation engines, let’s look closer at the most popular ones. Recommendation algorithms are categorised into non-personalised algorithms that provide identical content to all users, and personalised ones that recommend unique content to each user based on their clickstream data.

Latest Items algorithm shows the latest items to users according to timestamps, which allows a new item to be exposed to users in time.

Popular Items algorithm counts positive feedback within the specified window interval and chooses the most liked.

 

 

Item-based and user-based collaborative filtering algorithm discovers the similarities in the user’s past behaviour and makes predictions for the user based on a similar preference with other users. 

It uses a simple yet effective approach – people often get the most helpful recommendation from people with similar tastes. 

Matrix factorization algorithm identifies the relationship between items and users and returns recommended results.

Recommendation system by Magine Pro offers all these options, utilising a hybrid approach that incorporates a diverse range of rating and sorting algorithms.

 

The Cold Start Problem

Cold start problem refers to a situation where recommendation engines cannot provide optimal results due to suboptimal conditions, similar to a car engine having difficulty starting in cold weather. The two categories of cold start are item cold start and user cold start.

 

 

 

Cold start problem: 

What to recommend to a new user U4?

Who to show the new movie I4?

 

 

 

 

Collaborative filtering algorithms can find items to recommend when the number of user actions on an item increases. However, they fall short in case of new users or items since the model cannot predict anything for them.

Our recommendation service provides suggestions of the most popular and latest items to new users to solve the cold start problem.

 

Evaluation

Model evaluation is an essential part of the work on recommendation engines and refers to estimating the performance of a recommender system. 

The system automatically reserves a small part of the feedback to make an evaluation of it. The aim is to maximise the probability that the user likes recommended items so the positive feedback rate can be used as a metric.

In the recommendation engine, we use standard ML evaluation metrics to evaluate ratings and predictions: precision, area under curve (AUC), and recall, where 0 means the worst and 1 means we have the best-performing ML model.

 


As you can see on a screenshot from the recommendation engine dashboard, our model demonstrates impressive performance.

 

What’s next?

Currently, our team is going through the final steps of feature development, evaluating results to ensure that the system we have built is resilient, highly available, and provides the best possible results.

We are also looking into the potential of integrating a content promotion feature for our partners.

Furthermore, we have long-term plans, such as integrating another recommendation engine into our modular system and creating a scalable testing solution to select the most appropriate model that meets our partners’ needs.

 

Conclusion

This post has highlighted the significance of recommendation systems in OTT streaming services, explored various algorithms, and delved into the details of recommendation engines. Magine Pro is soon launching a new recommendation service that serves as an example to highlight common issues encountered in creating such services and to introduce frequently used performance metrics.


Don’t forget to subscribe to the Magine Pro e-newsletter to stay up-to-date with our latest news, partners, and products, and to find out which industry events we’ll be attending next.

Let’s introduce more girls to engineering

On March 29, we welcomed 13 girls aged between 13-19 years old, to our office in Stockholm, as we took part in “Introduce A Girl to Engineering Day” (IGEday) once more. IGEday is a yearly event in Sweden where tech companies and organizations, big and small, open up their offices for girls and non-binary to get a hands-on introduction to the life of an engineer.

IGEday is an initiative by the organization Womengineer, whose goal is to inspire future female engineers with practical learning experiences. Sweden considers itself to be one of the most equal countries in the world, but out of the 120,000 members of “The Swedish Association of Graduate Engineers,” only 28% of the members were women in 2018 (from the report “Teknisk Obalans”). This is why Womengineer believe the annual event “Introduce a Girl to Engineering Day” is a step in the right direction for Sweden’s future.

This year almost 2,000 girls participated in IGEday and more than 120 companies took part, Magine being one of them.  

Fabian Miiro, an engineer at Magine, who supported the girls during the day shares his thoughts on the experience: “My favorite moment is definitely seeing the transformation from the ones that start off being… skeptic. They mention they surely don’t know anything about programming. And then, just 15 minutes later they are the ones that say “What if…” and “Oh, let’s try…”. To see that transformation, when someone realizes that they can to do what an engineer does, that is what this is all about to me. Because everyone can be an engineer!”

During IGEday we ventured into the programming landscape and introduced the girls to design, hardware and software, programming Arduinos using a visual programming language based on scratch.

 

“It was fun to watch the girls challenge claims. For example, they were asked to choose any number, so they chose a number, then they were asked to choose another number, which they did.  Then they were asked to choose a new number again; the reason was that the first two numbers they chose were special, so they found out that “any number” in this case meant “any number but those two numbers” – Kadri-Annagret Petersen, Tester at Magine.

Magine is very proud and happy to have been part of IGEday and we strongly believe in its purpose, and as one of our Project Coordinators at Magine, Esther Pino, said: “Anyone can be a programmer or designer, the girls proved it that day!”

Get in touch to find out more about events and career opportunities at Magine.

Introduce a Girl to Engineering Day at Magine

Last month we had the privilege of taking part in Introduce A Girl to Engineering Day. A yearly event in Sweden where tech companies open their offices for girls aged between 13-19 years to get a hands-on introduction to engineering.

The event is organized by Womengineer, whose goal it is to have equal numbers of men and women graduate as engineers by 2030. IGEDay is one step towards accomplishing it as the day aims to inspire future female engineers with practical learning experiences. Magine is extremely proud to be part of the journey, and this year we welcomed 20 girls who selected us to be their host company.

Introduce a Girl to Engineering Day at Magine

“The IGEDay has been the most unexpected awesome day that I’ve had. The nerves I started the day with turned pretty fast into excitement and it was amazing to walk the girls through solving the challenges.” – Camelia Smeria, engineer at Magine

During IGEDay we ventured into the programming landscape and introduced the girls to hardware and software, programming arduinos using a visual programming language based on scratch. We also explored the importance of project management and put it into practice by designing and creating demo OTT services.

Ivett Backlund, an engineer at Magine who supported the girls during the day shares her thoughts on the experience, “It was exciting to see how fast our visitors got the technical concepts like agile, user-centric design and product ownership and it was fascinating seeing them create solutions to the services they use every day.”

Get in touch to find out more about events and career opportunities at Magine.

Introduce a Girl to Engineering: Sign Up Now!

This year Magine Pro is taking part in Introduce a Girl to Engineering Day on March 23rd 2018. It’s an important event in our calendar and gives young girls the opportunity to get hands-on experience in the workplace and learn more about this fascinating career option. We’re inviting girls between the ages of 12-19 to sign up now on the IGE website, please don’t miss this fantastic opportunity!

At Magine, we are proud to employ a diverse mix of highly-skilled engineers and feel it’s important to help to encourage younger generations, and in particular girls, into the field. We will be partnering with Womengineer to host Introduce a Girl to Engineering in our head office in Stockholm.

If you’re a girl between 12-19 and interested in signing up, or know someone that fits the criteria, spread the word! Sign up now on the IGE website by March 2nd 2018.

 

This website uses cookies

Cookies consist of small text files. They contain data that is stored on your device. To enable us to place certain types of cookies we need to obtain your consent. At Magine Pro AB, corp. ID no. 559301-7287, we use the following kinds of cookies. To read more about which cookies we use and storage times, click here to access our cookies policy.

Manage your cookie-settings

Necessary cookies

Necessary cookies are cookies that must be placed for basic functions to work on the website. Basic functions are, for example, cookies which are needed so that you can use menus on the website and navigate on the site.

Cookies for statistics

For us to measure your interactions with the website, we place cookies in order to keep statistics. These cookies anonymize personal data.

Cookies for ad-tracking

To enable us to offer better service and experience, we place cookies so that we can provide relevant advertising. Another aim of this processing is to enable us to promote products or services, provide customized offers or provide recommendations based on what you have purchased in the past.

Ad measurement user cookies

In order to show relevant ads we place cookies to tailor ads for you

Personalized ads cookies

To show relevant and personal ads we place cookies to provide unique offers that are tailored to your user data