Quality Assurance at Magine Pro – How we ensure the best streaming experience
My name is Ksenia, I’m a Quality Assurance Engineer within one of our backend teams and in this blog post, I will share insights about the Quality Assurance (QA) process here at Magine Pro.
The QA process is a critical component of any successful software project. QA is important in a product like a streaming platform for several reasons, both from the user and business perspectives.
From the user perspective, a high-quality streaming platform can significantly enhance the user experience, which is critical to user satisfaction. With a robust QA process, the platform can ensure streaming quality is consistent and reliable. And users are more likely to return to a platform.
From the business perspective, QA is crucial because a reliable and high-quality platform can attract more users, which can increase the platform’s revenue through subscriptions. Moreover, a platform with a poor QA process can result in negative reviews and feedback from users, which can harm the platform’s reputation and lead to lost revenue.
The QA process is responsible for ensuring the software meets the required quality standards and performs as expected. It involves a series of testing and analysis techniques that are designed to identify and resolve issues or bugs in the software.
We will explore the various stages of the QA process, including planning, design, and execution, and how they contribute to the overall success of a project. So, let’s dive in and explore the world of QA!
Our QA process consists of several steps:
- Requirements Analysis
- Test Planning
- Test Design
- Environment Setup
- Test Execution
- Test Closure
- Requirements Analysis
Our team has a refinement meeting where the product manager introduces a new feature to us and we ask questions about the functionality, purpose of the feature and clarify the requirements. As a QA engineer, I prepare high-level test cases which will make sure everything works as expected in the next steps of the development process. We use Jira here for keeping our requirements and Google Spreadsheets to track the traceability between requirements and tests.
- Test Planning
When the high-level test cases are ready I sort them by priority. Also in this step, it is essential to understand if we have dependencies with other functionality and if so include them in the planned checks.
- Test Design
Based on the requirements and high-level test cases, I go on to create detailed test cases and figure out what test data we need to run the different tests. And think over the test data we need for tests. Then I create automation tests for the most critical cases. To automate tests, we use the Assertible no-code tool in tandem with the Ngrok service, which provides Assertible with a secure and safe connection to our internal services. For our manual test cases and checklists, we use Google Spreadsheets.
- Test Environment Setup
I ensure the component versions are in the test environment and prepare test data for test cases.
- Test Execution
As I mentioned, I’m from the backend team, so I do testing mostly on the backend side where the logic is stored. I execute manual and automated test cases that were created on the Test Design step in our integration environment and check the feature works according to requirements, API calls correspond to the expectations, and ensure there is nothing suspicious in the logs.Also at this stage, we check the implemented functionality has not introduced new bugs, and run a suite of regression tests in Assertible. A regression test suite is a set of tests that cover existing functionality and are continually updated and extended. After a new feature is implemented, the tests developed for it go into the regression suite.I also run some tests on the front-end side to make sure it integrates well with the backend. I report the issues which were found during testing and once the defects are fixed, I retest functionality to ensure that the defects have been resolved and haven’t introduced new bugs. We have a lot of monitors and dashboards and a good logging system where we can see the system state in real-time and it is actively used during testing to check the status of the services.One of the important steps here is acceptance testing. This is the final test execution step which includes two activities in our process.
A. Mob testing: This is an activity in which we get together with QAs from other teams and our Product Manager and check new features against requirements. As soon as this step is done we can deliver the feature to production.
B. Acceptance test: Our partners can also make an acceptance test and ensure that the product meets the expectations.
- Test Closure.
I ensure once more that critical bugs found are fixed, set the regression schedule and post the report about the test results and links to test documentation and new automated tests in the Jira ticket.
All of our features go through these steps and we think about quality from users’ and partners’ perspectives. For example, during hack days for the Recommendation system, we checked all used internal components were integrated well without any issues with existing functionality. We checked if the customer could configure the recommendation collection and made sure that this collection is available for the user and shows unique content to each user based on their data.
This is our current process which constantly changes and adapts for business purposes. A well-designed QA process can help ensure that products meet the desired level of quality, and functionality, while also minimizing the risk of defects and errors.
_______________________________________________________________________________
Find out more about Magine Pro’s extensive library of technical resources and experienced team that enable our partners to deliver seamless streaming experiences on all devices here. You can also head over to our experience page to learn more about our partners.
Don’t forget to subscribe to the Magine Pro e-newsletter to stay up-to-date with our latest news, partners, products, and to find out which industry events we’ll be attending next.
Revolutionise your content delivery with AI-based recommendations
In this post, we discuss the advantages of a recommendation system for video streaming services and explore its positive impact on brand and content discoverability. Additionally, we gain insights into a new AI-powered recommendation engine by Magine Pro which will soon be released to our partners.
OTT streaming platforms offer an extensive range of content to their users. However, people often find it challenging to choose from numerous options, and may become quickly overwhelmed, resulting in poor choices or no decision at all.
According to consumer research at Netflix [1], a user typically loses interest in selecting content after browsing for just 60 to 90 seconds and reviewing 10 to 20 titles. If the user cannot find anything of interest, there is a substantial risk of them discontinuing the service.
Recommendation systems provide a solution by offering a personalised experience to users that make them feel as if the streaming service was created exclusively for them, which enhances organic engagement.
Where we started
During our last Hack Days at Magine Pro, we challenged ourselves to create a recommendation service for the platform. We brought together a cross-functional team of engineers from the Backend, Infrastructure, and Media teams to tackle this problem from different domain perspectives.
The aim was to create an end-to-end real-time recommendation system for a streaming service. Modern recommendation machine learning algorithms use datasets of users, items, and their interactions as inputs. This data has to be streamed, cleaned, transformed, and inserted into the model for the training.
Using modular programming, a technique of breaking down a product into smaller, self-contained units that can be developed and tested independently, we built separate components and went through all stages of the recommendation system creation process.
As a first proof of concept, we brought together the following services:
- Mux Analytics is used to examine clickstream data and get insights into video engagement and has an integration with AWS, our main cloud provider
- AWS Kinesis Data Streams provides the ability to stream large amounts of data in real-time
- AWS Lambda can be used to perform extract, transform, and load (ETL) jobs on the protobuf-encoded data we get from Mux
- AWS Personalize is a self-contained AI service that takes input data according to the schema and generates various types of recommendations
That worked like a charm! As a next step, our team tried to reduce the costs since using MUX/AWS integration and AWS Personalize was expensive. We also wanted to experiment with different recommendation engines to find the best suitable for our needs.
Minimising the costs
The team started to think about how we can make this feature less expensive for Magine Pro and our partners.
Here are the steps we performed to decrease the costs:
- Replaced the Mux Analytics module with the Apache Kafka, a distributed streaming platform, running in our AWS infrastructure setup
- Created a separate backend service for the ETL jobs, which listened to the Kafka topics, modified and enriched them with additional information, and provided the resulting data further
- Integrated Gorse, another recommendation engine, into our system, utilising Redis as our in-memory data store and a fully managed Postgres relational database, both running on AWS
The modularity of the system gives us flexibility in many ways. For example, we can introduce yet another recommendation engine and compare them to decide which one provides the best results. No need to rebuild the entire setup; only one component has to be modified.
Key concepts
Under the hood, Magine Pro uses a recommendation engine that automatically chooses the best machine learning technique for the most accurate results. The model processes real-time clickstream data from the platform, capturing diverse user behaviours. The following recommendation types are supported:
- Most popular
- Latest releases
- Because you watched X
- More like X
- Top picks for you
Magine Pro’s recommendation system includes a vast selection of diverse recommendation types which can be customised to align with your business requirements.
Recommendation algorithms
There are many algorithms that can be used in recommendation engines, let’s look closer at the most popular ones. Recommendation algorithms are categorised into non-personalised algorithms that provide identical content to all users, and personalised ones that recommend unique content to each user based on their clickstream data.
Latest Items algorithm shows the latest items to users according to timestamps, which allows a new item to be exposed to users in time.
Popular Items algorithm counts positive feedback within the specified window interval and chooses the most liked.
Item-based and user-based collaborative filtering algorithm discovers the similarities in the user’s past behaviour and makes predictions for the user based on a similar preference with other users.
It uses a simple yet effective approach – people often get the most helpful recommendation from people with similar tastes.
Matrix factorization algorithm identifies the relationship between items and users and returns recommended results.
Recommendation system by Magine Pro offers all these options, utilising a hybrid approach that incorporates a diverse range of rating and sorting algorithms.
The Cold Start Problem
Cold start problem refers to a situation where recommendation engines cannot provide optimal results due to suboptimal conditions, similar to a car engine having difficulty starting in cold weather. The two categories of cold start are item cold start and user cold start.
Cold start problem:
What to recommend to a new user U4?
Who to show the new movie I4?
Collaborative filtering algorithms can find items to recommend when the number of user actions on an item increases. However, they fall short in case of new users or items since the model cannot predict anything for them.
Our recommendation service provides suggestions of the most popular and latest items to new users to solve the cold start problem.
Evaluation
Model evaluation is an essential part of the work on recommendation engines and refers to estimating the performance of a recommender system.
The system automatically reserves a small part of the feedback to make an evaluation of it. The aim is to maximise the probability that the user likes recommended items so the positive feedback rate can be used as a metric.
In the recommendation engine, we use standard ML evaluation metrics to evaluate ratings and predictions: precision, area under curve (AUC), and recall, where 0 means the worst and 1 means we have the best-performing ML model.
As you can see on a screenshot from the recommendation engine dashboard, our model demonstrates impressive performance.
What’s next?
Currently, our team is going through the final steps of feature development, evaluating results to ensure that the system we have built is resilient, highly available, and provides the best possible results.
We are also looking into the potential of integrating a content promotion feature for our partners.
Furthermore, we have long-term plans, such as integrating another recommendation engine into our modular system and creating a scalable testing solution to select the most appropriate model that meets our partners’ needs.
Conclusion
This post has highlighted the significance of recommendation systems in OTT streaming services, explored various algorithms, and delved into the details of recommendation engines. Magine Pro is soon launching a new recommendation service that serves as an example to highlight common issues encountered in creating such services and to introduce frequently used performance metrics.
Don’t forget to subscribe to the Magine Pro e-newsletter to stay up-to-date with our latest news, partners, and products, and to find out which industry events we’ll be attending next.
How to set up a streaming service for success before signing a service contract
You’ve heard the old adage about making sure you have a solid business plan before you start any big project, and that saying is truer than ever with streaming services. Create your brand from the ground up, outline your payment options, structure your content and create a marketing strategy.
GET A PROFESSIONAL TO CREATE YOUR BRANDING FOR YOU
You need a great brand, and you need it to be unique. You need a catchy name, one that people will remember. You also need a logo and colours that will make your brand stand out from the crowd. You can create your own branding or hire someone to do it for you. It’s better to have someone with experience and knowledge of how streaming services work.
This will cost money, but it’s an investment that will pay off in the long run. A professional will be able to create a logo and colour scheme that reflects the feel of your brand, and they’ll make sure everything is designed in a way that’s easy to read on different devices.
WRITE TERMS AND POLICIES FOR YOUR SERVICE
Yes, most of your customers will not read your terms and conditions or your privacy policy. You need to have them anyway for two reasons:
- Payment gateway and app store providers will request them before you can sign a contract or release apps.
- They define the relationship with your customers. They give you a framework of what you can and can’t do. And tell your customers what they can and can’t do.
Writing legal texts usually takes the longest so you best get to it as soon as possible. If you are wondering what should be included you can have a look at other streaming services.
SIGN UP FOR A PAYMENT GATEWAY
Earning money with your content is probably your most important goal. If you want to sell subscriptions or rental content you will need a payment provider that
- can handle transactions
- support your preferred payment methods
- allow you to handle refunds and payment disputes
- helps you to prevent fraud and manage risk
- gives you the necessary financial reports
Magine Pro is already integrated with Adyen and Stripe, two reliable and professional payment gateways with a wide range of supported payment methods. Signing up with them can take some time. If you have a business plan and at least an outline of your terms and conditions you can start the conversation with your preferred provider.
When the contract is done the technical integration takes only a few hours and you can test the full purchase flow the same day.
You can also sell subscriptions through apps. Besides an app store account you need a merchant
agreement with the store provider. Don’t let the revenue share discourage you from using this opportunity. Apple, Google and Amazon have launched Small Business Programs that reduce the revenue share from 30% to 15%.
STRUCTURE YOUR CONTENT
Content is king! That saying still rings true. Potential customers will choose a streaming service mainly because they are interested in the content.
It is one thing to transcode the content. The big organisational part is actually knowing how you want to sell it. You can plan ahead by structuring your content into bundles.
Want to give access to everything? – Create one bundle.
Want to sell theme packages? – Create a bundle per theme.
Want to combine subscriptions with rentals or Pay per views? – Create a bundle for the subscription and one bundle per rental or Pay per view.
Prepare your metadata:
- Get images: 16:9 and poster
- Have a title, short synopsis, cast, directors, genre and any other essential information you think is necessary to present your content
- Add labels. In the Magine platform labels are used to categorise content e.g. for collections without presenting the label to the customers.
Last but not least, think about translations. If you offer your service in multiple countries with different languages you will need translations of your metadata.
CREATE A MARKETING STRATEGY
For your streaming service to stick out from the crowd you need to know your target users very well. If you know where they are it will be easier to find channels to address them. Partnerships can help you to increase your reach and raise brand awareness.
At Magine Pro we can support your efforts with:
- Dynamic links
- Push notifications
- Integrations with content discovery services as Just Watch and PlayPilot
- Marketing integrations through Google Tag Manager e.g. Gift cards, referral and affiliate programs
And when you think about the price point for your content, don’t give everything away for free. Sure discounts help to increase the interest in your service. Giving away your content for free first in a trial and then with 100% discount will leave your customers wondering if your content is actually worth something. Be brave, charge something even if it’s nominal.
PREPARE YOUR SERVICE FOR SUCCESS
Finding the right vendor for your streaming service can take time. There are plenty of details to discuss. Use the time to prepare yourself and your team. Planning out your streaming service before you agree to any contracts will make the process more manageable after the agreement is signed. It will help you to launch your streaming service on time and beat the competition. At Magine Pro our onboarding specialists will guide you through the setup process.
Magine Pro offer a flexible OTT platform that can be customised according to your needs. Create a video streaming service with us and monetise your live events, linear and VOD content seamlessly through subscription, advertising and/or transactional business models. What’s more, we have a team of experienced experts on hand who can help you launch and grow your OTT business successfully. And if you’re interested in learning more about how we can help you build and launch a successful OTT business, get in touch with us today.
Q&A: FlixOlé’s, María José Revaldería on the realities of running a video streaming service
Director-General of FlixOlé, María José Revaldería provides an insight into the conception and growth of Spanish Cinema streaming service, FlixOlé.
Director-General of FlixOlé, María José Revaldería provides an insight into the conception and growth of Spanish Cinema streaming service, FlixOlé.What inspired the creation of FlixOlé as a video streaming service?
One of the objectives of FlixOlé’s launch was to give young people access to the best films that Spanish cinema has created in its entire history, with the idea of promoting curiosity and knowledge to new generations on the national cultural heritage throughout the movies.
From an operator’s perspective, how big is the market opportunity for a thematic streaming service like FlixOlé?
FlixOlé has the most extended Spanish film catalog in the world; we believe the market is wide enough with ante extended to all Spanish film lovers around the world.
What opportunities or challenges have you faced as you’ve built the brand?
Our biggest challenge was the film restoration process. Recovering movies, we had to ensure the highest quality sound and images. At the same time, the platform had to have a sensitive price to be affordable to all pockets.
How important was it to understand your launch market and target audience prior to launch?
We as a company have been dedicated to film distribution and production for several decades, we were aware of consumer taste and viewing habits, which helped us to create an appealing content environment.
What positive early traction did FlixOlé achieved after launch? And have promotional partnerships played a role in this?
The alliance with a partner that distributes TV ́s, which included a FlixOlé subscription in its final price, was decisive to the growth of subscribers and income.
Later on, we launched FlixOlé with the third-largest telecom TV operator in the country, which was a determinant of continued growth.
What are FlixOlé’s long-term growth plans? And how does a platform provider like Magine Pro enable you to meet those goals?
We are currently dealing with several telecom operators to offer our service, and thanks to our partner Magine Pro the integration of our service is possible in the best conditions.
____________________________________________________________________________________
Read the Q&A in full in our latest white paper, Thematic Services: The future of OTT? and find out what advice Maria offers those considering starting their own streaming service.
To find out more about FlixOlé, a streaming service offering unlimited access to the world’s largest library of Spanish movies on multiple devices, check out our case study. For more information on Magine Pro’s OTT solutions and services, head over to our website.
OTT Content strategy: the secret to streaming success
It’s no secret that content can make or break the success of an OTT service. Viewers have more choices than ever before, so you can’t expect subscribers to stick around if you don’t offer the content they want to watch (or enough of it). Putting an OTT content strategy in place is important for every video service that hopes to see positive subscriber growth.
What is a content strategy?
It’s essentially focusing on the planning, creation, and delivery of your content, whether it’s video content for your service or copy for your website and marketing channels. It boils down to asking yourself what content you should be producing or buying for your video streaming service and why?
The goal is to offer sustainable, high-quality content that engages your viewers. The content you provide should ultimately inspire them to act, whether that’s to subscribe/purchase or return regularly to watch more.
Content strategy starting points
Here are a few things to consider before you begin outlining your OTT content strategy:
- Define your end goal: How can an OTT content strategy improve your streaming service? Before you begin mapping out, figure out what you want to achieve. Do you want to see an increase in subscriber numbers, for example? Reduce churn? Or, have it help inform your decisions around content acquisition or production? Knowing your end goal will help you devise a strategy that can meet it.
- Audit your content: Before you begin making plans to produce or purchase more content, review what you already have. What are your viewers watching? How long are they watching for? Are they engaging with content through other channels such as social? Understanding what content is already working and why will help you decide what you need more of or what needs to change.
- Strategy & Planning: Your OTT content strategy should always align with your communications and marketing strategies. Planning for content production/purchases is important but your timeline for release should be carefully considered. For example, some content may be topical and worth waiting to release until relevant. Regardless, you should always align new content releases with your marketing plans to boost its chances of discovery online.
- Maintain & Track: Continually review your content strategy throughout the year and be ready to adapt and make changes. Most importantly, keep reviewing your library and track what content is working. Global services should also consider location – some content may show low viewing numbers overall but could be particularly popular in some parts of the world.
For more guidance, check out our Top 5 Content Strategy Tips. You can also find out more about Magine Pro video streaming services, and in particular how we can help you build your own successful OTT service, here.
Don’t forget to subscribe to the Magine Pro newsletter to stay up-to-date with our latest news, partners, products, and to find out which industry events we’ll be attending next. You can book in a meeting now on our events page; meet the team and demo our OTT services firsthand.
Introducing PassionFlix by Magine Pro
A successful OTT service will actively engage customers with high-quality content, strong user experiences, and attractive pricing models. However, to scale and grow sustainably, services need to find new ways to stand out from the crowd. Competing for consumer time and wallet share isn’t easy, but you can boost your chances of success by ensuring you have a well-defined product, target audience, and marketing strategy in place before launch. That being said content is critical and the key to real long-term success for any OTT service.
Niche video services that appeal to particular interests and audiences are on the rise and provided they can offer original high-quality content, that’s not available everywhere, they are likely to see better user engagement levels and much lower churn.
We recently partnered with TV and movie producers to launch a new online video-on-demand service, PassionFlix. PassionFlix is home to a library of classic romantic content, as well as original movies and series that are based on best-selling romance novels and is designed to appeal to those actively interested in romance-themed content. Since launch, PassionFlix has seen considerable subscriber growth month-on-month, and with new titles released each week, viewers remain engaged and return regularly for more.
To find out more about PassionFlix and what went into creating this unique service, check out our case study.
Here’s a sneak peek at the service itself:
What is a CDN? How Content Delivery Networks Work
Are you thinking of investing in a video streaming service for your content but want to know a little more about the tech involved? You’ve come to the right place. Content delivery networks are an important part of Magine Pro’s delivery architecture and in this post we’ll explain how they work and why they’re essential to an OTT service.
First and foremost, what is a CDN?
A content delivery network (CDN) is a system of servers designed to efficiently deliver content, such as music, images or video, to requesting clients anywhere in the world. To do this effectively, each server houses a copy of the original content, which it can then distribute to users within range. A large-scale CDN will have servers strategically positioned around the globe, enabling people thousands of miles away to access the same great content as quickly as local communities that live close to the original source (for instance a local TV broadcaster).
CDN’s help companies like us to scale up and deliver great content to underserved communities around the world. They also help to ensure a seamless user experience, as the content doesn’t have to travel as far to reach the viewer, so they’re guaranteed quick access, great performance, and minimal latency; all pretty important for us here at Magine Pro.
How do CDN’s work?
We send a continuous stream of video file pieces from our systems to requesting user devices, be it a phone, tablet or laptop, ensuring our users and our partner’s users can watch and enjoy content anytime, anywhere.
Before CDN providers, companies like Magine Pro would operate with only one server (with several big hard drives) and deliver video data over the internet to requesting user devices. However, one computer can only perform so well. Adding more servers to accommodate more users though is inefficient and costly and would require additional internet bandwidth. Buying more computers isn’t the solution. It doesn’t scale well nor does it perform well.
Enter the CDN provider. A content delivery network provider has lots of secure servers scattered across the country (or several countries for that matter) and allow their clients to put copies of their video data (or any kind of data) onto them. For Magine Pro this means that when a user starts watching a video stream, the data is sent from one of these servers instead of our own. Consequently, we’re able to scale up the number of users on our services and our partner’s, without having to worry about scaling out our own central storage servers. And as the CDN is in close proximity to the user, they get a seamless streaming experience.
Overview of a Content Delivery Network (CDN)
CDN’s are integral to any OTT service looking to scale and deliver content seamlessly to audiences around the world. If you keen to find out more about CDN’s and how they enable Magine Pro to deliver great content globally, check out our case studies or get in touch!