MDS Spotlight: UBC MDS Vancouver Students Win $10,000 Grand Prize in Kabam Varsity Challenge
For companies like Vancouver’s Kabam, a developer of immersive, and highly social multiplayer games for mobile devices, it relies heavily on digital advertising for promotion and user advertising to generate revenue.
In conjunction with UBC Sauder School of Business, the Kabam Games Varsity Challenge was to create a predictive model that estimates the value of users for one of Kabam’s hit games.
The competition began on March 1 that included a two-day hackathon in the second week of April where participants could ask any last-minute questions. After that, finalists had to give a presentation towards the end of April. Teams were eventually whittled down to the top 4 that competed for the grand prize of $10,000.
Team Sigma, composed of two of our current UBC MDS Vancouver students (Bailey Lei and Betty Zhou) and third year UBC Computer Science student, June Wu, was amongst the 49 registered teams and eventually emerged as the grand prize winners of the Kabam Games Varsity Challenge.
The data sets given contained information on user interactions with one of Kabam’s games, which teams used to determine player quality scores (PQS).
Zhou explained that player quality scores are used to guide Kabam’s advertising budgets.|
“We were tasked to propose a player quality score that would help [Kabam] identify the value of a player in order to optimize advertisement spending and retain player engagement.”
The team was presented with a lot of features to determine the player quality score and it was their job to extract the important ones.
While the team did not know which of Kabam’s game the challenge’s data came from, they felt it was important to get an idea of what a Kabam game play was like. So the team downloaded Kabam’s “Marvel: Contest of Champion” in order gain better understanding of the gameplay and therefore, develop a better model.
The data they were presented was messy and the team spent a few weeks doing exploratory data analysis (EDA) and feature engineering to determine which features were important in designing the model.
Wu’s focus was on the player quality score and made a list of all the items that might correlate or contribute to player quality using “Marvel: Contest of Champions” to justify their score.
Kabam provided teams a mean absolute error (MAE) metric system that helped determine how well their model performed.
“We learned about [MAE] and model complexity in the MDS Program, so we understood how to evaluate our model based on these metrics,” said Lei.
Throughout the process, the team went back and forth doing EDA and getting feedback from each other. By the time the two-day hackathon occurred, the team had 80% of their model completed.
The second round of the competition was about explaining the model to others in a business context.
One of the things that Kabam wanted to do with the competition was mimic a real business setting where people would not be told what to do at every given point.
Lei said that Kabam wanted participants to be creative, use their own judgement and best practices to come about the solution.
He added that the team focused on the business aspect when designing the PQS throughout the whole competition.
Zhou asked the judges why their team won over others and they said that “we were able to really bring the technical side to a non-technical level. The panel consisted of non-technical and technical people and we needed to be able to connect to all of them in order for them to understand what we were proposing.”
Another key to their win was being able to translate the complex formulas that made up the PQS and encapsulate it with words.
“June came up with a lot of math to explain the player quality scores but we were able to link the math to stories,” said Zhou. “One of the judges said the math doesn't say much unless you can connect your audience with a story and we were really good at turning that math into stories to connect to the panelists.”
Some of the key takeaways for the team were knowing how to design a presentation and connecting with your audience as well as keeping things simple.
Finally, Lei highly recommends future MDS students to participate in the UBC Sauder/Kabam Varsity Challenge.
“We learned a lot of new things that we did not cover in MDS like with EDA you look at different packages of Python and for me we learned about the importance of the unit testing our code.” As well, he added the opportunity to work with real data was invaluable and “getting real experience in terms of how data scientists work in the real world.
Vanessa Ho is the Marketing Coordinator for the MDS program. She holds a Bachelor of Journalism degree from the University of Regina.