MDS Spotlight: Improving Patient Communication through AI
How can you use artificial intelligence to help improve patient communication? That was the challenge put forth to teams that participated in EasyMarkit’s first hackathon held on April 6, 2019. EasyMarkit is a Vancouver-based company that focuses on developing automated patient communication software specializing in dentistry.
EasyMarkit was looking for a way to make the process of booking appointments a lot easier. Teams were given clinic data and were tasked to see how likely patients were to respond to an appointment reminder phone call or text message.
12 teams participated in the hackathon and a trio of our current UBC MDS Vancouver cohort: Bailey Lei, Alex Pak and Betty Zhou (Team Sigma) participated and ended up with the winning solution.
It was the combination of healthcare and AI that attracted Team Sigma to participate in the EasyMarkit hackathon.
“This [hackathon] was one where we can utilize what we learned in Machine Learning and the MDS program to solve the problem [EasyMarkit] was trying to solve in terms of making a predictive model for patient response,” said Lei.
Zhou added that the hackathon allowed their team to apply their skills on a real data set versus the toy data sets they’ve been using in their course work.
Teams were given such data as geographic location, age and other demographic data. Teams had to see how all the attributes would predict on how likely patients will respond. The goal of the competition was mainly focused on predicting whether someone responds or not to an appointment reminder.
“By getting a model that can predict fairly accurately, [EasyMarkit] can look back at the model and determine which area is a good factor in explaining how well someone responds to an appointment reminder,” said Lei.
For example, Lei said that age may play a role in whether someone responds better via a phone call or text.
Since teams only had 8 hours to develop a predictive model, Team Sigma started the day doing an hour of exploratory data analysis (EDA) on the data set as well as background research related to the data set.
The EDA allowed the team to “get context with the data you are working with,” said Pak.
A Google Cloud Server was used to run the three models developed with Pak doing the model searching and parameter tuning.
While three different models were developed, the team could only submit one so they submitted the one with the lowest approximation error.
“We picked the simplest model that allowed us for fast prediction times and also on how well it can predict on new data,” said Lei.
In Lei’s blog post on the technical aspects of their solution, the team “did cross-validation and initially did a few models that ranged from logistic regression to random forest, but ultimately decided [on] a tree based learning algorithm - Light Gradient Boosting Machine (LightGBM).”
Lei noted in his blog that when the results were announced, the reported test accuracy was within 0.1% of what their model predicted with the validation set.
Zhou believed what stood their model apart from the competition was the fact that they were able to validate their model. “A lot of groups weren’t aware of the importance of validating models, so they didn't know whether it would be generalized to new data. Our model had greater accuracy because we were able to validate our model and tweak it depending on whether it was good or not on new data.”
She added that most teams were using the training data set to assess the performance of their models, which would not be a good reflection of the prediction accuracy of a model on new data.
“[In MDS], we knew how to split the given data into training and validation sets. We used the training set to build the model and used the validation set to validate our model. Other groups used all of the data to train the model and often don’t realize that they need to put aside some of the data for validation.”
After all was said and done, Lei found it fascinating to see how companies like EasyMarkit were applying Machine Learning and how it can be translated to a business model.
As well, aside from each winning a Google Home Mini, Pak said the ultimate prize was being able to make industry connections like meeting the CTO of EasyMarkit and being able to say “it was inspiring knowing that a year ago I would not have been able to do this.”
Vanessa Ho is the Marketing Coordinator for the MDS program. She holds a Bachelor of Journalism degree from the University of Regina.