From Posts to Personas: Data-Driven Patient Segmentation from Social Media

Imagine you're launching a health care campaign and need to understand the people you're trying to reach. Not just demographics. Real people, their concerns, frustrations, motivations, and beliefs. To help with this, Klick Health’s Applied AI team sponsored a group of UBC Master of Data Science (MDS) Vancouver students to explore a proof-of-concept methodology for statistically validating persona segments.

Klick Health is the world’s largest independent, full-service marketing agency and commercialization partner for life sciences. One such solution is Klick Virtual Personas (KVPs), its AI powered role-playing agents, which inspired this capstone project.

Currently, the persona generation process is backed by strategist expertise and deep primary research. This capstone explored whether bulk social media sampling could complement the existing expert-led process.

The project was broken down into five key stages: pre-processing, relevance filtering, feature annotation, clustering on those annotated features, and statistical validation.

The students spent most of their time on clustering and statistical validation.

In the pre-processing stage, it was provided with raw de-identified public social media data for academic purposes containing 143,000 social media posts about hair loss.

The first step was to clean the data: removing URLs, eliminating duplicate posts, filtering out very short posts, and keeping only English-language content. But clean doesn't necessarily mean relevant. Not every post about hair loss is a patient describing their own experience. The next question for the students was how do they find ones that are.

To answer this question, the students created a relevance gate that looked at the posts to see if any were actually related to hair loss. If yes, it is kept, if not, it is filtered out. In the end, only the relevant posts were kept and annotated.

After all this pre-processing and relevance filtering, the students ended up with 83,000 posts in their dataset.

The students then used AI to analyze and label each post. The team extracted 31 features including: sentiment, treatment opinions, concerns, experiences, emotional responses, and treatment behaviours.

To ensure accuracy, the sample posts were manually labeled and then compared against AI results . The students then optimized prompts using an AI-based prompt refinement system.

Next, the students applied unsupervised clustering to identify groups of patients with similar experiences and behaviours from the annotated social media posts. They tested three clustering methods and found that both K-means and hierarchical clustering independently identified the same four patient groups. The team then validated these results by showing the groups remained consistent when the analysis was repeated and were significantly stronger than patterns produced by random chance.

These four groups became the working patient personas: the Curious Lurker (26%, own their hair loss but haven’t started treatment); the Armchair Advisor (36%, give advice but rarely discuss themselves); the Frustrated Trialer (20%, treating, anxious, and struggling); and the Satisfied Committer (18%, settled into a routine and often recommending it). Two caveats: each post counts as one voice, so frequent posters weigh more, and the groups describe the online hair-loss conversation, not a representative patient sample.

The four personas are just the top layer. The same method splits them further where the data supports it, into 11 smaller sub-personas and 7 named micro-segments, and stops once the pieces are too alike to tell apart. That could give Klick a zoom control: four broad personas for a campaign, or finer segments for more targeted messaging.

The students delivered a prototype pipeline as their capstone deliverable. The approach shows promise as a possible complement to Klick's existing research methods.

Explore our Data Science Programs Explore Other Data in Action Stories