Saving Lives with AI: Lessons in Personalization and Engagement

Rex Briggs, Stefanie Friedhoff, and Erik Lundberg describe how the Ad Council rose to the challenge of educating people about the COVID-19 vaccination by using AI personalization to get the right message to the right person. The council’s work not only saved lives and reduced hospitalizations, it also demonstrated that AI personalization can increase engagement and improve advertising results.

Since its founding in 1941, the Ad Council has brought together the titans of the advertising industry to collectively do social good. During the COVID-19 pandemic the council embarked on the ‘It’s Up To You’ ad campaign to encourage people to visit ‘GetVaccineAnswers.org’ for accurate and practical information about vaccinations as well as where and how to get them.

The campaign, part of the council’s COVID-19 Vaccine Education Initiative, represents one of the largest public education efforts in U.S. history, with support from more than 300 major brands, media companies, community-based organizations, faith leaders, medical experts, and other trusted messengers. It was crafted to appeal to distinct audiences, and it is arguably one of the most engaging educational journeys a person could take, with detailed information about vaccinations, the effects of mRNA on the body, and the pandemic itself.

It turns out that engagement saves lives. An Ad Council regression analysis of visits to GetVaccineAnswers. org and vaccination rates by Designated Market Area (DMA) found that public engagement with the website increased vaccination rates. For every 300 vaccinations, one COVID-19 death is averted and six hospitalizations are avoided. The Ad Council’s campaign motivated about 2 percent of total vaccinations, according to our research, which was separately confirmed by Johns Hopkins University.¹

This engagement helped to alleviate the immeasurably heavy toll exacted by COVID hospitalization and death, along with their high costs. COVID-19 hospitalization costs $75,000 on average for each patient; the U.S. government puts the value of each life at $11.8 million. Lost productivity due to COVID illness further adds to the cost. In total, the outlay comes to more than $5 trillion per year, or about 20 percent of the GDP.²

We wondered if we could improve these results with predictive AI personalization. In theory, it would save more lives, keep more people out of the COVID wards in hospitals, and lessen the economic cost of the pandemic in the U.S. We determined to test our hypothesis by launching an AI experiment.

The AI application

AI personalization brings together two dynamic elements: message and context. The message is delivered within a certain context, whether it is a place, a day, a time, a website, or a device. Both the message and the context will influence the response. For example, an ad featuring a cowboy delivered to people in a rural ranching community might lead to more engagement than an ad showing the Las Vegas skyline. We define engagement simply by the number of conversions, that is people completing a desired action, in this case seeking more information.

On the GetVaccineAnswers.org website, AI observes different combinations of messages and contexts and their conversion rates. It then learns what underlying factors lead to higher conversion rates to predict other successful combinations. The AI predicts which message in combination with which context will most likely result in engagement. (This approach does not require personally identifiable information (PII) or cookies, which would have been controversial given the political climate.)

This AI-based approach is different from testing specific messages on set groups within the audience because the system can study many more message and context elements simultaneously.

As for developing ads, sometimes termed creatives, the traditional approach is to define segments, consider what motivates each segment, and develop a core message to match. The final output is typically one version of the advertisement. Sometimes there are several versions, or a few variations on the same theme, such as English and Spanish language versions.

In contrast, AI-assisted content creation yields hundreds or thousands of different versions, generated by a combination of humans and AI. The role of the human is to curate the ingredients the AI can use to assemble the ads. The role of the AI is to learn which combinations of message elements work best for different audiences in different contexts, and to automatically apply this understanding to optimize and increase engagement rates. The new way should be more effective at raising engagement than the old way.

The AI process

For our experiment, we worked with the nonprofit Immunize Nevada and its ad agency Estipona, with ArtsAI as the AI tech partner.³ Our first effort started with three images, and three taglines — a total of nine combinations. (If we were doing video, we could have selected female and male voice-overs, different music, and more.) Our pro bono creative partners on this project, Colleen Watny, Javad Ahmadi and ArtsAI, started with the following imagery:

1. An empty church
2. The Las Vegas strip skyline at night
3. A horseback rider carrying an American flag at a rodeo

We devised three headlines:

1. Let’s get back to this
2. Let’s get back to normal
3. Are You A Parent? 45,119 kids under 18 lost their parent to COVID-19

Granted, the first two are not that different. We were curious whether the word ‘normal’ would help, hurt, or make little difference.

The AI predictive personalization technology sorts out which message combinations produce higher engagement with different groups of people. It optimizes for differences in personal context including the content on the website, the viewer’s physical location (zip code, DMA, rural vs. urban), time of day, day of week, whether they are using a mobile or desktop device, and many more factors to determine the experience of a person receiving the message.

It works as follows:

• The purpose of personalization is to use a given person’s context to serve them the variation of a message that is predicted to generate the highest engagement. ArtsAI uses a hierarchical approach to assembling the message with AI to match a given person and their context.
  At the top level, there are templates that define the layout of the message and its included elements. The next level is the included elements, such as the image, headline, and calls to action. After the layout, imagery is generally the most important determinant of customer engagement by context. Headlines and calls to action can also make a difference in engagement rates.
• At the start of the campaign, when the volume of observations for the AI to train on is small, ArtsAI starts with clustering techniques (unsupervised machine learning) to select the creative template that is performing best with a particular cluster. Each impression is placed on a vector in the dataset using one-hot encoding. One-hot encoding, also known as one-of-K scheme, automatically converts the categorical data, such as the type of content someone is experiencing, into Boolean data that predictive AI can readily process.
  ArtsAI also clustered this space using K-modes to reduce the dimensionality so the AI can find underlying patterns more readily. For each cluster, the AI finds the best message template and serves it to the cluster. At this early point in the AI learning process, the message elements are randomized.
• With the accumulation of impression volume, the AI learns which images, headlines, and calls to action best engage different types of people, based on the context data the AI observes. The AI switches to supervised machine learning (ML). Specifically, the supervised ML is logistic regression algorithms because of the small size of the model, shorter training and operational time, and absence of need for calibration. AI generates models for each creative element, starting with the top-level template and ending with each specific creative variation.

AI assisted content creation yields hundreds or thousands of different versions, generated by a combination of humans and AI.

From this process the AI determines the calibrated conversion probability it expects to get after showing a template and a specific creative variation. The question put to the AI is this: What is the probability that the person will engage if they see this particular template and specific creative variation now, in this context?

The creative, or ad, is built individually for each person by first choosing a template and then selecting nested variants of elements with the highest predicted probability of engagement. If the model for a specific element does not pass the quality threshold, the AI serves this element randomly, thus gradually increasing its knowledge.

In essence, ArtsAI’s personalization selects the template and elements with the highest probability of conversion using its prediction for each model. AI personalization is different from digital ad server optimization. It is common to use ML to optimize ad delivery, but the ML used by ad servers does not treat the message itself as a hierarchical association of templates, images, calls to action, and so forth. Rather, the ad server treats each version as a separate entity because it is not assembling the elements itself. The result is much less optimization than is possible when messages are deconstructed and the AI assembles them in real time.

From nine to more than two hundred versions

After seeing a 43 percent increase in engagement rates in the Nevada experiment, we expanded the effort to the six states with the lowest vaccination rates. Our goal was to raise vaccination participation by giving people vaccine facts to combat misinformation. The ‘States’ campaign map shows the state where the ad was delivered. The AI then selected from four background colors, three headlines, and three calls to action for a total of 216 versions (6 states × 4 colors × 3 headlines × 3 calls to action = 216).

Note the commonalities and differences by state, headline, call to action, color.

The Ad Council went on to execute two additional AI-powered vaccination information campaigns, one with 158 versions and the next with 155 versions. Below are some of the images, headlines, and calls to action from these campaigns.

Measurement and results

The Ad Council launched four vaccination information campaigns total, with more than 86 million total impressions, meaning the ad was viewed more than 86 million times with an average frequency of 6.6 views apiece. We applied a randomized control group, by effectively turning the AI off. We could then calculate the incremental lift AI provided in conversion to site visit. We found that AI personalization increased lift by an average of 24 percent and motivated 1.1 million incremental vaccinations. Over 21,000 hospitalizations and a median estimate of 3,503 deaths were averted — we conservatively measured the incremental economic impact at over $43 billion.

These vaccination campaigns probably represent a conservative estimate of the typical impact of this AI personalization technology. Across seven total campaigns measured from July 2021 through the end of 2021 (three vaccination campaigns, two in automotive, one in apparel, and one in health care), covering 168 million total impressions, AI personalization roughly doubles conversion rates from 0.59 percent in the control group to 1.23 percent in the exposed cohort, for a 108 percent increase. (See Table 1)

	Exposure to Site Visit Rate
Randomized Control Group (AI Turned off)	0.5092%
AI Turned On	0.6336%
Percent Lift: (Al Turned On – Control) / Control)	24%
Lift Range (Lowest to Highest)	16% to 39%

Lessons learned

What makes AI personalization different from the previous generation of advertising is that the versions delivered to consumers are individually selected by AI, whereas the decisions in earlier methods were based on rules such as delivering one version on the weekend and another on a weekday.

Rules-based models can be labor intensive to program, especially when there are several layers of rules. In addition, rules-based systems assume that the person writing the rules knows which combinations of image, headline, call to action, etc., will work best for each profile. This rule writer has to determine the set of profile variables that define a person and usually does so fairly simplistically because a greater complexity of combinations becomes too cumbersome.

AI revolutionizes the execution of personalized advertising in two ways: first, by letting the AI use a much wider set of profile variables to create unique combinations, and second, by letting the AI learn how to best assemble message elements to get the highest engagement with each profile.

Over 21,000 hospitalizations and a median estimate of 3,503 deaths were averted — the incremental economic impact was conservatively measured at over $43 billion.

Specific lessons learned from the experiment:

Start small and expand after you succeed. This was ArtsAI’s advice for our first use of AI predictive personalization: Pick only three images, and try only three headlines. This combination generates nine versions in total. After this experience, we found it easier to develop more than 100 versions for our subsequent vaccine communication efforts.

Make sure message elements, especially images, are different. There is no point in testing, say, several shades of green. Also, make the message appropriate for the device. The message, ‘Are You A Parent? 45,119 kids under 18 lost their parent to COVID-19’ is fairly complex and generated far more engagement on desktop devices. A simpler message is necessary for engagement on a mobile device. Keep this in mind when developing message copy. The AI will sort out which message to deliver to a person on mobile vs. desktop for engagement, but this approach to AI can only optimize what human creative partners feed it.

Have a plan for success. If the AI model is successful, how quickly can you scale the approach? How will you train your agencies and creative partners to develop ads in components that can be assembled by the AI?

We probably could have saved more lives and further reduced the burden on hospitals if we had been ready to scale AI personalization further and faster, but we did not go in with a plan to scale. Instead, we were performing an ad hoc experiment. AI personalization requires a different approach to creative development, and it was a research team composing the ads. We did not have the lead agency creative teams trained and ready to create more campaigns with dynamic advertising.

AI personalization roughly doubles conversion rates from 0.59 percent in the control group to 1.23 percent in the exposed cohort, for a 108 percent increase.

Ad impressions in the ‘It’s Up To You’ campaign generated by AI were less than 3 percent of all digital impressions served. Overall, the Ad Council’s campaign engaged people, saved lives, and reduced hospitalizations. So what if at least 60 percent of the messages delivered had been powered by AI? Projecting the lift from 3,503 lives saved from 3 percent, we estimate that we might have averted as many as 70,000 deaths in the U.S. We are proud of the positive impact we made with AI personalization, but we are left to wonder what we could have achieved by adopting AI personalization earlier and more broadly.

Author Bios

Endnotes

1. Briggs, and Brown University School of Public health analysis in 2021, www.globabepidemics.org
2. Becker Hospital Review, October 20, 2021, https://www.beckershospitalreview.com/f inance/average-charge-forcovid-19-hospitalization-by-state.html, Statistical Life Value (SLV), USTSA value of $2.3million. U.S. Government guidance for 2021 is $11.8 https://www.transportation.gov/office-policy/transportation-policy/revised-departmentalguidance-on-valuation-of-a-statistical-life-ineconomic-analysis, Briggs & Brown University School of Public Health Analysis, 2021.
3. More information about these organizations may be found at: https://www.immunizenevada.org/, https://www.estiponagroup.com/, and https://artsai.com/