How to use the right data for growth marketing in a privacy-centric world

This article was contributed by Jonathan Martinez

Data has always been a critical component of the craft of growth marketing. From being able to pinpoint the time of day when users convert best on Facebook (or as the company is now known, Meta), to testing the best email subject lines (and the ongoing debate of whether to use emojis), to hyper-analyzing each step of the product funnel to eliminate drop-off. This is all changing because of our shift to a privacy-centric world. Companies like Apple are now enforcing stricter user-level data sharing. States are increasingly looking towards privacy laws like the California Consumer Privacy Act (CCPA), which gives consumers more control over their personal information which businesses can collect.

Due to this ever-changing landscape, data that growth marketers have traditionally used to make optimization decisions has become increasingly inaccessible. Solutions are needed now that we have far fewer data to work with. Without the critical data component of growth marketing, startups and corporations are equally impacted. So, how do we adapt to an environment which is far less data-rich than before?

Data degradation

​​First, we need to take a step back to a time before recent privacy changes to understand why this is a transitional time for growth marketing.

We’ve largely lived in a world where access to user-level data has been abundant and rich. But being able to analyze conversions to this level of specificity: e.g., a 34-year-old male in San Francisco in the top 10% HHI, is swiftly coming to an end.

Governments and companies are slowly but concretely making changes to the types of data that ad platforms can send and receive. In one of the latest examples of this industry shift, Apple forced app developers to display prompts asking users whether they’d like to share their data with 3rd parties in iOS14. According to Statista, the opt-in rate for this specific prompt is 21% as of September 6, 2021, which is unsurprising. This means that 79% of users are unwilling to provide their data to ad platforms in this instance, who then can’t pass the data back to growth marketers. It’s a trickle-down effect for growth marketers, who have historically relied on this information to make data-driven decisions.

Incrementality testing

So, how can growth marketers solve for not having as much data at their disposal? A concept which I’ve always considered vital for growth marketing must now become ubiquitous for every business.

The most accurate way to measure and understand marketing efforts is by conducting incrementality tests. The easiest way to explain incrementality testing is that tests show the true lift in conversions from marketing efforts. For example, how many users would have converted last week if a specific marketing campaign were turned off? This type of testing allows us to determine how our efforts are impacting consumer behavior. There are multiple ways to set these tests up, from hacky to precision accuracy.

For the sake of keeping this discussion relevant to startups without necessitating data science swarms, the following approach will describe a simplified method that will still get reliable results.

Above: Example illustration of test tubes

Image Credit: ©Jonathan Martinez 2021

The test tubes in the illustration above provide a visual representation of an incrementality test. Both tubes show the number of conversions (i.e., purchases) that a company acquires when a Facebook campaign is turned on or off. The first test tube shows the Facebook campaign on, and the second test tube shows the Facebook campaign off. We can observe that the first tube had approximately 40% more conversions (or green fill), which would be the “incremental” lift in conversions. These are the conversions that resulted because the Facebook campaign was on.

So, how do we go about setting up this incrementality test?

Data for growth marketing: an incrementality test example

Now that we know the basics and reasoning for incrementality, let’s dive into an example test. In this test, we’ll determine how incremental a Facebook campaign might be.

I like to call the setup for this test, “the light switch method”. It involves measuring conversions when the campaign is live, switching the campaign off, and then measuring the conversions again.

During testing

While the test is running, it’s imperative that no changes are made to the campaign or other channels and initiatives which may be live. If you’re running this test, and then launch something like a free one-month promotion, the conversions would likely spike and invalidate the data. This method relies on keeping everything consistent throughout the testing period, and across your growth marketing efforts.

Leveraging results

Above: Simple incrementality example analysis.

Image Credit: ©Jonathan Martinez 2021

In the example above, the Facebook campaign was live between January 8 to January 14, which resulted in 150 sign-ups. The campaign was then switched off between January 15 to 24, and 50 sign-ups still occurred within this second period.

Cost/sign up: $16.66

Cost/incremental sign up: $50.00

These results tell us our Facebook campaign is 200% incremental. It’s a simple example, but this test provides us with our incremental costs, which we can now apply and compare against other channels. Although there is less user-level data to optimize Facebook campaigns, incrementality tests are still a powerful way to understand the effectiveness of the marketing spend.

The power of scalar utilization

As times change and with politicians continuing to enact legislation forcing company action, I believe we’re moving towards an incremental and scalar-based attribution model in growth marketing. Last-click attribution will be a concept of the past, as this relies heavily on user-level data.

A scalar, as defined by Encyclopedia Britannica, is “…a physical quantity which is completely described by its magnitude”. In growth marketing, the use of scalars helps increase or decrease a channel or growth medium’s metrics, based on tests or historical data. There is a myriad of uses for scalars, but let’s analyze a timely example.

With the introduction of iOS14 and Apple’s SKAD timer effectively cutting all attribution data after a 24-hour window, app-first companies have scrambled for solutions to mitigate this loss of information. This is a perfect use case for implementing a scalar. How does one calculate, analyze and implement a scalar to get necessary data?

Take the following example, using Google UAC efforts, which have been impacted because of Apple’s iOS14 SKAD timer. Pre-iOS14, the sign-up> purchase CVR was 32%. With the introduction of iOS14+, the CVR has now dropped to 8%. This is a huge drop, especially considering nothing else has changed in the app flow or channel tactics.

Above: Example data for Google UAC campaign pre-scalar.

Image Credit: ©Jonathan Martinez 2021

The CACs increased from $125 to $526, which would make this channel seem inefficient, potentially leading to reduced budgets. But instead of outright pausing all iOS Google UAC campaigns, a scalar should be applied to account for data loss.

Above: Example data for Google UAC campaign post-scalar.

Image Credit: ©Jonathan Martinez 2021

We can divide 80/19, which is the delta in purchases pre-and-post iOS14. We land with a scalar of ~4.21, which we can then multiply our post-iOS14 purchases by, or 19 x 4.21 = 79.99. Our sign-up> purchase CVR is now normalized back at 32%, which is what consumer behavior typically is on our app.

There are other ways to implement scalars for growth marketing efforts, including leveraging historical data to inform scalars or using data sources that haven’t been impacted — e.g., Android campaigns. By using this method to make metrics more accurate, it helps prevent a lights-out scenario for seemingly low-performing channels.

Looking ahead at the future of data for growth marketing

Many see the degradation of data as a doomsday event. But I see it as an opportunity to become more innovative and to move ahead of the competition. Leveraging incrementality tests and scalars are just two of the strategies that focus on data for growth marketing, which can and should be employed to validate growth marketing spend. If anything, this increasingly privacy-centric era will force us to realize the true impact of our data and growth marketing efforts like never before.

As we look towards 2022, policies will continue to become enforced by governments and corporations alike. Google has already made it known that 3rd party cookies will become obsolete on Chrome in 2022-2023. They will likely also follow in the footsteps of Apple’s iOS enforcement. But as new policies are enforced, new strategies will be equally needed, and tools from mobile measurement partners or business intelligence platforms should be leveraged.

Take this new era in growth marketing to get crafty, because those who do, will eventually end up on top.

Jonathan is a former YouTuber, UC Berkeley alum, and growth marketing nerd who’s helped scale Uber, Postmates, Chime, and various startups.