Industry Changes Dictate Revisiting Measurement Methodologies

Guest post by Michael Kelley, Director of Data & Analytics at Transparent Partners.

Constructing a 360-degree view of the customer upon which to build a reliable cross-channel attribution model for media investment decisioning has always been very challenging, even with good data. Now, privacy concerns have changed the media data landscape through self-imposed publisher walled gardens, deprecated 3rd-party tracking, and an increased focus on 1st-party data, all of which have driven a clear need for reevaluation and change.

These rapid changes in the industry demand that marketers and marketing science professionals shift and broaden their focus to consider multiple methodologies for understanding and assigning value to media channel investments and refocus on what matters most to the business.

Measurement Must Address Business Questions & Without Decisioning, It’s Just Expensive Reporting

Fundamentally, measurement must be able to address a business’ most pressing questions. Measurement that does not is nothing more than expensive reporting. Business questions should be captured in a “Learning Agenda” that should, at a minimum, include prioritization, a timeline/roadmap for addressing, and a RACI matrix. Each learning agenda item must then be connected to the methodology most appropriate for answering the question. And lastly, a decisioning framework, established prior to collecting results from the analyses and testing, must be established to ensure commitment to action based on the results.

For example, if a business wants to understand how its prospecting efforts compare between two paid media channels, e.g., Facebook and Google from an ROI perspective, its learning agenda would include this specific question, along with a sense of priority/urgency, who will be addressing the question, and a selection methodology/approach for analysis and testing to address this question. Triangulating the outputs from MTA, MMM, and perhaps some in-market experimentation may yield that one, both, or neither of the publishers meet or exceed financial ROI targets, which directly dictates the appropriate investment decisions until further analysis and testing is conducted.

Overview of Diversified Measurement Methodologies

Addressing measurement in marketing is nothing new. As marketing channels evolve and/or new channels emerge, and as the martech landscape continually changes (in terms of tracking capabilities, publisher technology, site analytics capabilities, 3rd-party measurement tools, etc.) the favored methodologies employed have evolved as well. While there is no measurement “silver bullet,” the image and list below generally represents the most commonly utilized methodologies.

• Multi-touch Attribution (MTA) historically pieces together user-level consumer path data to understand and estimate the marketing touchpoints that were responsible for a consumer taking a particular action (usually conversion activity).
• Media Mix Modeling (MMM) statistically models marketing channel metrics (spend, clicks, leads, etc.) and other external data (e.g., seasonality, competitor activity, etc.) against conversions (sales, registrations, etc.) to estimate the impact of marketing on business outcomes, typically for the purposes of forecasting and planning/budgeting. (Learn more about the beta release of Rockerbox MMM for Shopify.)
• Incrementality Testing refers to the technique of creating scientific in-market experiments designed to isolate and estimate the incremental impact that specific media has on intended outcomes, which also includes the ability to “scale test” (i.e., test media performance simultaneously at dramatically different spend levels at a fraction of the cost).
• Site Analytics is available through all major website analytics platforms (Adobe, Google Analytics, etc.) and offers either a custom implementation or out-of-the-box attribution solution that provides some user-level insight into click-based media and site activity.
• Publisher-Reported refers to media data provided by the publishers themselves, based on their tracking capabilities, which typically includes metrics such as spend, engagement, and various conversion metrics.

Beginning in 2017 with the launch of Intelligent Tracking Prevention (ITP), Apple has forced the issue of digital privacy onto the advertising ecosystem. With the launch of iOS 14.5, and the deprecation of default-on IDFAs, this drive continues. These changes are having a large impact on the digital advertising industry, prompting reactions from the largest players including Google to Facebook.

Each platform is now operating with its own set of rules. Safari blocks third party cookies, while Chrome doesn’t (yet). iOS prompts users to enable cross property tracking while Android doesn’t (yet anyways). A company like Facebook could theoretically build completely different systems that leverage the benefits of each platform but logistically that would be next to impossible to maintain. Given this, some broad changes to anticipate include:

• Modeling is more important than ever: Modeling has always been needed for harder to track channels like linear TV, radio, and billboards. These techniques are increasingly important to digital channels moving forward.
• Attribution windows continue to shrink: The days of 28 day click + view attribution are gone. Upper-funnel/brand media will now need to look elsewhere for good measurement, but otherwise this eliminates publishers from greedily taking too much credit for media exposures near the end of a long attribution window.
• Deterministic view-through is dying: With the death of 3rd-party cookies, deterministically connecting a viewed impression on CNN to a conversion on your website via cookies is impossible. Sharing other identifiers (Unified ID 2.0 is seemingly getting some traction) may be able to fill in some of the gaps here but the scale available here is unknown.

Given some of the uncertainty around user-level tracking, there has been renewed interest in "traditional" methods, most notably MMM. MMM compliments MTA nicely in that it can fill in gaps where user-level tracking is difficult or incomplete. Incrementality testing is another methodology that can address the shortcomings listed above.

Many enterprise brands are now taking a multi-pronged approach to cross-channel measurement, optimization, and planning, typically covering the “Big 3” (MMM, MTA, and incrementality testing) with site analytics reporting, publish-reported data, and qualitative methods often playing a supporting role (although they play a central role in other ways such as consumer insights, brand health, user experience, etc.)

How Rockerbox and Transparent Partners Win Together

While many brands typically adopt one or more of the “Big 3”, there’s no measurement “silver bullet” for every situation and every brand should carefully bring together the most appropriate specific set of methodologies, along with best-in-class tools and partners, that ultimately brings to life carefully curated analyses and tests that are best suited to address prioritized business questions.

It’s also important to recognize that measurement doesn’t exist in a vacuum and other measurement signals exist that drive decisions within organizations. Transparent works with brands to help evaluate, contextualize, and ultimately “triangulate” all relevant measurement outputs and signals to provide confidence in the estimation of channel/tactic performance and to drive next steps. When triangulating measurement, Transparent applies a variety of best practices, including:

1. Understanding what measurement outputs currently exist and how they are being used in decisioning
2. Understanding the context (seasonality, recency, etc.) of each output
3. Mapping how measurement outputs are used in relation to financial/performance targets
4. Ensuring measurement is connected back to business questions and outcomes

There are also implications in terms of cost and complexity in onboarding a powerful measurement platform and enabling a workable measurement framework. Transparent Partners teams specifically with best-in-class measurement partners, such as with Rockerbox to ensure long-term success and help establish organizational best practices around:

• Data Infrastructure: Create a foundational data and measurement system to make the most critical data easily accessible, clean, and actionable to address consumer and business needs.
• Metric Alignment & Decisioning Framework: Utilize measurement methodologies to reconcile media metrics against business KPIs to create clear view of performance across all channels and establish decisioning framework based on organizational financial targets.
• Organizational Buy-In: Ensure marketing and finance are aligned on “north star” measurement KPIs and commit to action prior to measurement results.

Organizations that recognize mounting economic headwinds but nonetheless choose to address the growing opportunity cost of doing nothing today can be in a better position to defend their budgets by focusing on the right methodologies, technologies, and measurement partners that deliver the necessary intelligence and insight to confidently and most effectively invest their next marketing dollar.