Defining and Measuring "Performance": How are you driving incrementality for your brand?

TJ Walz, Senior Director, Data Strategy & Analytics, MMGY Global, HSMAI Marketing Advisory Board Member

The HSMAI Marketing Advisory Board had a thought-provoking discussion around measurement and how the travel marketing world defines performance and how we attribute performance to ad spend.

Some questions for your marketing team to consider:

Are you making sure that you’re thinking about the difference between correlation and causation as it relates to marketing performance?
Do your teams internally think about correlation-based attribution to describe performance from marketing spend? Or do you think about it in terms of a causal impact that is resulting from the marketing spend?
How do you have conversations with stakeholders about revenue vs. ROI?
What is the relative difference between the return as it’s defined on a correlation based versus a causation based? What is that like factor difference that you’re seeing?

Read on for viewpoints from our discussion on these key questions.

Are you making sure that you’re thinking about the difference between, correlation and causation, as it relates to marketing performance?

Most of the time when we speak about performance, we are talking about a correlation-based performance, or measured output. So, the attribution of the conversion is being given to the ad, even if the person would have converted anyway. It’s important to always be thinking about the difference between the cause, the correlation, and causation effect.

Do your teams internally think about correlation-based attribution to describe performance from marketing spend? Or do you think about it in terms of a causal impact that is resulting from the marketing spend?

Approaches to consider:

We break it down by marketing tactic and type. When we’re looking at lower funnel activities, we are often relying on measurement for performance on that last click. We know that the guest and customer is at their moment of need. When we look further up the funnel, that’s when we start thinking about causation.
We are tracking based on a last click, and attribute return on ad spend in this way in many cases. It’s a flat or perhaps even a negative return on ad spends, depending on how high up the funnel you are and the marketing messages. We will always track an ROI, because it’s this simple math equation, but we do look at influence. We have a great analytics team and marketing performance team. They do a lot of great work in terms of incrementality and looking at attribution for broader campaigns.
Culturally, we use correlation as opposed to causation because there’s different competing budgets. And when different departments are looking to stake their claim on what they produced, sometimes those numbers get a little subjective. So, internally we certainly strive to make decisions based on causation, but when we report out, it’s often correlated. This can have a negative effect of perpetuating a model that over promises in what was delivered in revenue generation.
There’s also a point where the data from a correlation standpoint is compelling enough that you can leave an inference, into a causation, particularly if you measure it against market share for a particular channel, and you track the growth in market share for brand.com or OTA based on that campaign. If you see a direct increase while the campaign is running, there’s an argument that could be made to say that this contributed to the win, if you will.

How do you have conversations with the owners about revenue vs. ROI?

It’s a time investment that you have with your owners on an education piece resetting expectations to not be fixated on ROIs.
When you look at the mid funnel tactics, owners are understanding more that it’s awareness based and that the ROI is not always there. They know that what we’re doing is getting the name out there. They’re okay in knowing that they don’t need to have this random ROI that they need to strive to hit this goal that somebody made up at one time…revenue should be the most important, ROI should not be.
Being able to paint the picture internally is an important one. It’s certainly a conversation that I have with our clients and in onboarding processes with new clients. That’s one of the first conversations I have is that we’re going to have a different conversation about what it means to be performing. All the correlation-based metrics are still going to be there, but we need to reestablish what it means to actually put a dollar into the marketing campaign and understand what is coming out of it. Especially when you get to talking about branded search terms and very low funnel display targeting.

What is the relative difference between the return as it’s defined on a correlation based versus a causation based? What is that like factor difference that you’re seeing?

The one test or incrementality analysis the key takeaway for me in the anecdote that I share to hotels, particularly when they’re asking about performance marketing, and the value of our performance marketing program, I share that key study with owners. So, they can see the value that maybe they had never experienced on those channels. We want to make sure that they understand what we do and how we do it and what they can expect from a return perspective.
Something to consider when you talk about the true return, is also considering what that guest or traveler’s lifetime value is. That’s not always considered hard return on an ad spend. It is all the way that return is defined. Right?