A podcast for marketing leaders.


« Back

3. Why is it so hard to track marketing's ROI?

There’s not a marketer alive that wouldn’t love to be able to attribute every dollar they’ve spent on marketing directly to revenue, but it’s just not possible. Kyle & Jeff explore an example scenario highlighting the difficulties of marketing attribution, and offer some strategies to make attribution less of a concern.


(Primarily autogenerated, let us know if you see any errors!)

Kyle: Hello and welcome to unReactive where we look to help marketing leaders reclaim their time and sanity by exploring the topics that keep us stuck in a state of reactivity. I’m Kyle Morck, and I’m joined as always by my co-host Jeff Reynolds. How are you today, Jeff?

Jeff: Well, it’s sunny. It’s Friday when we’re recording this, which I guess you’re not supposed to say, but those things come together to make it a pretty good day.

Kyle: Yeah, it’s starting to feel like spring. It’s supposed to snow again next week, but I’ll pretend like that’s not gonna happen.

Jeff: So what are we talking about today, Kyle?

Kyle: Today I want to explore why it is so hard to track marketing’s ROI. And I’ll say when I started working, particularly in the agency world for marketing, I personally had kind of an obsession with trying to track attribution and really figure out, you know, where every dollar was going in a marketing budget.

What I pretty quickly found out was that that was actually kind of an insane thing to try to do.

When you just really think about it most. are not purchasing in any way that is something that you’re able to track through attribution or anything like that. So the idea of trying to figure out what’s that return on investment is actually much, much harder done than said.

Jeff: Well, you’re surely not saying that it’s not valuable to know if your marketing’s working or going through sort of the measurement process at all, you have to believe in that to some degree, right?

Kyle: I believe that you want to try to figure out what’s working and what’s not, and you want to try to tie that to business results, but I think that the reality of making that happen is much more difficult than we like to believe. And then what happens is that in an attempt to measure the effectiveness of marketing spend departments start to shift their focus to what can be measured rather than just what works.

Jeff: So this is this idea that I’ve written about in the past, which is the measurement isn’t the mission, but suddenly what happens is because of the tools and the technology, and frankly the stories that those brands tell, we get obsessed with going after what’s easy to measure, whether or not it’s the right thing to measure.

And I think personally, I think a lot of this is like societal, we are a metrics growth-based society, right? People who start businesses, we have like, you know, are you a million dollar business? How many employees do you have? When people ask you these things, they’re trying to figure out if you’re successful.

We’re obsessed with measuring. And then I think in the marketing realm, when all these tools and technology came out, we got super excited and went all the way into it without really regard to whether or not it fits reality. Right. So I guess my question to you then is can you give us an example of what reality is versus how we sometimes think about it?

Kyle: Yeah, imagine that your business sells some kind of B2B SaaS productivity software, right? And this is something that I, as kind of a productivity nerd, am looking at, looking at the ads for them all the time. And What I’ll describe is , not at all an out of the ordinary customer journey.

So, if I’m talking about this from my perspective, oftentimes I’m on my phone waiting in line to get food, something like that, and I’ll see an ad for this software on LinkedIn or Facebook. And a lot of times I don’t even click on it. Maybe I do, maybe I don’t. But the important thing is that the ad whether it’s the copy, what have you, it connects with me enough to think, oh, that’s something that might solve my problem.

And then I store away the name for later. But in that moment in time, you know, I’m not in a context in place where I want to start…

Jeff: Yeah, you’re not ready to shop. If you’re waiting in line at the fast food place or whatever, that is not the moment that you’re ready to like, okay, how do I whip out a work credit card, make a matrix to figure out if this has the features I want, and whatever. So you

Kyle: Yeah. And on top of that, in this case, you know, it’s usually desktop software, so you can’t even explore it on your phone, but the vast majority of ads are gonna be served on mobile devices because that’s where most of the traffic time and attention is. So then maybe I get back to the office, I jump on my work computer and then I Google the name of that software that I had remembered of, oh, hey, this is something I wanna look into.

And, you know, maybe the company’s running brand defense ads that they’re Targeting the name of their software and maybe that’s what I’ll click on. But a lot of times just my brain mentally skips ads anyway. And I’m gonna go down and try to click on the actual homepage. But the important thing is that I’m typing that name directly into Google

Jeff: So for them just to say it if they’re using attribution software, they would say, oh, look how great Google is for me, when maybe it was a Facebook ad that actually prompted the


Kyle: Yeah, exactly.

And so you explore the site, you read all the copy on it, you see the screenshots, end up signing up for a free trial, but you know, just got back from lunch, have other things I need to do that day. It is not in my plan to sit down and configure new software and fully explore it so life gets busy and I forget about it entirely.

That happens all the time. And then oftentimes, what actually ends up prompting me to go back to that software is that I’ll get their monthly newsletter or something in my inbox, and there’s a very good chance that I don’t even open that email. It might go to my promotions tab and I just see the name of the software or what have you, but like I’m not interested in reading their newsletter at that moment in time, but seeing the name of their software reminds me, oh yeah, I wanted to check that out. So then in that time and place, I might be in the right context where I’m sitting down, have some time on my hands, and I have the time to properly evaluate that software and follow through with that purchase decision. So if you think about that, which I really do think is a very common customer journey. None of those pieces are really attributable to marketing. It’s fully possible that I could go through that entire journey from learning about the software through purchase without having clicked on a single ad or even opened up an email.

But there’s no doubt that marketing played a major influencing factor in my decision to buy.

Jeff: Well, and it’s, and even more okay. There’s two elements to this. You can both not be traceable, right, or not be visible through the attribution system. But you could also, and maybe even more riskily be visible at certain parts in the process that misleads the marketer who is interpreting that data, right?

So like that example where I stopped you earlier, now they’re gonna double down on Google search ads that you are ignoring because that was the one piece of trackable data. Let’s say you had clicked on an ad, but that wasn’t even really the right place in that journey that they should really be worried about influencing the decision.

That was not the key moment. The key moment was the ad that made you aware of it in the first place.

Kyle: And that’s not even to mention all the potential brand groundwork that may have been led prior to even seeing that ad. You know, there’s word of mouth or maybe I was researching productivity software before and went to an aggregator site that was just listing off ”here’s all of the software that matches that”.

And I had seen, you know, mention of it before, I had seen a YouTube video of an influencer in productivity talking about it in their feed. There’s all of those things, even before seeing that first ad that are coming from marketing and that brand awareness, that would never be something you could attribute in any way.

Jeff: Yeah, and I guess this is particularly dangerous in a B2B context when you have very long extended sales cycles and things like this, and you get obsessed with a piece of software or sort of a concept like performance marketing or, you know, whatever

Kyle: Yeah. Growth hacking. Yep.

Jeff: Yeah.

Right. And the problem is that all these tools may give you a sense of control, but it’s like an illusion of control because that dashboard doesn’t show that you read somebody’s LinkedIn post , right? That is unaffiliated with the company that might have shaped your entire buying decision.

Kyle: Also, it’s not just about the technology side of it, cuz there’s something like on the demand generation side, there’s oftentimes this movement to include, ”how did you hear about us?” questions on that free trial form, something like that. And that’s an attempt to get technology out of the attribution game.

But then that’s just relying on the customer’s ability to be introspective. You know can they remember where they first heard about the software and what was influencing that decision? And it also forces single source attribution that if, you know, in that case, there’s a very good chance that I would say, oh, I heard about you through Google, because that’s what I was doing at that time.

So then you start to double down on Google when it actually had very little to do with my decision to purchase that software.

Jeff: A hundred percent. And my example of this is always, this is 20 years ago, I was working in a real estate project in a resort community and we had this similar to self-reporting you know, card. How did you hear about us right when you registered at the sales office and the number one answer by far was signage.

Now, when I first got this data, somebody said, see signage, our billboards must be killing it. Well, no, there were no billboards. Oh, then our directional signs that took ‘em into this resort community, which was sort of off the beaten path, must be killing it. No, there were none of those either.

There’s no signage. It was illegal from the city, actually made it illegal to do such signage. And ultimately when I went back and traced what signage somebody could have experienced. It was literally the monument sign in front of the community center, which is where the sales office was. You know, that’s like a little A frame that said sales office.

That was the only sign that existed. Yet if I would’ve listened to what people are telling me, we would’ve doubled down on the signage budget, of which there was none. It was literally just recency effect. In our audience’s mind, they had saw a sign, they just wanted to give an answer. They’re not interested in being marketing researchers.

They’re literally just trying to fill out your stupid form so they can get to the next step in learning the process. Right. So I guess what I’m saying is counting on introspection or, you know just reflection or memory of your audience is usually pretty weak sauce,

Kyle: Yeah. And, people do not want to believe that they are influenced by marketing. So if you are asking that question, you’re playing into their natural, ” oh. I can’t be marketed to” situation that a lot of people play into.

Jeff: A hundred percent and I just gave a real estate example and you gave a SaaS example. It sure seems like SaaS, you know, mostly an online experience or largely an online experience, even has a bigger advantage.

Kyle: Yeah.

Jeff: In attribution, I mean.

Kyle: SaaS should be one of the only types of products that really could potentially do attribution when you can be a part of that entire chain from awareness all the way through purchase. But if you’re selling through retail, through distribution, or even on Amazon, oftentimes you don’t even have visibility into that entire process.

So you can kiss any sense of attribution goodbye entirely.

Jeff: Yeah, but technology will save us, right? There’ll be more technology that they’ll be refined and solved this problem. Right, Kyle?

Kyle: No, that’s not right. The technology that drives attribution is just going to get worse. Not better, with privacy concerns. Cuz I even think like email, the, there’s hey.com that is trying to make it so that you can’t even track opens on emails, things like that. As privacy becomes more and more of a concern and especially as companies like Apple start to push forward as kind of a brand piece from their side this is just going to become an even bigger problem that we’re way past the point of of the golden age of attribution technology and things are not going to get better.

Jeff: Yeah.

Kyle: When we look at the future of measurement of marketing activities what do you think are some of the biggest pieces that are really what makes that difficult to put into such a neatly defined package as we want to?

Jeff: Well, the first thing is just the human nature, just to say it, that a lot of what we’re dealing with is the gap between what humans want, i e marketers want, which is control, visibility, things tied up in a neat little package. But we can’t have those things, so we just tell ourselves stories.

So that’s, to me, like the underlying fact of what’s happening here. Like why we’re susceptible to believing this is all gonna work so well.

Kyle: It’s a very great point because it really comes down to kind of that wishful thinking of, you know, mediums that we can track the channels that we can track. We want to believe that those are the most important piece of our marketing efforts because we can look to those and have numbers around them that can make us feel better.

And it really is kind of that wishful thinking that’s so dangerous because then we start to only focus our efforts in those places.

Jeff: Exactly, but I do think there’s at least four other that come to mind. Four other sort of more concrete reasons. That attribution will continue to struggle. One is time or time distance. The gap between sort of my interest in something or, research in something and, and my actual purchase.

And if you’re, you know, I work a lot in day-to-day with large B2B companies where sales cycles stretch months or years. And so that time simply makes it impossible to actually accurately track where the influences.

Kyle: like that’s a great point cuz I think it really applies to just about every type of. Business because the gap between becoming aware of a product’s existence and possibly needing that product is oftentimes really long.

Jeff: let alone the trigger to actually make you wanna take action on one, you know, sort of wanting versus needing or whatever .

Kyle: exactly. So what’s number two?

Jeff: I’d say just lack of clarity on what data even matters. Sort of what are we actually measuring? What are the milestones that really matter?

I mean, does a, a website click matter to most businesses, you know? When people say, oh, we need to be able to measure everything. I say, well, open up your refrigerator. Look at all the brands that you’ve interacted with in the last, say, couple weeks a month, or if you’re a college student, maybe months in your fridge, you know, and ask yourself, how many times have you visited the website of those products?

And usually the answer’s zero because, and it doesn’t matter. But yet some marketing manager at Hines is definitely tracking their website clicks, even though it probably has very little. On who’s buying ketchup that week. Right. So, so just I guess in general, lack of clarity on what even matters in what we’re measuring

Kyle: Like I think about that all the time with the context of Coke, cuz Coke is oftentimes just seen as this shining beacon of marketing. And I buy Diet Coke all the time and I don’t think I’ve ever in my life been to Coke’s website. So the idea that their marketing could be measured based on really any metrics.

Cuz like I am very influenced by Coke’s ads and I guarantee you they could never in a million years tie any of their ads to any of my purchases.

Jeff: So we, so far we have time, sort of lack of clarity or understanding on what really matters. I’d also say codependency of sort of channels is really a key one. A simple example is if you run an ad, you know, a direct response ad and take somebody to a landing page for them to purchase, which was it?

Was it the ad or the landing page that mattered most or really mattered?

But our channels are so intertwined. Even if you couldn’t measure them, it’s hard to tell how you allocate sort of percentage of responsibility between those codependent channels.

Kyle: And that’s even just talking about channels, but then you have all of the other pieces of it, of just, you know, brand affinity, positioning, all of those parts that are underscoring that landing page and that ad.

Jeff: Exactly. Exactly. So those are even the harder to measure inherently and harder to measure elements that play a huge part in it. So then the fourth one, so we have time, lack of clarity, codependency of sort of the channels or attributes or all those different pieces and parts.

And then I would say just simply the way the tools are built, the limitations of each of the tools are different. So we’re inherently not gonna be comparing apples to apples. So when we say X percentage of people opened an email, how are we actually measuring that?

Because I can tell you right now that different tools, different analytics tools will measure that different. Some, you know, different devices will impact that different email clients quote, open an email, you know, like preview the email, which launches the triggers, the pixel that you know, fires the pixel that sends the data that said it was open. Different tools have different approaches to that, so you can’t even compare apples to apples. And we did a test once on one certain type of email where we ran three different analytics tools tied to that one email to see how the data came in. And the data came in as much as 20% different in certain categories from one tool to another. And I’m talking base categories, like open rates, click through rates, these sorts of things. And that’s simply because the tools function differently or have a different purpose. So I guess all I’m trying to say is when somebody comes to me all excitedly and they’re like, I have a 30% open rate on my email.

That stat alone is sort of a meaningless stat because I need to understand how they’re measuring that. If an open rate even is that what matters? You know, those sorts of things. So the tools themselves, the technology themselves, and it’s ever-changingness is another hurdle.

Kyle: Yeah. And I, and I think another way of phrasing that is just the relativity of it all. That everything is very relative and you can’t really take any. Number in a silo and really get any meaningful evidence towards what’s going well and what’s not it you really need to be able to take in the entire scope of your marketing activities to have any real understanding about what’s happening.

Jeff: That’s a better way to say that. And I’m gonna steal that cuz that’s, you’re right. Relativity is really the issue. So back to this point on this is inherently a human issue, that we want to tell ourselves stories and make sure everything’s attributable.

Why do you think we’re just so eager like humans, and as marketing managers or leaders to have measurable results in the first place.

Kyle: I think it’s really natural to want to measure your results. First of all, from a leadership side within an organization, you know, marketing leaders looking at their bosses. Their bosses wanna feel like they’re being good financial stewards of the organization. So when they’re looking at your marketing budget, all of those pieces, they wanna feel like they have an idea of what they’re spending their money on.

But it’s also really natural for the marketers themselves to wanna know how effective their marketing is. Like, something that I’ve been thinking about is, Recently I’ve been spending a lot of my time programming, and that’s actually really relaxing for me. It’s something that I end up doing in my spare time, not just because I’m a nerd, but it’s also giving me instant feedback on what’s right and wrong.

Every time that I write a line of code, I can run it and see if it worked, how I expected it to, and know exactly what it’s doing, and that’s very different from my day-to-day work. And one of the reasons that marketing is such a difficult profession is that there is no built-in feedback loops for your daily work.

It’s really a lot more like being a writer than a programmer. You need to have faith in your abilities faith in your taste to know if you’re doing good work. And you need to be kind of committed to it for the long haul. So it makes complete sense that if you don’t have confidence in your marketing and that you are doing the right thing, that you’re gonna look to, you know, oh, I’m gonna spend the day writing a Facebook post and get excited about getting eight likes on it, because then at least I know that’s doing something. Versus, you know, planning out the next year of your brand campaigns.

Jeff: the risk or the downside is you can spend all your day getting your likes and not actually accomplish the thing that matters. And so like what I’m trying to get at is like Stephen King you know? I remember seeing an interview with him where he was talking about not knowing which of his books were gonna become hits or not.

He could not be the judge of it. His job was just to write and create and put it out in the world. And if you think about that, maybe Stephen King’s not a great example, but other, you know, more you know, sort of literature oriented writers that spend years sometimes working on a book. I’m sure Stephen King probably does too. I just don’t know. But you know, some people will spend two or three years writing a book and having no idea what the response is gonna be other than, sort of like little bit of feedback here and there. So I guess what you’re describing is very normal actually.

In a lot of ways, much of what we do is not measurable.

Kyle: One thing that I think is really important to kind of transition a little bit into, okay. That’s why it’s really hard to measure results as a marketer, but I’m not arguing that we shouldn’t be trying to understand what we’re doing well and what we’re not doing well, but I would argue that we need to be looking at a much different picture of our work to be able to do that well. And the first thing that I would suggest is that marketing leaders need to start taking ownership of much bigger metrics and longer timelines within their organization, meaning that you need to feel a responsibility.

You know, quarterly financial results, yearly financial results. In the SaaS example, you know, you should really try to own that number of how many trials are we able to get going? How many of those are we able to convert? Because if you’re only looking at what you feel like you have direct kind of responsibility for of how well is this ad performing, then you’re not really doing marketing, you’re just running ads, but you need to be concerned about the totality of how well you’re able to drive those business results.

Jeff: And as a leader, I’d say two things. One is we need to be encouraging leadership to build in incentives that take us out of the, you know, the tactical side of things. So like how do, how do we work with our leadership to make sure that marketing or you as a leader are being rewarded when you take the long view?

So align incentives, and then how do we also do that as leaders looking down- down, I hate that word. You know, looking at our direct reports and try to make sure that they have a connection so that they’re actually being incentivized for, you know, things that matter and not just LinkedIn clicks, for example.

Kyle: Yeah, that’s a great point. And when we’re looking at these metrics and measurement, I think that incentive structure is a really important piece to keep in mind because that’s a lot of what it comes down to is that you want to have a way to be incentivized. You wanna have a way to incentivize your team, and it’s your job as a leader to make sure that those incentives match what could actually produce results. That if you’re getting incentivized for how many likes you have on your social media pages, then you are not actually driving results for the organization. And it’s your job to change that narrative and make sure that your incentives are aligned with what actually matters.

Jeff: Preach, preach. I mean, a lot of this reminds me, I mean it seems like in order to do that you’d have to have a really deep understanding of the customer and you know, I know upMarketer has the principle of own the customer, and it really feels like that would be key to this.

Kyle: Yeah, I think that. There are two things that you really need to do to be able to take that ownership over the bigger metrics and that, number one, I think the biggest thing that any marketer could do to improve at their job is to actually think deeply about how they themselves make buying decisions.

I’ve heard marketers before say that they can’t be influenced by marketing, and that’s just, so wild to me because it’s like, why would you be in this job if you don’t think that marketing can have an influence? Like it definitely can, but if you don’t believe that, then you are not able to do your job.

If you don’t think deeply day-to-day about what drove the decisions that you make, what influenced that? Then you are really just kind of doing malpractice as a marketer. And then you need to take that knowledge from yourself and start doing that research to know what drives your customers, because different people are going to come to those buying decisions in very different ways. They have different things that influence them based on the different challenges that they face in their life. So it’s really important to have a very complete and deep understanding of what it is that drives your customer to make that sale so that you can start to look at the ways that you can influence that outside of the measurable tools that you have at hand.

Jeff: And also explain to others, educate others on the reality of it, the subtleties of it.

Kyle: The thing that I would add to it is that it’s not that data can’t have a part to play in marketing because it can be very important to measure, you know, smaller, measurable systems of what you’re doing. Oftentimes, that’s still on a longer term. We’re talking months to six months, but being able to take, okay, so this is how our customers buy. These are the things that we can influence. Then you can start to get things into relative space where you are making apples to apples comparisons to understand, ”Okay, I know this is what drives my customers. I know that this is the channel that my customers spend their time on. ”So then you can start to measure, ”Does this copy work better than this copy?” But you are doing that with that relativity in mind that it’s not necessarily just about, the conversion rate of the ads, it might be much more about engagement or something like that, that you can see ”I know who my customer is and I know that I’m tracking them well, so does this speak to them in the way that it needs to for it to be successful?”

Jeff: The bottom line for me is I would hate for anybody to go away from listening to this and thinking we’re anti data or attribution.

Like, I don’t feel that way at all. I’m very pro it, we use it on an everyday basis, but to me, it’s sort of like when I’m driving in rush hour traffic, it’s very nice to have my speedometer to kind of pay attention to how fast I’m going. But the reality is I’m looking at the full context of the situation.

You know, I don’t drive 70 miles an hour just because that’s the speed limit, regardless of who else is on the road or the weather conditions or anything else, that is just a data point that helps me use my brain to drive safely and effectively. Right. And I think that’s, I hope, I hope that message that, you know, we’re just talking about the limitations of it , you know, and why it’s hard, not that it isn’t useful.

Kyle: And I’d also say that you can’t let it drive you. That if you’re checking your metrics daily and trying to figure out, you know, what, can I go in and change today? If I change that word or AB test this thing to try to get that number up, then you have a much bigger strategy and overall confidence problem in your marketing if you know that what you’re doing should work. Then you can look at your metrics on much longer timelines, you know, monthly, what have you, rather than getting into them every day and worrying.

Jeff: Or use them to get more different perspectives on it. So to understand things like seasonality and, and you know, how different segments perform and behave with different messages and these sorts of things versus some magical holistic measurement that measures. Whether our customers want our stuff or interested in our stuff or not.

Kyle: Yeah, and I think that that really sums it up. Metrics are never going to be that magic bullet that every marketer seems to be looking for. You can never look at data and know, oh, this is the proper way to market to my customers. You have to do much deeper work to get to that point.

Jeff: And I do feel like that marketers are becoming more and more aware of that fact of what you just said. I do feel like we’re, we’re having this conversation and I think we’re saying out loud what a lot of people are thinking.

Kyle: I hope so, cuz it will make their lives a lot better so here’s hoping. Thank you all for listening today. If you wanna get more content like this directly in your inbox, be sure to sign up for the unReactive newsletter at upMarketer.Io. And if you have any topics you’d like to hear discussed, please reach out to us on our website or social media. We’d love to hear from you.

Thanks for chatting today, Jeff.

Jeff: Thanks, Kyle. It’s fun.

Now Playing:
22. Why do we hunt for silver bullets?