How to build a dedicated marketing analytics stack

H

The design of your marketing stack is a make or break choice on which your millions of USD (or EUR) monthly marketing budget depends. You want to choose a stack that is lean, adaptable and, most importantly, reliable. The problem is that there are dozens of components to implement and thousands of tools and platforms available — What do I look out for when building a marketing analytics stack?

The choice might seem daunting at first. I have worked closely with many clients, top performance marketing teams, as well as BI and marketing analytics professionals and identified the 3 main challenges that you have to solve when building a modern, performant marketing analytics stack:

  • Implementing and maintaining 3rd party APIs
  • Working out data reconciliation
  • Providing marketing specific data visualization

The digital marketing landscape in 2020

The range of channels that CMOs and marketing managers have to implement today has increased exponentially. Performance marketing is getting more and more complex, new acquisition channels are added daily and you have to be ROI positive on most of them to justify the investment.

Here’s a “short” list of more than 50 digital channels that you have to manage. Each channel requires a specific strategy, it comes with its data and provides its connector.

The famous Martech 5000 infographic identifies more than 7000 tools and platforms that a marketing chef could implement.

Implementing an effective marketing plan means acquiring, converting, engaging and reactivating users. Each of these stages has its own data to be measured. The most important question is: where is data generated? How can I make sure that everything is consistent?

The building blocks of your marketing tech stack

Out of all acquisition channels, only a handful (e.g. email or direct chat) live on your own virtual turf. You will look for new users on 3rd party network, segment them mixing data from your CRM platform and/or your database, rely on attribution from external vendors, etc.

To keep it very simple you need to collect event data (interaction, spend, etc) from your advertising channels (Google, Facebook, Twitter, etc) and fire back conversions to allow automatic optimization. 

You also need an attribution tool to determine who’s responsible for each conversion among all channels (ad networks, email) and platforms (web and/or mobile). Lastly, you need to segment your users to either reactivate them on your own (email and notification) or external channels, for example in a CRM.

Marketing data needs special care considering that most of the data points and events are generated outside of your domains. 3rd party data is tricky and needs contextual transformations to make sense in your marketing intelligence platform.

Implementing and maintaining 3rd party APIs

The biggest part of your marketing effort will be around acquiring new users. Logically the majority of the data is on 3rd party servers, where ads are shown and clicks/taps redirect to your website/app.

To get the most out of an advertising channel you will have to integrate a proprietary conversion tracking (script or SDK) to allow independent placement optimization on each channel. This can be done with a platform that collects and routes your user data to other servers (tag manager or DMP) or by implementing each script manually.

To build a proper marketing intelligence infrastructure you will also want to extract spend and engagement data from your marketing channels and attribute it to your own proprietary data. The first option is to develop and host an in-house data ingestion infrastructure that funnels the data to a central data warehouse. This will require you to maintain APIs for most of the advertising channels on which you spend budget. And because each API is handled by a different company, you’ve got to keep on top and make sure to fix your integration whenever the third-party provider changes their system. This is time and cost expensive and will require you to hire data engineers or borrow (gasp) backend engineers from your core developer team. Additional risks are that some companies will simply remove functionality from their APIs without giving any prior notice or, in the best case, giving very short notice to the API consumers. Because this information is usually buried in the API documentation, it becomes hard to keep track of it and to act accordingly.

Alternatively, you can rely on several tools on different platforms. Here’s an example structure built with Google scripts and Google Analytics to fetch data from Facebook. Keep in mind that Facebook is undeniably one of the most popular ad channels out there, so expect the integrations with other platforms to be much more complex and less documented, requiring you to recourse to hacks and proprietary solutions. 

Most smart marketing experts will agree that maintaining API and data ingestion infrastructure deters from their main focus: acquiring new users and increasing revenues. The last and probably most efficient option is to implement a data connector (or data management platform) which takes care of all 3rd party APIs and data ingestion and can pipe data from your marketing channels to your warehouse or marketing intelligence platform.

Working out event mapping and data reconciliation

After getting ad channels’ data into your database you’ve only completed a third of your marketing intelligence journey. Spend, click and engagement data-points are meaningless if you can’t trace them back to users’ actions on your website or app.

And while measuring the ROI (return on investment) of your ad campaigns is your final goal, it might be surprisingly hard to answer this question with absolute precision.

Running on different channels means disparate data structures and level (impression/click based vs user based), also having web and mobile means different attribution methods. And that’s only the start!

New user acquisition data tracking funnel

After you’ve ingested the data from many different ad channels and tracking partners, mapping this to your own tracking and business logic can be very tricky, but there are a few practices that can help you minimize the hurdle.

First of all: most tracking tools (UTM params or mobile tracking) as well as Ad channels allow between 3 to 5 maximum parameters to be attached to each click/tap event. Remember that although most tracking and naming schemes allow each parameter to be independent of the other, for example the google analytics standard UTMs, others, including most mobile tracking platforms, work like russian dolls meaning an “adgroup” belongs in a “campaign” parameter.

Let’s take a look at some well documented naming practices:

Google Analytics UTMMobile tracking (example)SEM (Google Ads search)Social media (Facebook ads)
SourceNetwork
Medium
AccountAccount
CampaignCampaignCampaignCampaign
AdgroupAd groupAdvert Set
ContentCreativeAdAds
TermKeyword

Most advertising channels allow you complete freedom when it comes to naming campaign, ad group and ads. This could be a double edged sword, one of the most common mistakes that marketeers do is to name campaigns on the fly. When designing your marketing intelligence structure pay attention to creating a consistent naming convention for campaigns.

A good strategy would be to select the relevant product variables (Product, category, color) and targeting ones (Geo, gender, placement) and attribute them consistently to each one of tracking “boxes”, for example: Campaign = Geo|Product|Category, Ad group = Placement|Ad type, Ad = Ad name|Text.

After getting data into your warehouse you should be able to slice and dice your naming convention as much as you want and map it to your own events. Easy right? Unfortunately not quite… Solving this data join problem is arguably the biggest issue for marketers.

Data reconciliation is plagued by attribution bias as well as attribution model and data asymmetry:

  • advertising platforms will usually over-attribute to their sources;
  • mobile tracking solutions have rigid attribution models (typically last click);
  • your data is user centric (a conversion event is always tied to a user) while ad channels data is impression or click based.

Solving this marketing data reconciliation can indeed become a problem and require many hours of work of qualified data engineers. Your best solution is to follow a persistent campaign naming structure and, either spend proper time and resources in data warehousing and modeling, making sure that the data is clean and consistent, or adopt a specialized platform that allows you to easily map cross-channel and cross-device data to your own user-events providing the one source of truth for your marketing department.

Providing marketing specific data visualization

Picture this situation: you have 45 minutes to deep dive into the cause of a sudden drop in ROAS (return on ad spend) before a meeting. After the first 10 frustrating minutes fumbling with the company’s advanced BI platform to create an ad-hoc chart you decide to ask a data analyst for help. You figure out it’s not going to be ready before the end of the week and you panic. You shamefully download the whole data as csv and upload it on Excel. You sigh with relief as you CTRL+C & CTRL+V with confidence and prepare yourself to manually create pivot charts to smoke out the culprit of your performance drop. You’re still afraid you might have made mistakes, you will have to re-do the same thing manually next time, but you saved the day (or hour)!

If you identify with this scenario, you are not alone. Business intelligence platforms are powerful tools that can extract and transform data and visualize it in dozens of flexible graphs. Try though to “quickly” set up a data drill-down that was not created by your team of BI engineers and you might get disappointed, fast.

Product vs marketing analytics

Product analytics and marketing analytics are two different beasts. More often than not, they are served by the same BI platforms, and while BI teams are all about data modeling and pretty dashboards, marketing ones are more about flexible reporting and speed.

You should therefore consider speed and flexibility above features like variety of graphs, when evaluating your business intelligence platform. Marketing reporting tends to focus on a handful of consistent KPIs (CR, CPA, ROI/ROAS) over a shorter time span (1 to max 28 days) but has a huge variety of targeting options and levels of depth.

Manual vs auto-reporting

Company-wide reporting is great, but when you see a drop in numbers there’s no way to pinpoint the real cause as only the topline level of detail is ready for you to analyze.

Your marketers deserve the means to drill down in the data themselves and not ask the BI team.

It’s not uncommon, even for mid sized companies, to have 1 or 2 full time analysts pulling data from various sources and building complex and unreliable spreadsheets reports. These reports are often ingested via visualization tools, bringing along the risk of manual errors. With manual data extraction comes further compromises, as you can only dig down 1 or 2 levels at best before the data export gets too complex.

Marketing specific reporting

Lack of graphs is never the issue. No matter what tools you use, you won’t be lacking dashboards for each of them. Each tracking, ad network and ad agency will provide one. But marketers have grown tired of cookie cutter auto-generated dashboards as most are neither for the pros nor the un-sophisticated marketers.

As mentioned, you will be spending most of your time on tables rather than other types of charts (pie charts, dotted lines etc) and you should plan for a tool that allows you to showcase marketing specific charts like ranking of ad creatives by ROAS ROI including the banner themselves.

Smart data for smart(er) marketing

To recap: marketing data is a huge stream of distinct data sets; ingesting and making sure that the data structure is reliable is the foundation of your BI designing; mapping data from 3rd party platform to your own requires preparation and a well designed naming convention; data visualization and analysis, the most important part of your work, require marketing specific tools from the get go.

About the author

lelecanfora

lelecanfora

Get in touch