Programmatic Media Buying 101: What is a Data Lake?

A data lake is a centralized place, like a lake, that allows you to hold a lot of raw data in its native format, structured and unstructured, at any scale. You can store your data as-is, without having to first structure the data or define it until its needed.  It can then be used for creating reporting dashboards and visualizations, real-time analytics, and machine learning to guide better programmatic advertising decisions.

As data grows and diversifies, many marketing and especially digital strategy teams are finding that traditional methods of collecting data are becoming outdated and are pushing for something more centralized like a data lake.  According to Aberdeen research done in September 2017, the average company is seeing the volume of their data grow at a rate that exceeds 50% per year. Additionally, these companies are managing an average of 33 unique data sources, according to the research study. With data split into silos by team, like search, social or direct marketing, CMOs are being challenged with how to efficiently manage the analysis for their media campaigns.  If they don’t consolidate their data, they risk targeting the same consumer more than once or even exposing them to the wrong message.

Why Do You Need a Data Lake?

Most data platforms will only store data if it’s been formatted to fit a particular structure, like rows and columns.  So unstructured data like log files, data from click-streams, social media, and internet connected devices typically can’t be uploaded into a data platform until the data has been defined.  A Data Lake allows you to import all marketing data in real-time, from multiple sources and in its original format. It also allows you to scale data of any size.  Then you can figure out how to use it in an automatic yet personalized way to attract and retain customers through digital advertising.  Companies like Digilant can help you set up a Data Lake and use it for media activation.

What is the difference between a data lake and a Demand Management Platform (DMP)?

If you are a digital marketer, a Data Lake allows companies to collect PII data (Personally Identifiable Information), which DMPs do not.  A DMP’s is main function is the collection of cookie data for media audience activation where a Data Lake is often the first step used by data scientists to expand the knowledge of the DMP.  The DMP often connects directly to the media activation tool which for programmatic is most likely a DSP (Demand Side Platform).  A DMP will establish connections between several external data providers, and the data lake then supplements it with new internal data like social media feeds or connected device data.

Four Main Advantages to Having a Data Lake

1. DATA INDEXING
Data Lakes allow you to store relational data (a collection of data items organized as a set of formally-described tables from which data can be accessed or reassembled in many different ways without having to reorganize the database tables.) —operational databases (data collected in real-time), and data from line of business applications, and non-relational data like mobile apps, connected devices, and social media. They also give you the ability to understand what data is in the lake through crawling, cataloging, and indexing of data.

2. ANALYTICS

Data Lakes allow data scientists, data developers, and operations analysts to access data with their choice of analytic tools and frameworks. This also includes open source data frameworks such as Apache Hadoop, Presto, and Apache Spark, and commercial offerings from data warehouse and business intelligence vendors. Data Lakes allow you to run Analytics without the need to move your data from one system to another.
3. MACHINE LEARNING

Data Lakes will allow organizations to generate different types of marketing and operational insights including reporting on historical data, and doing machine learning where financial models are built to forecast likely outcomes, and suggest a range of actions, if taken, have the ability to achieve optimal results.

4. IMPROVED CUSTOMER INTERACTIONS

A Data Lake can combine customer data from a CRM platform with social media data analytics, as well as a marketing platform that includes buying history to empower the business to understand the most profitable audiences, the root of customer churn, and what promotions or rewards could increase loyalty.

In Summary

Marketers and Media Buyers would want to implement a data lake for three main reasons. First, they want to take advantage of more advanced and sophisticated analytical tools and dashboards, using a more complex and diverse foundation of information. Secondly, they also want to make traditional activities — like data access and speed of retrieval — more efficient and easier to accomplish. The third reason is they want to bring all the data from the different parts of the organization into one place creating efficiencies of time as well as cost savings.  While not every company succeeds at achieving all three objectives simultaneously, the most effective ones will able to see positive results on their ability to make better programmatic media buying decisions.

Programmatic Media Buying 101: Digital Marketing Data Sources, How to Apply Them?

The rise of digital advertising has given marketers fuel to reach their ultimate marketing goal: connect the actions of users into a meaningful whole. When marketers know a lot about their consumers, advertising is personally relevant and more effective. Personalization of brand experiences is a powerful way to drive conversion – and in digital advertising, personalization is driven by data.

Pre-digital direct marketers relied on research of public records of births, marriages, and property deeds. They bought mailing lists from catalog companies to reach out to potential customers. Audience segmentation and tracking purchase intent are not new strategies, but digital data collection has significantly increased their impact. And online consumers leave a digital data trail behind wherever they go.

The Digital Data Trail

Now that there is much more data available for marketers to use, behavioral targeting has become a much more popular tactic for media buying. Most digital advertising still relies on at least some information about individual attributes — like gender, geo-location, and age. But programmatic buyers and also have the ability to collect data related to online behaviors.

While a lot of privacy-conscious consumers delete cookies or use the “limit ad tracking” features on mobile devices, advertisers aren’t interested in personally sensitive information – like cheating on taxes or a romantic partner. They are more interested in behavioral data signals that indicates whether consumers are in the market for a car, house, or coffee maker.
Social media sites or other login-based services have access to personal identifiable information (PII), but tying it to other data points would drive away customers. The ongoing stream of “shares”, “tweets”, “likes” — or other “reactions” — is revealing enough info to be worth a lot to advertisers – even without names and emails addresses.

Online activity allows data providers to identify specific audience segments, These segments can be defined by variables like (1) geolocation, (2) device type, (3) marital status, (4) income level, (5) profession, (6) shopping habits, (7) travel plans and a variety of other factors. These valuable segments can be sold to the highest bidder wanting to reach segments as specific as men in trouble or burdened by debt: small-town singles.

There are many players in the data game – and some play multiple roles. 

  • Brands collect information about their customers, such as email addresses and purchase history, which is helpful when tailoring their experience through product recommendations or incentive offers. 
  • Publishers sell data about readers who visit their site, which is valuable to advertisers purchasing ad space on their site – but can also be used to track their site visitors in other places online.
  • Other data collectors who may sell data may not be thought of as “ad industry” types at all, such as:

1. Political campaign groups often rent out their lists to firms as a fundraising strategy

2. Credit-card companies issued directly or through banks sell anonymized data to advertising companies – in cases where cards are issued directly, customers can be cookie’d whenever they login then go elsewhere online.

3. Online auctions offer a huge amount of user info, again anonymized, but revealing.

4. Any company, site or service that requires a login can collect data. For example, social media sites garner huge amounts of data as users “like” “tweet” and “share” about their interests.

How Companies Collect & Use Data?

The data game players are defined by how they collect and use data – either as first-party or third-party.
First-party data is any information that’s collected by a company that has a direct relationship with the consumer. First party data tends to be considered as more valuable because it usually is more accurate and is free for the advertiser.

  • If we take the fictional example of Heart and Sole shoes, they would like collect data about customers’ style preferences and shoe size through the user’s’ purchase history.
  • If we look at a fictional online publisher, say Knitting News Weekly, they may associate an ID to anonymously track the articles that their visitors read and share
  • Another fictional example, like Gloria’s Games App might use an API to gather data themselves, for example, the most common geographic locations where the mobile app is used.

Some apps also use external SDKs — or a software development kits — from a technology provider that will track user data. In this scenario, the technology provider is a 3rd party data collector. And third party data is any data collected by platforms or services that do not have a direct relationship with the consumer. These service providers obtain data in many different ways, such as:

  • Paying a publisher to collect data about their visitors.
  • Or piecing together behavioral or interest profiles and audience segments to be sold to advertisers.

Let’s not forget the new comer to the data party – 2nd party data, which is essentially first-party data that you are getting directly from the source. Buyers and sellers can arrange a deal in which the first-source party offers specific data points, audiences, or hierarchies to the buyer. The sharing of high-quality, first-party data gives marketers access to many hard-to-find audiences they may not have been able to reach.
Whether first-party or third-party, data is often housed and managed for programmatic buyers by a Data Management Platform – or DMP. DMP’s give marketers — and sellers of data — a centralized way to:

  • Create target audiences, based on a combination of in-depth first-party and third-party audience data;
  • Accurately target these audiences across third-party ad networks and exchanges,
  • And measure performance with accuracy.

Summary

  • Marketing based on consumer data has always been around; digital has just made it more powerful.
  • Personally Identifiable data isn’t critical; marketers can get a lot of value from behavioral and interest-based data.
  • First party data is often considered most accurate and more valuable.
  • DMPs help marketers collect and manage and organize consumer data so that they can use the insights to activate a marketing or a programmatic media buying strategy.

Digilant Awarded Best Use of Big Data for Their Consumer Persona Product at The 2017 MITX Awards

Celebrating Data Science, Marketing and Programmatic Innovation with a win at the 21st Annual MITX Awards

BOSTON, May 30, 2017 /PRNewswire/ — Digilant has been selected as a winner in the category of Best Use of Big Data at the 2017 MITX Awards for their Consumer Persona Product.

The MITX Awards show is the largest and most prestigious annual awards competition in the country for technology and digital innovation. The categories in this year’s program showcase the disruptive marketing, creative, and emerging technologies happening in New England. Winners were announced in 24 categories at a sold-out event held on May 25th at Royale Boston.

MITX Award 2017 - Company Announcement EmailDigilant’s Consumer Persona is unique in the programmatic advertising industry in that it curates online data of consumers that are actually engaging and converting, uncovering REAL customers and creating unique data segments, based upon each advertiser’s individual data signals.

“At Digilant we’ve listened to our clients and appreciate that it is not enough to just develop complex programmatic algorithms,” said Adam Cahill, President of Digilant US. “In the digital advertising ecosystem, a large chunk of the value of the data being produced is lost because it is not translated into a format that clients can experience and use. We came up with a satisfying visual output that was a real representation of the intricate nature of our client’s data.  This output is focused on three of our advertisers’ main business objectives: Performance Lift, Reach (new customers), and the Discovery of New Audiences.  We are honored to have Consumer Persona recognized by the New England digital technology and marketing community.”

“This year’s entries exemplified the curious intellect that drives innovation in Boston,” said Amy Quigley, President of MITX.  “It was an exciting evening marking 21 years of creativity and industry leading ideas in technology. I was overwhelmed by the energy.”

For the complete list of winners and other information please visit the MITX Awards Website.

About Digilant

Unlock Data. Uncover Customers.
Digilant offers programmatic buying solutions and services designed for independent agencies and brands that are increasing their programmatic spending. Using data science to unlock ‘new’ automated buying strategies, Digilant enables brands to uncover proprietary and complex audience data that gives them the actionable intelligence they need to compete across every important media channel.
Digilant is an ispDigital Group Company. For more information, visit us at www.digilant.com or follow us on Twitter @Digilant_US.

About MITX

Inspire. Connect. Provoke.
For the restless companies that comprise the Massachusetts technology and innovation eco-system, MITX is the ultimate resource: inspiring members with progressive thinking, meaningful connections and provocative conversation. Celebrating 21 years of connecting tech and innovation professionals in New England, MITX is a dynamic community of more than 7,500 thought leaders and collaborators in search of insight, education and opportunity. MITX is headquartered in Boston, MA. For more information, visit mitx.org.

Like what you see? Join the 500+ clients that have partnered with Digilant.