We'll keep you in the loop. We'll keep you ahead.


Return To Blog

Solid Data Acquisition Strategies Save Time and Money


I understand the value of the Ready, Fire, Aim approach to some projects. Sometimes action is the priority, and you’re sure you’ll learn and refine once you gain some experience. But when it comes to using data to gain actionable insights, I strongly advocate developing your data acquisition strategy before you invest too much in implementation and process.

Know Before You Go

I’m reminded of an experience I had kayaking a classic section of the Tuolumne River. While the river is a lot of fun, running the shuttle on the other hand, is long and arduous. Imagine the surprise and consternation of one group that came off the river only to realize they had left the key to their club security device back at the put-in. They went ahead in a desperate attempt to navigate the difficult access road using the available 15 degrees of steering wheel. That was funny to watch for a few moments, but ultimately everyone was unhappy with the extra time, effort and expense it caused. For strategic Big Data (BD) projects – just as in remote river trips – you’ve got to plan and double check requirements if you want to avoid unforeseen problems.

In my last post I started a thread discussing the promise and pitfalls of Big Data (BD)– that is, the increasingly popular practice of collecting and using massive amounts of marketing and operational data in order to achieve insights and optimize business. While it’s generally true that better decisions are borne from more information, there are a number of important considerations that should be taken into account before new data-oriented projects are started. Here we discuss some of the critical factors related to the generation and collection of data that can have profound impact on any intelligence program’s success. In particular, we will take a look at common challenges that frame business intelligence (BI) strategy using a data acquisition (DA) perspective.

A good DA strategy is born from clear analytic goals that readily inform data requirements. In the earliest planning stage, connect the information necessary to support specific insights with specific sources and types of data. However, the challenges of data collection are more complex than just instrumenting applications and processes with logs. When thinking about data requirements, take time to view goals through several different ‘utility’ lenses to identify obstacles and ensure you’ll get what you need. Let’s discuss a few aspects of data we think are important to consider at the planning stage.

Applicability – How will this data support our key objectives?

We cannot overstate value as a variable to be used in data acquisition decisions. Some folks get caught up in the mass of big data, focused on the traditional 3Vs (Volume, Variety, and Value) and move to create and fill data warehouses as quickly as possible. The 3Vs are valid bid data factors, but aggregate mass may mean less back on earth, where the weight of specific items is what we care about. Make sure you are always in touch with what important problem is solved by which data.

Use applicability requirements to organize your DA strategy. Consider creating a functional framework to more easily communicate the project requirements and enable others to contribute or to leverage the outcome.

Use applicability requirements to organize your DA strategy. Consider creating a functional framework to more easily communicate the project requirements and enable others to contribute or to leverage the outcome.

One way to begin framing DA strategy is to borrow from existing, standardized models of data-based decision support. For example, consider how data is used in frameworks for business intelligence (e.g. they way it informs decisions), business analytics (e.g. the testing of new models and the monitoring of solutions), and within data mining (e.g. to uncover previously obscure relationships, cycles and trends). In many cases data acquisition strategy directly flows from the future processes that use the information.

Scope – How well does this data cover our requirements?

Once you start looking, you may find endless data types and elements that are possible to collect. We recommend starting with the smallest, most critical set, and then constantly evaluating relevancy before adding additional items. In some projects, such as optimizing the customer journey, the over-arching needs can be.   

Source: Gary Angel 

Use your framework to maintain organization, and evaluate scope relative to each discrete goal, as well as the mission objective. Apply Test-case and Use-case scenarios to verify that target information is sufficient to solve real world business applications. Evaluate data source methods too – is it easier to filter the output from a firehose or to aggregate results from discrete sampling?  Are sources extensible and will they be future proof? Can processes be instrumented, or does DA require human record keeping to meet scope? Can scope be attained with sufficient quality? In many cases new tools will be required, either to support data acquisition directly with existing technologies and process, or to replace former practices with solutions that allow tracking.

Timeliness – Can this data be used for agile decisions?

It is very difficult to avoid problems, much less accelerate, when you navigate using only the rear-view mirror. Data that only reflects past performance, such as quarterly or yearly comparisons, is generally insufficient for tactical action. Focus on generating and using real time data in order to capture best value. By increasing pace, there may be a decrease in accuracy. Consider fast data but don’t overlook precision or detail requirements. And timeliness requirements should be matched to objectives—rapid data may be only useful in the short term, but if actionable, then rapid data may be sufficient. For example, imagine a caller who has viewed an ad with a tracking telephone number. Knowing the specific lead source may help in capturing and converting a lead. Knowing the personal and demographic details of that person may help even more. How fast can you gather this data?

Usefulness – Does the data have multiple applications?

Beyond sufficient scope, the utility of data types and elements is worth examination and most likely ought to impact priority. Utility requirements may also encompass ETL (Extraction/Transformation/Loading of normalized data) or relative data joins (conformity and uniformity to intersect data sets). Usefulness may also include the other side of timeliness, which is lifespan. Ask if the data has continuity of value as the company grows or changes.

Just as your framework can inform others and solicit input, expect additional data usage requirements to emerge or develop. Data items that have multiple modes of utility are likely more valuable to the organization as a whole. It is worth being mindful of the sub-optimization trap—that is, creating tools to optimize one part of an organization’s mission, while not considering the whole.  For example, perhaps you’d like to increase your customer base. Using performance data, you might be able create and deploy more and better online advertising. But if you lack the staff to effectively engage and mange a significantly increased rate of prospects, the objective might fail. Even worse, if the prospects feel they are treated poorly, they may review and rate your business publically, leading to business decline.

The sub-utilization— the inability or unwillingness to leverage valuable information of data—may be worse than sub-optimization. Sometimes it’s difficult to overcome internal barriers when there is a lack of common understanding or alignment on requirements. Recognize that it can be difficult to educate others on utility and necessity for certain DA requirements, and consider the impact carefully when global and limited scope DA choices are made.

Privacy – Would our customers be concerned that we have or use this data?

There are plenty of good reasons for businesses to collect and analyze customer data, but it pays to be sensitive to customer concerns around personal information. Given the recent high profile, bi-partisan inquiry  on privacy issues, there is little excuse for not understanding the FTC’s consumer privacy protection framework. Minimally you should endeavor to follow the core privacy principles:

  1.  Notice/Awareness
  2. Choice/Consent
  3. Access/Participation
  4.  Integrity/Security
  5.  Enforcement/Redresss

Success is an Imperative

Data-driven business intelligence projects are successful when they have sufficient preparation, adequate execution, and wide-spread adoption. Failure is not an option. Particularly for business leaders who are responsible for digital efforts, the lack of adequate tracking, analytics and optimization can be an organization and/or career killer. Organizational trust in data-driven decision support is key. As Steve Smith at MediaPost aptly put it, “Getting punked by crappy data or ROI claims based on iffy metrics is a sure way to kill trust in the digital unit.”

Let’s say you’ve heard from others that the trip down the data river is too big to manage, or that they didn’t get where they wanted. Or maybe worse, they got there and found themselves lacking a critical key. Don’t lose heart. The epic failures of others (or ourselves) should be lessons for the rest of us – in this case, don’t sacrifice functionality requirements for convenience. This holds true especially for data-based intelligence projects, since you can’t analyze data you don’t have.

Next posts: thoughts on the challenges of processing, then interpreting, and finally acting on data.