This tutorial is part of our series exploring getting from information to understanding. (See Applying the Scientific Method to Business). Analysis is the second step in that process, i.e. answering the question: What do we do with all that data we’ve collected?. Analysis explores seven analytical processes that shed light on the meaning behind the data.
In addition to sections on Background and Methodology, you’ve probably seen research reports with sections on:
- Findings
- Conclusions
- Recommendations
What are these “Findings”? Raw data? “Refined” data? A summary of various statistical manipulations or database queries? An interpretive narrative?
Usually, all of the above…and then some.
There are numerous ways to take data apart, some useful, some not. If you recall from the previous step, Information, we said that “information by itself isn’t worth very much. It’s what you DO with it that counts.”
What do YOU intend to do with the information?
Academic interest aside, the objective of most commercial research is to uncover ways of improving a situation. So, your analysis should focus on the IMPLICATION of your research, or what you might think of as the “so what?” factor.
That “so what?” will drive the kinds and depth of analytical hoops you want to make the data jump through: to FIND relevant information AND derive valid CONCLUSIONS (understanding the information) which enable you to make good decisions (“Recommendations”)…at a reasonable cost and a quick enough pace to stay ahead of the game.
The “7 Keys” that follow are common analytical processes, accompanied by cautions about equally common pitfalls. This is hardly an exhaustive coverage of moving from information to understanding; rather, it represents basic techniques that most organizations use, and sometimes misunderstand. Keys 1 and 2 are fairly lengthy and detailed, but take heart. Keys 3-7 are mercifully brief. 😉
1. Slicing and Dicing (part one)
Characterizing an elephant from its truck alone
Beware the trap of micro-segmentation and generalizing to the whole marketplace or a particular niche from too small a sample size. (This is NOT the same as gathering information from individual customers for the purpose of one-to-one marketing!)
Here’s an example.
Years ago, a fellow consultant asked us to assist on a customer satisfaction assignment for an industrial client. The company had three types of commercial customer, sold multiple varieties and sizes of product through a network of reps with established geographic territories across the U.S.
Due to budgetary constraints, the survey was limited to 150 telephone contacts of current and former customers, and included both quantitative (numerical or “choice” answers) and qualitative (open-ended, such as “how can we better meet your needs?”). Altogether there were nearly 130 data points (each open-ended question counted as a single data point) for each survey questionnaire.
We did get some fine information and were able to develop a set of recommendations for change in some of the client’s rep compensation policies and a few product improvement suggestions. But, when it came to understanding how to better serve each of the three types of commercial customer, the results were “iffy” at best. Here’s why.
First, the 150 contacts were divided among eight geographic territories, each served by a different sales rep, so each territory had approximately 20 companies surveyed. Now most of these territories had all three types of customer industries, so about 7 companies of each type were surveyed per territory…and some were current, some were former customers.
Of course, not all of these customers bought all of the client’s products in question (9 in all, not counting size…from individual units to cases to pallet loads). Further, climate influenced how the products were used. Ultimately, most of the product type/customer type combinations were represented by fewer than five respondents. And this did not account for the differences in sales rep capability and territory composition (other issues entirely!). Hardly a case for imputing statistical significance, although we most assuredly derived valuable one-to-one information for the client.
The lesson here is, don’t segment your respondent groups too finely if you’re wanting to generalize to your overall market because you’re likely to place undue reliance on far too few observations.
2. Slicing and Dicing (part two)
Settling for gristle and missing the meat
You’ve no doubt noticed how most surveys are of the multiple-choice variety. The few surveys we’ve seen (other than our own) that venture using open-ended questions usually relegate a single question to the end, one that reads something like: “if there were one thing you could have different, what might that be?” This is fine and good, but most often wind up not being answered at all or, if answered, repeat one of the multiple-choice themes.
As we noted in Key 6 (Building your own information) of “10 Keys to Collecting Information” in the previous issue, a more effective way to elicit NEW information is to intersperse open-ended, “discussion” questions among the “choice” variety.
This, however, usually strikes real terror into the hearts of data analysts. “You can’t sort memo fields,” they will tell you. “You can’t quantify narrative,” they moan.
This is true.
However, with a bit of skill and a bit more of work, you CAN extract the treasures of information hidden among all the rambling words of your respondents. Let me tell you how we do this.
We import the questionnaire replies, memo (narrative) fields included, into a relational database such as Paradox or Access.(We’ll cover questionnaire design issues, such as question sequencing and avoiding leading questions in another issue). Sorting and querying of the multiple-choice questions is done in the normal fashion to obtain demographic information, seek out relationship patterns, get a statistical profile of the responses.
Working with narrative responses is certainly less routine than working with numerical or standard text selections. We begin by identifying and listing what we call “key phrases” contained in the narrative fields, often placing them in lookup tables.
This is NOT a job for a data entry clerk, especially one who does not know your business. You need to exercise a level of judgment in assigning key phrases to rambling bunches of words. Surveys will typically generate in the vicinity of 15-20 key phrases per question. Key phrases will often overlap in questions that are related.
Tech note: It’s important to set up “one-to- many” relationships between each new “key phrase” table and your new reference table derived from the original narrative responses. You need to be able to capture multiple key phrases that will be present in many open-ended responses.
Transfer these key phrases to new forms which will be used to generate reports of the key phrases used.
A cautionary note: We’ve observed that many comments made in response to open-ended questions are more appropriate to a different question than the one asked. For example, you may have asked “how can we improve?” and part of the response will refer to competitor strengths, another question you probably asked elsewhere.
You DID, didn’t you? 😉
For purposes of really understanding what your respondents are telling you and HOW MANY are telling you something, it’s best to place each key phrase in its proper question category.
Once you’ve got the key phrases identified, you will see general categories emerging. These may be issues such as quality problems, confusion over pricing, inadequate customer service or even your entire value proposition. Some of these will be critical success factors, some of less importance. By assigning the key phrases to the general categories, you will be able to quantify your market’s perception of your performance far more reliably than depending entirely on “choose one” types of questions, where human nature often gets in the way and people tend to respond without thinking very much or not wanting to give you a “low score” on a troublesome issue.
It is the set of Conclusions based on these and traditional quantitative Findings that set part of the stage for the Recommendations. Other Findings (briefly covered below in keys 3 and 4) set more of the stage.
Hard work? Yes, especially the first time around. But think of it as creating the information infrastructure which will support the tough management and leadership decisions you will need to make to meet the demands of the future… faster, cheaper, and better.
3. Slicing and Dicing (part three)
Relating the unrelated
Studies of grocery shopping habits have shown that there’s a higher-than-expected correlation between beer and diaper purchases. How so? Young families have not yet reached a “champagne” budget. In looking for relationships, slice and dice away and out of the confining boxes of beer/chips or diaper/baby food combinations.
4. Watch What They Do, Depend Less on What They Say
Then decide how you need to change
No matter what people say, their actions shout far louder than words. Does your survey indicate that they visit the library and church weekly but eat fast food only once a month? Check the “hard” evidence of cash register and other tallies against the local demographics and run some averages.
5. Perception may be Reality to the Perceiver…
…but not necessarily reality to the researcher
Ask people how many seconds they’ve waited for an elevator or in the checkout lane. More than 90% of the time their “inner clock” senses a lot more elapsed time than the watch on their wrist. So, when you’re making improvements, making a psychological “fix” may be cheaper, faster, better than a mechanical (or technological) one.
6. Inevitability, Universal Truths, and Common Sense
Don’t forget to take into account what I call inevitabilities. Demographic reality is one: for example, the aging population in developed countries. What implications does this “reality” have for healthcare, housing and transportation services and products?
Then there are universal truths. You know: the poor will always be among us; it will always be “the best of times, the worst of times”; and there will always be exciting new things to learn and mysteries to unravel. What needs will always have to be addressed?
Finally, common sense (or lack of it). For example: fiscal foolishness, such as bank lending practices or stock prices that ignore the laws of gravity: what goes up too fast must eventually come down to a rational level.
Consumer purchases just cannot be legislated or mandated, e.g. the resounding silence of the population expected to purchase electric vehicles when the technology is still at the stage that requires 8 hours for recharging, go all of 80 miles between charges, and cost $10 thousand more than conventional cars.
How many of you routinely buy recycled paper products for personal or business use (generally costing about 10% more than those made with virgin materials – until critical mass is achieved)?
Are your product or service offers at risk? Are there opportunities for you that others are missing?
7. Getting Understanding is the Journey, not the Destination
Curiosity didn’t kill the cat. Complacency did. Data collection and analysis should be on-going and a lot of it can be done in “near-real” time. You don’t need to have terabyte size data volumes. Small organizations can adopt many of the large-scale data warehousing and data mining models for PC-based systems.
Next issue we’ll discuss “A picture is worth a thousand words” … tools for helping others (and yourself) understand the meaning behind the words and numbers.
So far we’ve been exploring external, i.e. customer or market data. We’re planning to address acquiring and monitoring internal data in the near future.
Knowledge Branches:
- Management. Functions: Business Intelligence; Strategy Development
- Management. Sector: RELEVANT ACROSS SECTORS
Information gathering and its transformation into business intelligence is a vital component of success in the marketplace. In today’s climate of lean, downsized organizations, there are frequently too few internal resources available to do what it takes to truly understand what’s happening with your customers, prospects, competition, suppliers and advances in technology. If you’re looking for an external partner to assist you in these areas, to help you attain the goals you’ve set for your organization, we’d be pleased to hear from you.