About 30 years ago, I worked on Capitol Hill, handling health and environment legislation for (now) Senator Barbara Mikulski. During that time, I did what I had done all my professional career – research issues by reading studies and analyzing data. As an evaluator, I often go behind the data to see if they “made sense” from both theoretical and (statistical) sampling standpoints.
One of the things that has always interested me are indicators – numbers that are published because they guide decision-making, Leading indicators point to potential future trends; trailing indicators tell us how we’ve been doing. For instance, unemployment rates often are indicators of the strength of our economy.
During the most recent recession, we often heard about the official unemployment rate being over 8%, and the youth unemployment rate (ages 18-25) being twice that amount. (As you know, it was the second number that led me to launch MentorOurKids.org, but that’s another story). We also heard that the “real” unemployment rates were higher.
Recently, someone asked me why that comment was made. In 1981, President Reagan became president, inheriting not just American hostages in Iran but also a weak economy with high unemployment and high interest rates. One of the initial things his administration did was change the change the definition of “unemployment” from people looking for jobs to people looking for jobs under six months”. The assumption in the change was that people looking over six months “weren’t seriously looking”. Surprise, surprise – after the definition changed, the US unemployment rate dropped. So people who state that “real” unemployment rates are higher, are people who understand that serious people might be looking for jobs unsuccessfully for longer than six months, especially in today’s economy.
At the time, it occurred to me that there are probably lots of “inappropriate indicators” that people rely on – and when viewed in a vacuum can mislead you to make poor decisions. Think about a manager’s dashboard. Company A reports double digit sales increase. Sounds good? But profits are down by double digits, Sounds bad? Which is it? Knowing that sales reflect unwanted, excess inventory and the company plans to focus on new areas of growth with higher profit potential, would suggest that long-term prospects are good.
The world is full of indicators – and it’s our job to challenge the assumptions to determine which are valid and which are not. A recent article on SAT scores being used as major indicators of potential success by some colleges, reminds me of the time “classic” IQ scores were considered quality indicators of success; today we know that there are many types of IQ (e.g., emotional, social, intellectual, etc.). As someone once concluded, an IQ score is the result of taking that IQ test – and may not indicate much more.
Someone at IBM once said “don’t confuse activity with results” The sales person who calls twice as many people as his colleagues, may also be producing half of their sales, if the quality of the calls and the follow through of other sales activities aren’t as good as those of the other people. It’s critical that we measure the right indicators and collect data on all appropriate ones, rather than make judgments based on one or two in isolation.
What are some of the common inappropriate indicators that you know mislead people? How to you ensure that you’re collecting and evaluating all relevant data so you can make accurate decisions and produce superior results? Share your pet peeves and concerns, and in a future blog, I’ll share them with you!