The 3491 foot tall Mt. Greylock is the highest elevation in Massachusetts, the centerpiece of a 20,000 acre state reserve and a rest point on the Appalachian Trail. That is where yours truly was “Ranger Danger” for the summer as the night patrol working solo. The job was simple enough, help anyone in trouble and keep the partying kids off the mountain. The problem was that these were all my friends and if I was not working the night shift, I might have been one of those kids.
The measure of my success was simple. If the park superintendent saw beer/bottles and/or felt warm ashes the next morning, then there must have been unauthorized activity on the mountain the night before. Bottom line, I was measured by beer cans & ashes. If you really think that through, there are lots of ways to have outstanding job performance in this system and still have the park full of undesirable activity at night. Lanterns and recycling come to mind. It was a great way to spend the summer and the park superintendent often congratulated me for outstanding performance.
As leaders, we need to be absolutely sure the metrics we put in place align with the real results we want. We need to think through the unintended consequences of the metrics to ensure we are not creating an environment or activity different than our desired end state. The up side of using beer cans & ashes was the simplicity of the metric and the ease of data collection. The down side was it did not meet the other metric objectives.
Have you seen a good metric gone bad?
Friday, May 8, 2009
Beer Cans & Ashes
Subscribe to:
Post Comments (Atom)
Posts
22 comments:
John - well said.
Things that you can count don’t always count. Things that count can’t always be counted. - Einstein
The time-honored mantra for managers is to manage through metrics. But without knowing it, there's a balance sheet in play - on one side we empower someone to begin to play the game to win - on the other side we create a gamer instead of a creative genius.
The famous japanese statesman said:
“We are going to win, and the industrial West is going to lose: There’s nothing much you can do about it because the reasons for your failure are within yourselves.”
--Konosuke Matsushita
Perhaps it is time to leave behind the "time honored and PROVEN" techniques we don't question and move toward purposeful leadership and let the creative human genius out of the box we built.
We would love to have your contributions on leadership on our new Aerospace & Defense Supply Chain group - http://www.linkedin.com/groups?gid=1877433
John,
This is a great post, thank you.
Being in the software development industry I see a wide variety of metrics used to gauge both billing practices and project success. It never ceases to amaze me how leadership continues to look at the total billable hours of a consultant or contractor in order to assess how productive they are.
Making matters worse they start to pile on meeting after meeting, time entry requirement after requirement in an effort to "gain better insight" into where the productivity holes are. All the while the very fact that they've spent 1/2 their day in a meeting room instead of getting real work done completely avoids their notice. Perhaps I've gone off on a little tangent but I think it's still relevant.
Looking forward to what this topic churns up .
Cheers,
Alex Kersha
John
Hilarious! I had a similar experience in college.
I attended a small conservative church affiliated school. the policy of the school at the time (consistent with the standards of the denomination) was that students could not drink - anywhere. the campus was very heavily Greek (10 sororities and 9 Fraternaties for a student body of about 1200), and it was possible that on occasion the rules on drinking might have been....momentarily forgotten or overlooked. One day the Dean of Students cam to the President of my house and suggested that there were "reports" of large quantities of empty beer cans in our trash on Monday morning and suggested that a pattern of such observations and reports might cause some in-depth inquiries. the standard was clear - no drinking. the measure was clear - no evidence. We we not investigated!
Bob
this is one of my favorite of your posts, John :).
What a great story. It's a very simple metric, and made a lot of sense if that was the ultimate goal.
As a diversity professional, I find it hard to measure the diversity of an organization. We can measure the race and gender of people in an organization, but that onlygives us one dimension. Somtimes we can look at age, but diversity is so much more than age, race, and gender.
In this case, the question we can ask is do people feel like they're included and have the same chances, regardless of these or other factors. But how to measure that is the tricky part!
John -
I have to agree with all posting here and appreciate the integration of humor, past lives and business common sense!
Who says we can't can't have fun while still delivering real business value!
We (CasMe) recently hosted a number of "Measurement" related RoundTable Meetings with Global 2000 SMEs participant and it was amazing some of the perspectives that we shared.
Benchmarking as a blend of "Art and Science" is something that we believe we're hitting upon.
I appreciate your thread here, and look forward to connecting further directly with you and any other interested parties.
Michael
My favorite Bad Metric is "lines of programming code". Personally, I feel that utilizing "lines of code" as a programmer's performance metric is like using weight as an airframe parametric, ala "That plane's over a hundred thousand pounds; now it'll fly twice as well as when it was a piddling fifty thou... Nice job, team."
Another Bad Metric is "man hours" of programming labor. I once had an application vendor brag to me that "there were OVER A THOUSAND MAN YEARS of development invested in their product." I looked them right in the face and muttered, "Ah, the Mythical Man Millenium..."
This is so true. I have seen many metrics miss their intended mark because someone took the easy way out when establishing them. Good metrics are worth the effort to set up correctly because they are an indicator of performance and an indentifier for areas of improvement. Anyone serious about performance and improvement has to be serious about metrics.
well, the title drew me in. Mt. Greylock peaked my curiosity even more (hiked there several times when I lived in Mass.and took my husband there when we went back to North Hampton, and toured the Berkshires).
So true, you have got to be sure that the carrot rewards the correct actions. We learn that the hard way in dog training. I am a firm believer that you train, give incentive and discourage people the same way you do dogs-- consistently and often, an make sure that you are rewarding the right behavior.
Great comments. I have enjoyed reading your posts John.
What bad baehavior have people seen metrics drive?
Good metric gone bad: first flight of 767 back in the 1980s, fuel filled should have been gallons, not litres. Plane almost crashed from lack of fuel.
Bravo! things that need to be said and said again....why are we always measuring people? Once I lost a job because I typed 55 wpm instead of 60...?
Nice posting John.
Take care and God bless,
Peter
John,
As in a former life I did work as and for park rangers, I felt the call of the wild and am responding but with a non-park example.
The measure I saw was a security guard who had been told his job was to examine each padlock on each facility bay to be sure the lock was securly closed by the prior shift. He dutifly looked at each one and the locks on his watch were closed. Trouble is no one ever told him to check to be sure that lock when through both the upper and lower fixture. Therefore all the locks were securly locked but some of the facilites were not.
Clare
"Have you seen a good metric gone bad?"
John:
Oh, you bet I have! I have worked for companies where the metrics appeared to have been good and well intentioned, but these being the numbers by which senior leaders' performances where evaluated, their focus became these specific numbers and not all those that would affect bottom line performance or customer satisfaction -- you can imagine the results!
I watched month after month as future month's sales were pulled into the present month so the "magic number" would be reached. With kind of metric, you can quickly see the "snow ball" effect that followed -- expedited freight for materials, higher inventory levels, early shipments to job sites not prepared to receive products, customer complaints, damaged goods that sat too long on a job site, frustrated suppliers, material expedite fees, and (one of the worse offenses) employees frustrated as they were told to go home on Tuesday and Wednesday only to be told on Friday that they would be required to work on Saturday - with the same pattern repeated the very next week!!! INSANITY!!!!
Needless to say, working as a senior manager in this environment was very frustrating and the only thing I could do as a supply chain leader in such an environment was prepare the supply chain for such activity through honest communication and building a bigger "shock absorber" to protect the supply chain from our vicious "yanks" on the chain.
The worst part, was the fact that other departmental metrics were not devised in such a manner as to allow for supporting the means by which the primary metric was achieved!!
Oh....it pains me just to remember those days! On the bright side, I learned very clearly what kind of questions to ask in an interview to discover how future employers would evaluate business performance.
What a stroll down memory lane......
Bjorn
John - It must have been a really fun summer! If you want to see good metrics gone bad, look at any call center. Their criteria for doing a good job is the number of calls taken and the "shortness" of those calls. Granted some centers actually track customer satisfaction, but the underlying criteria is to take as many calls as possible and keep them as short as possible regardless of the situation. If you have ever been "accidentally" hung up on or transferred to another rep, you may have been a victim of wanting to talk for too long.
Another example are recruiting (headhunter) offices where each recruiter is expected to be on the phone for a specified number of hours. Want to know how many recruiters know of a dead line that still rings so they can call it, put their phone on hold and take a lunch break while still getting credit for being on the phone?
Pam
This is a target rich subject. The first example that comes to mind is purchase price variance. I worked at a factory where we chronically missed customer delivery dates because material deliveries with high unfavorable ppv impact were delayed by a few days or a week so they would not impact the current accounting period.
At our company, we often see metrics that are incomplete, or that have to be explained to the people who are using them. Good metrics should speak for themselves.
What a great topic! And, yes.....
1) The CEO of a 1600 employee plant was such a heavy micromanager (and meddler in areas of which he knew nothing), he personally set up the workers comp. 'return to work' policy. The man was a chemist and knew literally nothing about benefits, etc. Anyway, the policy he established made it literally - and I do mean LITERALLY) impossible for healed and healthy employees to return to work. He couldn't understand why his worker comp. costs were so high. The HR Director, who really knew how to take chances, sent out the CEO's resume to hundreds of companies. One of them made him an offer he couldn't refuse.
2) The "help" department of a very large computer company set up rewards for help techs based solely on the number of calls they received. (Just like Pam's example) No accounting of the same client calling back repeatedly or of misinformation given to clients. So, the techs took calls and got off the phone with each call as soon as possible. Their customer satisfaction plummeted.
3) An order-fulfillment center was measured on how many orders they sent out; not if they were the right orders, etc. A lot of people got merchandise they never ordered and a lot never received what they did order.
Again, what a great topic.
Cynthia
John;
You have opened a topic that can go on for many lifetimes.
As a consulting leader for many years, one of my favorite misused metrics was billability. This more often than not lead to odd behavior and not always to a desirable result. Consultants and PMs tried to sell services not needed by the client. There is obviously a need to measure revenues in a consulting company but billability is not it. I have struggled with the right one, so I do not have the right answer.
In Supply Chain one measure that can be argued to be bad is forecast accuracy. You would not need forecasts if there was immediate response to a change in demand or if demand did not change. Obviously this is not possible in the physical world so forecasts are needed and accuracy needs to be measured. I have seen too many organizations hide behind this when they really should be working on reducing lead times, strategically positioning inventory and etc.
Do let me know any others you find interesting.
Cheers
Pari
John and Pari, love your metric speak, "billability", "beer cans & ashes". For some reason it triggered an old memory from a Seinfeld episode. Do you remember this metric?, "sponge-worthy" ;)
Best
-Doug
Consider the issue in reverse: why was it that kids partying on the mountain was a problem? Let's speculate that the real issue was not merely the fact that kids were partying at night, but rather the party consequences: bottles left lying around and the risk of wildfire from the bonfires. So, by recycling the bottles and using lanterns instead of bonfires, you eliminate the real issue and the fact that kids are partying is no longer a problem. Maybe the bottles and ashes weren't just empty metrics after all.
Post a Comment