NDMA Home Page
Index of topics on this NDMA website
Search this NDMA website on Google
© 2024 N. Dean Meyer and Associates Inc.
Excerpt from www.NDMA.COM, © 2024 N. Dean Meyer and Associates Inc.

Executive Summary: Metrics

quick overview of how to design effective metrics

by N. Dean Meyer

Metrics gauge how people are doing.

There are three types of metrics:

  • Organizationwide

    Organizationwide metrics tell people how well the whole organization is performing. They can give people a sense of shared destiny, especially if there are rewards, like profit sharing and employee-equity growth, associated with those organizationwide performance indicators.

    They can be used to build concern for the state of the organization and motivate a desire for change (if they raise concerns), as well as celebrate successes (if they show progress).

    But they only motivate performance if people understand how they can directly affect enterprise success. That requires an investment in education about the business, and how each function contributes to its success.

    But be careful. Organizationwide metrics can cause a breakdown in internal customer focus. People might say, "I'm doing this for the good of the company; so I'm going to give you what I think the company needs, not what you want to buy." When that happens, internal customers are disempowered, teamwork breaks down, and the whole operating model comes unraveled.

    There's another unintended consequence. A tech company measured people on their contribution to profits and strategic goals. They couldn't get anybody to write documentation, because its bottom-line impact is too obscure.

    Finally, research tells us that when people are pursuing the same metric, human nature creates a pseudo-competition with peers. There's a tendency to want to be first, rather than be the best you can be. That can lead people to sabotage their peers. And when they think they're ahead of peers, they slack off.

    Organizationwide metrics may be useful as a basis for special incentives such as bonuses. However, it would be unfair to connect them to individuals' performance appraisals. That would hold people accountable for things they can't control. There are far better ways to align individuals with organizational goals. For example, if you're measured on your internal customers' satisfaction, alignment will ripple through the entire organization; pleasing one's customers is the same thing as making your contribution to enterprise initiatives.

  • Job-specific performance appraisals

    Job-specific metrics include, but are certainly not limited to, annual performance appraisals.

    Annual reviews are not the best way to impact performance. Imagine the oversimplified case of a machine operator who reads a dial and controls a knob. When the dial indicates that the machine is drifting out of tolerance, he adjusts the knob. Now imagine that the dial is moved to the supervisor's office, and the supervisor tells the operator how he did at the end of each year. What are the odds of a quality product? Zero!

    The primary reason performance appraisals are considered an important metric is their connection to rewards (e.g., bonuses, raises, and promotions).

    Three rules apply to the design of performance metrics:

    • Measure ends, not means.

      In the spirit of empowerment and managing people by results, performance appraisals should focus strictly on outcomes, not processes. Metrics of how well people follow processes are disempowering; they force people to do assigned tasks rather than apply their knowledge and creativity to attaining the intended results. They encourage people to "go through the motions," whether or not those motions accomplish anything. And they may create "double jeopardy" where people are graded poorly for their means, and then again for poor results.

    • Only measure things that are controllable.

      There are plenty of psychology studies that demonstrate the demotivational effects of rewarding or punishing people for things beyond their control. When there's only a loose connection between their behaviors and their metrics, ultimately people will give up trying to impact the metrics.

      Worse, rewards/punishments that are seeminly random and uncontrollable can lead to mental instability.

    • It's better to be comprehensive than precise.

      "Not everything that counts can be counted,
      and not everything that can be counted counts."
      Albert Einstein

      If you only measure a subset of what's expected of people, you're encouraging them to optimize that subset and ignore the rest of their duties.

      So, don't just measure short-term results. The long term is at least as important. For example, it's not okay to fail at innovation, or to upset internal customers, just to get one's projects done on time.

      Note that "objectives and key results" (OKRs) typically represent near-term deliverables. That's a small subset of what's expected of people. Furthermore, in an agile organization, OKRs can change dynamically throughout the year. A moving target is not an effective metric. And the effort involved in evaluating such a metric can be immense.

      And don't just measure things that are easy to measure, or easy to quantify. Metrics are not always quantitative. In practice, we often find that the more interesting the concept, the less tangible the metric. Subjective metrics are better than missing metrics.

    Beware of creating unintended consequences. When designing metrics, think through how people can game the system, and what undesirable things they can do to achieve that inappropriate metric.

    Examples of ill-conceived metrics....

    • Too frequent

      Continual feedback is a good thing. But performance metrics done too frequently, e.g., twice a year, cause people to resist taking on any longer projects.

    • Grade on a curve

      When one's evaluation depends on beating one's peers, there's a strong disincentive to cooperation, sharing, and teamwork. In the worst case, there's an incentive for sabotaging one's peers.

      Adding in incentives for cooperation don't help. People may just accuse peers of being uncooperative.

      Furthermore, it induces game playing. Managers keep low-performers until after the ratings period, so that they don't have to grade others down just to make the curve, and so that they can take the low-performers' bonus money and give it to the high-performers.

      Worse, managers are discouraged from building a team of all high-performers. And if they do, punishing the one who's not quite at the same level as the rest (but perhaps well beyond staff in other areas) is unfair and demotivational, and may lead to the loss of some great talent.

    In the business-within-a-business paradigm, managers are measured as entrepreneurs. Managers are evaluated on customer satisfaction because they have clearly defined internal customers. Evaluations can also include entrepreneurial metrics like cost control (using external benchmarks such as outsourcing), and economic value including the many types of innovation that keep each business-within-a-business viable in the future.

    Examples of metrics of internal entrepreneurs....

    • Customer satisfaction (internal and external; immediate customer,
      not your customer's customer)

    • Market share (external or internal)

    • Your group's profit/loss (break-even, not budget variances)

    • Competitive rates (market benchmarks of unit costs)

    • Profit targets for those who sell to external customers.

    • Product/service quality

    • Your reliability, integrity

    • Safety and compliance

    • Innovation

    • The viability and expected value of long-term business strategies

    • Compliance with, and contribution to, our organizational operating model

    • Subordinate employees' engagement and satisfaction (through engagement scores and surveys), turnover

    • Supplier relationships (especially internal suppliers)

    • Positive contribution to the community (including the environment)

  • Job-specific dashboards

    Metrics have the greatest impact on results when they're delivered to the people doing the work (not their supervisors), in time for them to adjust their behaviors.

    Direct and timely feedback loops, termed "in-process" metrics (as opposed to after-the-fact evaluations), are a tool staff can use to optimize their day-to-day work.

    While performance appraisals should be limited to outcomes, dashboards should include whatever people need to perform better, both processes and outcomes.

    Note that the dashboard on a race car gives the driver critical information needed to operate the machine effectively. But the driver isn't evaluated on what gear they're in. The only meaningful performance metric is whether or not they win the race.

    Just as the dashboard on a car is different than the dashboard on an airplane, job-specific dashboards should be tailored to the unique requirements of each job. They should include whatever people feel they need to do a better job.

Abstracts

Free library

Books

Speech abstracts

NDMA coaching/consulting services

UP....

NEXT PAGE....