Bill Gaw's Triple-Step, World Class, Lean Manufacturing Training Program
World Class Manufacturing

Increase the effectiveness of your
Lean Manufacturing Training Program
Lean Manufacturing Simulation Game 

Characteristics of Performance Measurement

There are three characteristics of performance measure­ment that are important to the understanding of a perfor­mance measurement system. These are precision vs. accu­racy, positional measurement vs. directional measure­ment and intended vs. unintended consequences of measurement. Each of these will be discussed in turn.

Many measurement systems focus on the precision of the measurement, often calculating measures to 2, 3 and even more decimal places. This is done to give the illusion of accuracy. However, a characteristic can be measured to 5 decimal places and still have an accuracy of +/- 20%. This type of precision adds no value to the measurement and can be misleading and costly to obtain.

More valuable is an understanding of what accuracy is and the degree of accuracy associated with any standard or actual measurement. We know from the study of statistical analysis that there are two types of measurement, at­tribute and variable. Attribute measures are of the binary variety—yes/no, good/no good, attained the objective/did not attain the objective. These measures are clearly defined but do not convey much information. They only indicate that performance fell within one range of values or the other, thus they are, by definition, approximations.

Variable measurement selects a value to the accuracy of the measurement tool available. In order to convey infor­mation, this value must be compared to a goal or standard to establish a measure of performance. Although this is a better measure than an attribute, it too has shortcomings. First, if the event measured is a one time occurrence, the standard is, by definition, a forecast of expected or desired behavior and differences between performance and goal can be attributable either to imperfect performance or imperfect goal setting. Second, if the event measured is one of a population or series of events, it exists in a probability distribution and, even when if differs from the standard, may be an acceptable performance if it belongs to the population from which the standard is derived. The ability to determine whether one event (or a small number of events) belongs to the population used to determine the standard and, thus, is acceptable performance is statisti­cally limited. Thus to use variables to measure individual events is statistically questionable.

Finally, the standard itself, if not a guess (projection or forecast) is only the mean (average) value of a probability distribution and to measure against the mean without also measuring the standard deviation limits the validity of the measure and eliminates access to the most important piece of information in many performance measures, the com­mon cause variation represented by the probability distri­bution of the event population being measured. Thus, many performance measures currently used are providing in­valid, inaccurate and inadequate information.

In addition, to the extent these measures focus on indi­vidual performance neglects the fact that much inadequate performance may be systemic, not individual. Deming tells us that 85% of quality problems are systemic. In the same way 85% of performance problems may also be systemic, hence they are management problems. Most performance measurement systems are using invalid information to look in the wrong place for solutions to problems.

The second characteristic of performance measurement is the distinction between positional measurement and direc­tional measurement. Positional or static measures locate the position of an attribute or variable at a specific point in time or over a defined period of time. These measures are, by definition, a single observation or an average of observations taken over the defined time period. As such they have all the statistical shortcomings mentioned above, plus they provide only limited information about the processes operating within an organization. For example, if the static measure of customer service for a time period is 95%, this measure provides limited information unless it is framed in a series of measurements which indicate whether the customer service trend is improving, staying the same or deteriorating.

To be Continued