« Strategy and the ROI in Change | Main | Harvesting Tacit Knowledge »

June 19, 2005

Evaluation versus Value

Evaluation closes the loop of management activity that begins with idea definition and continues through design, implementation, operation and support.

Following and actually overlapping "support", "evaluation" determines when the effects of operation are complying with initial standards and objectives, and whether the effects are compatible with requirements that have emerged and become newly current during the elapsed time of the idea's actualization and employment.

For managers, that current compliance and future compatibility mean something primarily because of the initial justification for the idea being actualized. This justification is often simply called the "baseline", and one of the primary goals of evaluation is to generate information that supports decisions about how to respond to changes from the baseline. As a result, performance information feeds change management to protect value.

Normally, the justification is a description of "value" to be gained by realizing the idea. Compliance will mean that the terms of the original justification are being met, but compatibility to new requirements means that expectations about future value are still reasonably positive.

An evaluation that cannot include both perspectives -- that is, of original and current expectations -- is at least incomplete, if not downright suspect. For example:
- an evaluation based only on the (earlier) original expectations might describe work done properly to produce something whose intended impact is no longer possible or relevant by the time it is actually delivered.
- an evaluation based only on current expectations, ignoring the previous forethought behind how the current conditions developed, might mistake a presently mismatched organizational stance towards requirements for a lack of capabilities instead of a lack of direction.

Another problem in evaluations is confusion about what approach yields information of the right type of importance. For example, these three following approaches are distinctive enough in purpose and capability; they can and should be complementary but are unlikely to successfully do each other's job:

Assessment - a determination of what kind of value should be associated with something, not just how much something is worth. In determining what kind of value, the assessment process examines how something actually creates value, and/or how it fails to do so.

Measurement - a determination of the degree to which something has a given quality or property.

Analysis - a determination of something's constituent parts and especially their inter-relationships, explaining a particular characteristic of something.

As an example of why these must be correctly used, look at the common need that organizations have to conduct performance assessments, performance measurements, and performance analyses. Casual conversation might use these three terms interchangeably, but in actual practice they will not correctly provide the necessary decision support if misused.

A performance assessment should investigate, and report on, the likelihood that an operation can meet the demands of given types of requirements. Along with this, but regardless of what current requirements are the established ones of record, a performance assessment should expose what types of requirements the operation is most likely constitutionally fit to meet, based on knowledge about how such requirement types have been successfully met in the past. This is usually where notions such as best practices are referenced.

A performance measurement should investigate and report on what level of compliance the operation has achieved regarding specified target levels of impact for certain types of impact. Typically, benchmarks are referenced in measurements instead of elsewhere.

A performance analysis should investigate and report on why measurable performance levels have occurred, regardless of what the measured levels are; then based on the reasons why, the analysis should predict the levels and types of performance that are most likely from the same dynamics. Forecasting is usually expected here.

However, the three approaches are complementary:
- Assessments and Measurements can share categories
- Measurements and Analyses can share data
- Analyses and Assessments can share models in common.

Taken together, the three approaches provide a complete reference for describing the relationship of operational effects to the objectives that for managers represent creation of value.

A full evaluation that includes assessment, measurement and analysis will give a description that traces the logic of what is actually happening in a way that accounts for expectations. In that way, managerially practical comparison of original expectations to current expectations should be possible, identifying the underlying conditions that are shaping or being shaped by operations.

Posted by Malcolm Ryder at June 19, 2005 7:41 AM

Trackback Pings

TrackBack URL for this entry:
http://www.malcolmryder.com/cgi-bin/mt-tb.cgi/67

Comments

Post a comment