« The Enterprise Logic of IT Investment | Main | Change How You Change »

February 3, 2006

Decide, Design, Deploy -- Then Assess

More than any other role in the enterprise, managers are the focal point for accountability of execution results. On the one hand, this means that the organization's overall visibility of its progress is created by managers. On the other hand, it means that poor visibility is attributable to the way managers view things. It therefore becomes crucial for management and the organization's other stakeholders to have a given view that they both understand the same way.

Evaluations and assessments strive to create that shared view. But while most organizations mandate evaluations, many use them without any strategic impact on the direction of change or the continuity of improvement. Confusion is easy when the effort is to describe comprehensive responsibility for consequences. Managers sit at the center of this disappointment, but they need not. The solution is to be sure that all parties know beforehand what it is that the evaluation can really explain. For managers, evaluations should explain not just a nebulous idea of "results", but a specific idea -- progress. And for all parties, it is necessary to understand that evaluations and assesments are not the same; without this understanding it is far more likely that one or both of them will be ineffective or mis-used. Too much effort will be spent boiling an ocean or too much spent looking in the wrong place for answers.

I.

Three kinds of issues dominate visibility of progress.

One: "performance" must have the same definition for everyone. The generic definition would be "the degree of achievement towards a set goal, as generated by execution under demand." This is the context within which "progress" makes sense. The equation must be that the results of execution must amount to achievement, not just difference.

Two: a consistent understanding must be established of what is meant by "results". Here, the relationship between execution goals and impact goals is the subject. That is, since execution adds, moves, and/or changes things, one result is about whether those differences work out just as prescribed; but another result is that the influence they have works out as is needed. Progress is really the direct measure of whether the prescribed differences were obtained. The equation is that change equals requirements.

Three: the ability must be developed and used to accurately identify both what is "necessary" and what is "sufficient" -- and distinguish them! This subject is often complicated by confusion over where it is in the realm of execution that "needs" ought to hold top sway. This is not so much a debate about whether satisfying needs should be the driving purpose, but instead about who gets to declare the needs to which all production should align.

For those three interlocking reasons, the idea of evaluating performance is inherently a complex one. All organizations do a number of things that they call "performance evaluation" -- but the huge variety in how one outfit does them versus another is exactly why there is an industry for "best-practice" advice on the matter instead of only for precision techniques.

II.

Given the above, any organization could likely find significant differences in its view of achievement, progress and production -- once it knew to look for differences. This is one practice-level aspect of evaluations.

That said, two different organizations using the same practices should be able to understand things the same way, albeit this makes sense only to the extent that they both ought to -- and a given practice still allows wide differences in technique. So why bother with evaluating things on the practice level?

Answer: a "practice" is not mainly about "the right way to do something". Instead, it is really about associating ways of doing things with value. In this case, the point is to get value from the evaluation effort, instead of just going through the motions of supervisory responsibility.

Typically, organizations have a lot more familiarity and experience with the technical aspects of evaluation -- while not necessarily knowing where their technique fits into general best practice. This might signal a situation where there is ambiguity about whether the evaluation is actually solving the right problem (or any problem at all.) What issue is the examination really expected to resolve, and why?

An important starting point in clearing that up is to apply not only the above distinctions about "results" but also -- and even more importantly -- the following ones that reflect the difference between the beneficiary's stakes and the provider's stakes.

III.

What do organizations want "good performance" to mean? Essentially, it should mean that the organization's target stakeholder agrees that the organization's activity has created value for the stakeholder. The stakeholder's sense of received value can be quite varied -- ranging from relief to opportunity, and from intangibles to tangibles. From the provider's viewpoint, this extrinsic value is a goal of the provider's intrinsic performance.

Unfortunately, it is very common for external measures of an organization to be called "performance measures" -- but most often what is really being measured there is the extrinsic value of the organization's effort. Properly recognized intrinsic performance, which is an internal goal of the organization, is essentially a characteristic that the organization wants to acquire.

To be specific, the desired characteristic is the ability to produce an extrinsically valuable achievement. To get that characteristic, the organization has to get internal progress from its execution.

That gets us back to the issue of accountability and progress. The general question that performance evaluations try to answer is, "Did the way we do things get us the results we wanted?" If the answer is yes, management should be able to take the credit. But if the answer is no, then management could likewise take the blame.

In the credit and blame game, it's easy to react to a broad range of direct and indirect consequences, and lose focus on what should be specifically discussed about management -- namely, progress. Evaluations that seek to show comprehensive "responsibility for consequences" are too often boiling an ocean or looking too much in the wrong places for answers. Avoiding those problems is easier when we stay focused on accountability for progress.

IV.

How do we describe "the way we do things" and begin to account for progress? Most management activity can account for eventual progress in the way the management activity affected things at three different phases:
- decisions;
- designs; and,
- deployment.

In "making" progress, management has a short list of essentials to worry about -- all fitting under the umbrella concern of "can we do well what we are required to do?" Requirements generally come from:
- the dynamics of prevailing circumstances that are creating opportunity or risk,
- from formal restraints or orders, and
- from literal limitations of resources.

Correspondingly, the essential elements of execution that are to be managed are:
- Awareness (appropriateness of choice of response)
- Competency (effectiveness of response versus demand)
- Discipline (behavior or application consistency versus rules)
- Capability (the inherent functionality of a resource)

Managers "make" progress by cultivating and coordinating those elements -- through decisions, designs and deployments -- in ways that are specifically compatible with the terms on which activity is approved, by the organization, for responding to designated circumstances. As a final concern of management, the approval of the actual activity may come before the action or afterwards. While the terms of approval may be constant, the action may be prescribed beforehand, or justified afterwards -- reflecting the existence of some range for management discretion. Use of that discretion may become a particular point of evaluation -- one that questions the skill of the manager or the logic of the terms of approval, or both.

Taking progress as the subject, an evaluation is not really a "performance" evaluation except in terms of the performance of the manager who is responsible because of his or her presumed capability to manage.

Instead, regarding the measured execution results, the evaluation is a progress evaluation and what that evaluation is trying to explain is how the actual execution relates to the actual differences made towards meeting requirements.

V.

One of the profound realizations from that clarification is that execution always points in the direction of requirements. Wherever the requirements go, execution is responsible for meeting them. In real life, this means that the "fallout" of proper execution can easily feature enormous amounts of change, cost and discontinuity, because it was unavoidable as a consequence of the requirements. Therefore, if an evaluation of execution seeks to judge it by how much it was able to rein in change, cost, and discontinuity, then the evaluation itself is not measuring progress -- thus it is not measuring the intrinsic worth (the logical benefit) of execution. In such a case it is easy to see that execution might get modified as a reaction to factors that have not actually been linked to enabling the execution to better contribute to intrinsic performance. Anyone preparing or reviewing a business case will wrestle with this threat.

Clearly, however, factors such as change, cost and discontinuity are necessary to factor into the big picture -- the one shared by multiple stakeholders. After all, if execution is not sustainable, or if the side-effects of progress are corrosive to the organization's stability, then fundamental rethinking should be taking place. The difference between an evaluation and an assessment lies exactly here -- an assessment explains how the subject's results relate to the larger sphere of enterprise influence.

An evaluation can explain whether a job was really well done. But an assessment can explain what an evaluation cannot: namely, whether the job was the best "right" thing to do.

Posted by Malcolm Ryder at February 3, 2006 5:35 AM

Trackback Pings

TrackBack URL for this entry:
http://www.malcolmryder.com/cgi-bin/mt-tb.cgi/203

Comments

Post a comment