User Satisfaction ?

[Gary Weimer 253-7796]

In article <1991May22.145925.10470 at eagle.lerc.nasa.gov> you write:
|> This was posted recently under comp.admin.policy ... I received some
|> helpful replies, but am looking for more input: 
|> 
|> Our upper management, which is not very computer literate, wants to
|> receive quarterly reports (oral: 15 minutes!) which are "METRICs" of
|> customer satisfaction.  Our customers are hundreds of researchers and
|> engineers, as well as secretaries and office staff.
|> 
|> Do any of you have *metrics* for user satisfaction?  
|> 
|> Somehow, telling upper managers about the mean time between failures
|> or the integrity of disk data doesn't do it.  Neither does surveying
|> a random sampling of users with insipid questions like "On a scale
|> of 1 to 10 ...".
|> 
|> We do have a Help Desk which tracks telephone requests for help ...
|> but focusing the metrics on problem reports could be negative, and we
|> want to be positive.
|> 
|> E-mail to uudot at venus.lerc.nasa.gov will be appreciated ... or post here.

Some of the metrics that management seems to like to see:

1) Number of problems handled. Shows (in managements view):
   a) how busy are you (how valuable other people perceive you to be)
   b) how much expertice you have (if you're answering questions)
   c) how reliable system is (if you're fixing problems)

2) Time to respond to a reqest. Shows:
   a) how organized you are
   b) how much you are overloaded

3) Time to resolve a request. Shows:
   a) complexity of problems (or your incompitence at resolving them 1/2(-:)
   b) how much you are overloaded

Note that each metric really only implies those things listed under
them, they don't really prove them. However, I've noticed that people
looking at metrics tend to read more into them than is really there.

My favority example of metric/statistic interpretation:
Students who get A's and B's tend to sit in the first 3 rows of the
classroom. Therefor, moving to the first 3 rows will improve your
grades. Reality: students getting A's and B's are trying to pay
attention, so they sit up front where there are fewer distractions.
Forcing yourselft to sit up front will not increase your desire to pay
attention. To test this hypothesis, make everyone sit in the first three
rows. I'll bet the grades don't change.

weimer at ssd.kodak.com ( Gary Weimer )