Abstract:
Improving the quality of decisions made by decision-makers is the ultimate goal
of research into decision-making, and the ability to assess the quality of decisions is
central to decision research. The ability to assess the quality of decisions is crucial to
determining whether actual decision-making conforms to theories of decision making,
whether decision-making tools affect real-world decision-making, and determining which
series of complex decisions (plans) would be best to implement. Much research has been
conducted into decision-making and decision quality, but most of this research concerns
problems that are well-structured (the best answer is knowable) and that are of limited
complexity. Decision research into this class of problems typically uses the desirability
of outcomes or the rationality of the decision process as the basis for evaluating the
quality of decisions. While these methods may be appropriate for well-structured
problem, they do not seem appropriate for complex, ill-structured problems in which rational decision processes do not necessarily lead to a single best solution, the solution
generated may not be implemented or may not lead to a single best outcome. Therefore
another method, ideally a direct assessment of decision quality, is needed in order to
evaluate decision-making in complex, ill-structured problems.
Since complex, ill-structured problems are likely to have few objective measures
and therefore require subjective judgments on the part of decision-makers, the evaluation
of the quality of the decisions made to address these problems likewise needs to be
primarily subjective. However, little research has been conducted into either subjectively
assessing decision quality or directly assessing decision quality (without the use of a
proxy) for complex, ill-structured problems. The research documented here evaluates the
effectiveness of using a structured, subjective method for directly evaluating decision
quality. The use of a structured subjective method was investigated in two cases studies
in which real world military problems of different complexities were addressed using
different decision-making processes. Together, the case studies demonstrated that a
structured, subjective approach was effective in directly evaluating the quality of
decisions and a that a structured, subject evaluation is robust in that it can be used to
evaluate the decision quality of decisions for complex, ill-structured problems of varying
complexity.
