An Assessment of the Use of a Structured, Subjective Method to Evaluate the Quality of Decisions in Complex, Ill-structured Problems
Date
2014-10-07
Authors
Powell, Walter A
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Improving the quality of decisions made by decision-makers is the ultimate goal of research into decision-making, and the ability to assess the quality of decisions is central to decision research. The ability to assess the quality of decisions is crucial to determining whether actual decision-making conforms to theories of decision making, whether decision-making tools affect real-world decision-making, and determining which series of complex decisions (plans) would be best to implement. Much research has been conducted into decision-making and decision quality, but most of this research concerns problems that are well-structured (the best answer is knowable) and that are of limited complexity. Decision research into this class of problems typically uses the desirability of outcomes or the rationality of the decision process as the basis for evaluating the quality of decisions. While these methods may be appropriate for well-structured problem, they do not seem appropriate for complex, ill-structured problems in which rational decision processes do not necessarily lead to a single best solution, the solution generated may not be implemented or may not lead to a single best outcome. Therefore another method, ideally a direct assessment of decision quality, is needed in order to evaluate decision-making in complex, ill-structured problems. Since complex, ill-structured problems are likely to have few objective measures and therefore require subjective judgments on the part of decision-makers, the evaluation of the quality of the decisions made to address these problems likewise needs to be primarily subjective. However, little research has been conducted into either subjectively assessing decision quality or directly assessing decision quality (without the use of a proxy) for complex, ill-structured problems. The research documented here evaluates the effectiveness of using a structured, subjective method for directly evaluating decision quality. The use of a structured subjective method was investigated in two cases studies in which real world military problems of different complexities were addressed using different decision-making processes. Together, the case studies demonstrated that a structured, subjective approach was effective in directly evaluating the quality of decisions and a that a structured, subject evaluation is robust in that it can be used to evaluate the decision quality of decisions for complex, ill-structured problems of varying complexity.
Description
Keywords
Decision making, Tailored Likert Scales, Decision quality, Complex ill-structured problems, Subjective evaluation