The Application of Cognitive Diagnostic Approaches Via Neural Network Analysis of Serious Educational Games
Date
2013-08
Authors
Lamb, Richard L.
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Serious Educational Games (SEGs) have been a topic of increased popularity within the educational realm since the early millennia. SEGs are generalized form of Serious Games to mean games for purposes other than entertainment but, that also specifically include training, educational purpose and pedagogy within their design. This rise in popularity (for SEGs) has occurred at a time when school systems have increased the type, number, and presentations of student achievement tests for decision-making purposes. These tests often task the form of end of course (year) tests and periodic benchmark testing. As the use of these tests, has increased policymakers have suggested their use as a measure for teacher accountability. The change in testing resulted from a push by school districts and policy makers at various component levels for a data-driven decision-making (D3M) approach. With the data-driven decision making approaches by school districts, there has been an increased focus on the measurement and assessment of student content knowledge with little focus on the contributing factors and cognitive attributes within learning that cross multiple-content areas. One-way to increase the focus on these aspects of learning (factors and attributes) that are additional to content learning is through assessments based in cognitive diagnostics. Cognitive diagnostics are a family of methodological approaches in which tasks tie to specific cognitive attributes for analytical purposes. This study explores data derived from computer data logging (n=158,000) in an observational design, using traditional statistical techniques such as clustering (exploratory and confirmatory), item response theory and through data mining techniques such as artificial neural network analysis. From these analyses, a model of student learning emerges illustrating student thinking and learning while engaged in SEG Design. This study seeks to use cognitive diagnostic type approaches to measure student learning while designing science task based SEGs. In addition, the study suggests that it may be possible to use SEGs to provide a means to administer cognitive diagnostic based assessments in real time. Results of this study suggest the confirmation of four families (factors) of traits illustrating a simple factor loading structure. Item response theory (IRT) results illustrate a 2-parameter logistic model (2PLM) fit allowing for parameterization using the IRT-True Score Method (X2=1.70, df=1, p=0.19). Finally, fit statistics for the artificial neural network suggest the developed model adequately fits the current data set and provides a means to explore cognitive attributes and their effect on task outcomes. This study has developed a justification for combining and developing two distinct areas of research related to student learning. The first is the use of cognitive diagnostic approaches to assess student learning as it relates to the cognitive attributes used during science processing. The second area is an examination and modeling of the relationship between attributes as propagated in an artificial neural network. Results of the study provide for an ANN model of student cognition while designing science based SEGs (r2=0.73, RMSE= 0.21) at a convergence of 1000 training iterations. The literature presented in this dissertation work integrates work from multiple field areas. Fields represented in this work range from science education, educational psychology, measurement, and computational psychology.
Description
Keywords
Education, Science education, Educational tests & measurements, Cognitive Diagnostics, Educational Informatics and Analytics, Educational Measurement, Neural Network Analysis, Science Education, Serious Educational Games