AI-Enabled Classroom Tool For Visual Learning Analytics
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This work focuses on simulation, design, development, and evaluation of a visual Learning Analytics (LA) tool - Real-time Educational AI-powered Classroom Tool (REACT) - to support educators’ data-driven decision-making. The educational institutions face one of the biggest challenges, such as predicting student performance, detecting undesirable student behaviors, profiling, or grouping students, etc., due to the exponential growth of educational data. The educators play a crucial role, where one of their primary responsibilities is effective, high-quality teaching. To do so, they must stay updated with students’ responses, efforts, and outcomes, for providing timely feedback to promote students’ improvement. Additionally, some of these educators are also academic advisors who provide advice to students, which is a critical aspect of judging institutional effectiveness. Considering these challenges, a solution in terms of an Artificial Intelligence (AI) driven visual LA tool is proposed in this work. This work begins with a simulation approach for understanding the effects of a LA tool with alerts and recommendations on student performance. These simulations are performed by developing and testing an Agent-based Model (ABM) for the Department of Physics and Astronomy at a large public university. The positive results from this simulation study indicated that the alerts and recommendations might help to increase student performance. Further, to understand the importance of the tool's design and its features, a high-fidelity prototype of REACT is developed using Shiny framework in R. The design and development of this tool followed recommendations from the golden rules of interface design and the Gestalt principles from visualization literature. Furthermore, considering the involvement of humans in educational applications, model-agnostic explanations are included on REACT for bringing explainability and interpretability in the process of decision-making. Finally, a study was conducted to understand the effectiveness, experience, and usability of REACT. The participants were 33 educators from Science, Engineering, and Humanities & Social Sciences. This study was performed using a hybrid approach of think-aloud interviews and questionnaires for exploring educators’ perceptions. The study concludes that REACT was rated as highly usable by educators from the Science and Engineering domains who perceive their experience similarly. Their perception and experience in using this technology-focused tool differed from the educators of the Humanities & Social Sciences domain due to the technological knowledge gap in these fields, as exposed by the study's findings. The results also demonstrated that REACT has higher effectiveness and a higher likelihood of motivating behavior changes in educators from the Science and Engineering domains.
Description
Keywords
Artificial Intelligence, Correlation Analysis, Human-centered Evaluations, Learning Analytics, Usability, Visual LA tool