An Investigation of Machine Learning Techniques for Use in Training Agents for Military Simulations

Abstract

Agents assist users with performing tasks in computer-based applications. The current practice of building an agent involves a developer programming it for each task it must perform, but agents constructed in this manner are difficult to modify and cannot be trained by a user. Agent- Disciple is a system for training instructable agents through user-agent interaction. In Agent-Disciple a user trains an instructable agent through the interface of the user’s application by providing specific examples of tasks and their solutions, explanations of these solutions and supervises the agent as it performs new tasks. We report here on our work that uses Agent-Disciple to provide a learning agent that can command simulated military forces. Military simulations currently have many limitations in modeling human behavior. While it is relatively straightforward to build models of doctrine, it is difficult to have agents utilize this doctrine in varying contexts. There are too many factors to consider when building deterministic models of behavior, even in well-defined situations. We applied Agent-Disciple to circumvent this problem by using heuristic learning methods. A case study is presented in developing an instructable Company Commander Agent for the Modular Semi-Automated Forces (ModSAF) simulation – a state-of-the-art, real-time, distributed interactive military simulation currently utilized in large-scale training exercises. A ModSAF user can train the Company Commander Agent interactively, using the ModSAF interface, to perform various military missions using the Captain system based on Agent-Disciple. A training session with the agent illustrates the different types of learning interactions available in Agent-Disciple.