Machine Learning and Inference Laboratory, College of Public Health
Permanent URI for this collection
The Machine Learning and Inference (MLI) Laboratory conducts fundamental and experimental research on the development of intelligent systems capable of advanced forms of learning, inference, and knowledge generation, and applies them to real-world problems.
Major research areas include:
- theory and computational models of learning and inference
- data mining and knowledge discovery
- machine learning and natural induction
- inductive databases and knowledge scouts
- behavior modeling and computer intrusion detection
- non-Darwinian evolutionary computation
- multistrategy learning and knowledge mining
- intelligent systems for education
- models of human plausible reasoning
- machine vision with learning capabilities
Browse
Browsing Machine Learning and Inference Laboratory, College of Public Health by Author "Cervone, Guido"
Now showing 1 - 9 of 9
Results Per Page
Sort Options
Item Adaptive Anchoring Discretization for Learnable Evolution Model: The ANCHOR Method(2001-05) Michalski, Ryszard S.; Cervone, GuidoTo apply a symbolic learning method to learning in a continuous representation space, the variables spanning the space need to be discretized. When the space is very large, a problem arises as to how to determine a discretization scheme for each variable that is both efficient and effective. This task is particularly important when applying Learnable Evolution Model to optimization problems with very large number of continuous variables. The presented method, called ANCHOR, starts with a low discretization precision of the variables, and then increases the discretization precision in the subranges indicated by the analysis of the descriptions learned using a lower precision. The method has been incorporated in the LEM2 system implementing the Learnable Evolution Model. Experiments with ANCHOR have demonstrated a significant advantage of the method over a fixed discretization method, and enabled LEM2 to optimize functions of large number of continuous variables very effectively.Item An Application of Symbolic Learning to Intrusion Detection: Preliminary Results from the LUS Methodology(2003-06) Kaufman, Kenneth A.; Cervone, Guido; Michalski, Ryszard S.This paper describes briefly a method for applying AQ symbolic learning to problems of computer user modeling and intrusion detection. The method, called LUS (Learning User Signatures), learns models of users’ interaction in the form of sets of rules in attributional calculus, and signals a possible intrusion when a user interaction with a computer violates the model. An important characteristic of LUS is that the generated user signatures are easy to interpret and understand. We describe briefly the LUS method, the machine learning and inference tools developed to support it, and selected initial experimental results from its application to real-world data.Item Combining Machine Learning with Evolutionary Computation: Recent Results on LEM(2000-06) Cervone, Guido; Michalski, Ryszard S.; Kaufman, Kenneth A.; Panait, Liviu A.The Learnable Evolution Model (LEM), first presented at the Fourth International Workshop on Multistrategy Learning, employs machine learing to guide evolutionary computation. Specifically, LEM integrates two modes of operation: Machine Learning mode, which employs a machine learning algorithm, and Darwinian Evolution mode, which employs a conventional evolutionary algorithm. The central new idea of LEM is that in machine learning mode, new individuals are "genetically engineered" by a repeated process of hypothesis formation and instantiation, rather than created by random operators of mutation and/or recombination, as in Darwinian-type evolutionary algorithms. At each stage of evoluation, hypotheses are induced by a machine learning system from examples of high and low performance individuals. New individuals are created by instantiating the hypotheses in different ways. In recent experiments concerned with complex function optimization problems, LEM has significantly outperformed selected evolutionary computation algorithms, sometimes achieving speed-ups of the evolutionary process by two or more orders of magnitude (in terms of the number of generations). In another recent application involving a problem of optimizing heat exchangers, LEM produced designs equal or superior to best expert designs. The recent results have confirmed earlier findings that LEM is able to significantly speed-up evolutionary processes (in terms of the number of generations) for certain problems. Further research is needed to determine classes of problems for which LEM is most advantagious.Item Experimental Validations of the Learnable Evolution Model(2000-07) Cervone, Guido; Kaufman, Kenneth A.; Michalski, Ryszard S.A recently developed approach to evolutionary computation, called Learnable Evolution Model or LEM, employs machine learning to guide processes of generating new populations. The central new idea of LEM is that it generates new individuals by processes of hypothesis generation and instantiation, rather than by mutation and/or recombination, as in conventional evolutionary computation methods. The hypotheses are generated by a machine learning program from examples of high and low performance individuals. When applied to problems of function optimization and parameter estimation for nonlinear filters, LEM significantly outperformed the evolutionary computation algorithms used in experiments, sometimes achieving two or more orders of magnitude of evolution speed-up in terms of the number of generations (or births). An application of LEM to the problem of optimizing heat exchangers has produced designs equal to or exceeding the best human designs.Item Modeling User Behavior by Integrating AQ Learning with a Database: Initial Results(2002-06) Cervone, Guido; Michalski, Ryszard S.The paper describes recent results from developing and testing LUS methodology for user modeling. LUS employs AQ learning for automatically creating user models from datasets representing activities of computer users. The datasets are stored in a relational database and employed in the learning process through an SQL-style command that automatically executes the AQ20 rule learning program and generates user models. The models are in the form of attributional rulesets that are more expressive than conventional decision rules, and are easy to interpret and understand. Early experimental results from the testing of the LUS method gave highly encouraging results.Item Recent Results from the Experimental Evaluation of the Learnable Evolution Model(2002-07) Cervone, Guido; Kaufman, Kenneth A.; Michalski, Ryszard S.The Learnable Evolution Model (LEM) represents a form of non-Darwinian evolutionary computation that is guided by a learning system. Specifically, LEM "genetically engineers" new populations via hypothesis formation and instantiation. This paper presents results from new studies in which LEM was systematically tested on a range of optimization problems and a complex real world design task. The study involved LEM2, a new implementation oriented toward function optimization, and LEMd-ISHED, an implementation oriented toward engineering design. LEM2 strongly outperformed tested evolutionary algorithms in terms of the evolution length, measured by the number of fitness function evaluations needed to reach the solution. This evolutionary speedup also translated to an execution speedup whenever the fitness evaluation time was above a small threshold (a fraction of a second). Experiments with LEMd-ISHED on problems of optimizing heat exchangers (evaporators) produced designs that matched or exceeded designs produced by human experts.Item Speeding Up Evolution through Learning: LEM(2000-06) Michalski, Ryszard S.; Cervone, Guido; Kaufman, Kenneth A.This paper reports briefly on the development of a new approach to evolutionary computation, called the Learnable Evolution Model or LEM. In contrast to conventional Darwinian-type evolutionary algorithms that employ mutation and/or recombination, LEM employs machine learning to generate new populations. At each step of evolution, LEM determines hypotheses explaining why certain individuals in the population are superior to others in performing the designated class of tasks. These hypotheses are then instantiated to create a next generation. In the testing studies described here, we compared a program implementing LEM with selected evolutionary computation algorithms on a range optimization problems and a filter design problem. In these studies, LEM significantly outperformed the evolutionary computation algorithms, sometimes speeding up the evolution by two or more orders of magnitude in the number of evolutionary steps (births). LEM was also applied to a real-world problem of designing optimized heat exchangers. The resulting designs matched or outperformed the best human designs.Item The Development of the AQ20 Learning System and Initial Experiments(2001-) Michalski, Ryszard S.; Cervone, Guido; Panait, Liviu A.Research on a new system implementing the AQ learning methodology, called AQ20, is briefly described, and illustrated by initial results from an experimental version. Like its predecessors, AQ20 is a multi-purpose learning system for inducing general concepts descriptions from concept examples and counter-examples. AQ20 is viewed as a natural induction system because it aims at producing descriptions that are not only accurate but also easy to understand and interpret. This feature is achieved by representing descriptions in the form of attributional rulesets that have a higher representation power than decision trees or conventional decision rules. Among new features implemented in AQ20 are the ability to handle continuous variables without prior discretization, to control the degree of generality of rules by a continuous parameter, and to generate more than one rule from a star. Initial experimental results from applying AQ20 to selected problems in the UCI repository demonstrate a high utility of the new system.Item The Development of the Inductive Database System VINLEN: A Review of Current Research(2003-06) Cervone, Guido; Kaufman, Kenneth A.; Michalski, Ryszard S.The recently introduced Learnable Evolution Model (LEM) represents a form of non-Darwinian evolutionary computation that is guided by a learning system. Specifically, LEM "genetically engineers" new populations via hypothesis formation and instantiation. Initial experiments with a preliminary implementation of LEM were highly encouraging, but tentative. This paper presents results from a new study in which LEM was systematically tested on a range of optimization problems and a complex real world design task. The study involved LEM2, a new implementation oriented toward function optimization, and ISHED, an implementation oriented toward engineering design. In all cases of function optimization, LEM2 strongly outperformed tested evolutionary algorithms in terms of the evolution length, measured by the number of fitness function evaluations needed to reach the desired solution. This evolutionary speedup also translated to an execution speedup whenever the fitness evaluation time was above a small threshold (a fraction of a second). The most important result of the study was that the advantage of LEM2 over the tested Darwinian-style evolutionary methods in terms of evolution length grew rapidly with the growth of the complexity of the optimized function. Experiments with ISHED on problems of optimizing heat exchangers (evaporators) produced designs that matched or exceeded designs produced by human experts. The obtained very strong results from the application of the LEM methodology to two diverse domains suggest that it may be useful also in other application domains, especially, those in which the fitness function evaluation is time-consuming or complex.