Gained knowledge exchange and analysis for meta-learning Norbert Jankowski and Krzysztof Gr ˛ abczewski Department of Informatics Nicolaus Copernicus University Toru´ n, Poland http://www.is.umk.pl/ {norbert|kgrabcze}@is.umk.pl Abstract— Building accurate and reliable complex machines is not trivial (but necessary in most real life problems). Typical ensembles are often unsatisfactory. Meta-learning techniques can be much more powerful in composing optimal or close to optimal solutions to given tasks. Efficient meta-learning is possible only within a versatile and flexible data mining framework providing uniform procedures for dealing with different kinds of methods and tools for thorough analysis of learning processes and their results. We propose a methodology for information exchange be- tween machines of different abstraction levels. Inter-machine communication is based on uniform representation of gained knowledge. Implemented in a general data mining framework, it provides tools for sophisticated analysis of adaptive processes of heterogeneous machines. The resulting meta-knowledge is a brilliant information source for further meta-learning. I. I NTRODUCTION Most real life classification (and other data mining) tasks are so hard, that single simple classifiers are very unlikely to successfully solve them. The need for complex machines including different data transformations and classification en- sembles is undeniable. Even simple ensemble construction methods often end up with poor results because of different reasons. To obtain high classification scores for different datasets of different kinds, we need tools which facilitate not only construction of classifiers, but also proper data transformation and proper validation of complex structures of machines. Techniques leading to satisfactory solutions of different problems are the subject of research called meta-learning. Although this term has been used in numerous articles in the sense of ranking algorithms according to descending probability of being successful when applied to given data [1], [2] or in the sense of simple ensembles construction, we use it in much broader sense of learning how to learn different tasks. Our meta-learning ideas combine different heuristic search procedures based on knowledge extracted from past learning scenarios with active analysis of the results of application of different methods to given data. Efficient meta-learning techniques will more and more often present us successful models, which would be very difficult for humans to find, because of their unusual structures. Some of our research has already born the fruits of very high accuracies of our classifiers solving tasks of the NIPS 2003 Feature Selection Challenge 1 [3] and the Handwritten Digit Recognition Competition 2 organized with The Eighth International Conference on Artificial Intelligence and Soft Computing in 2006. The models we found were usually com- plex model structures consisting of some data transformations like standardization, feature selection, features construction based on principal components analysis and some committees of classifiers. We have also examined some aspects of member- model competence in classification committees [4]. All such meta-learning approaches require large amount of calculations (e.g. to validate the candidate machines) before they point to most attractive solutions. Many candidates must be examined, numerous combinations validated often with different optimization criteria. To make it all possible we need a general data mining system, which efficiently manipulates such complex machines. Such system must provide: • uniform way of machine configuration and machine creation—the possibilities of adding, configuring, train- ing, testing and removing machines in a standard way, implemented as a set of project management routines in such a way that does not burden the authors of particular machines with the administration efforts, • uniform access to results of machine learning and tests, so that meta-learning machines do not need much (or even any) knowledge about the specificity of particular machines, • uniform query system for gathering information from submachines, facilitating versatile and efficient analysis of gathered results. The mechanisms must be uniform but not too restrictive, i.e. general enough to fit any kind of adaptive machines (also the results of machines which will be constructed in future). The abstraction of management routines facilitates com- munication between different machines within the project on appropriate, different levels of abstraction. Dependently on particular needs, general or detailed questions may be asked in a common language without the necessity to know the details of the machines being used. This provides an excellent source 1 http://www.clopinet.com/isabelle/Projects/NIPS2003/ 2 http://www.icaisc.pcz.czest.pl/competition.htm