Multiagent-based processing and integration of system data

This paper presents a multiagent-based ETL (Extract, Transform, Load) unit for the processing and integration of system operational data in order to improve its value. Operational data plays a vital role in managing and optimising systems. Although KDD (Knowledge Discovery and Data Mining) techniques and concepts have long existed, it is only now that we are seeing real applications being extended onto network and systems management. However, the massive data pre-processing (e.g. feature extraction and data integration) which is needed prior to putting KDD tools in action, is still limiting the extent of exploitation. We propose and design the multiagent-based ETL unit which uses Support Vector Machine and Natural Language Processing techniques to efficiently extract information features from operational data. The unit uses an mSPIDER algorithm to discover INclusion Dependencies (INDs) which are used to integrate data across its peers within the system. We demonstrate efficiency of the unit and the used approaches using operational data from a mailing system.