My scientific discipline is computer and information sciences. My research interests are detecting outliers in data and complex data sets.
Daily, I work with the Institute of Computer Science of the University of Silesia with academic scientist Agnieszka Nowak-Brzezińska, PhD, DSc, Associate Professor.
27th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems. KES International , Athens, Greece, September 6-8, 2023
International Conference Proceedings Series Lecture Notes in Computer Science Published by Springer Nature
The scientific gathering was dedicated to exploring both the theoretical frameworks and practical uses associated with rough set theory, which is a mathematical instrument employed for handling ambiguous and incomplete data when representing and processing knowledge.
Outlier detection is vital in data analysis, cleansing, anomaly detection, and predictive modelling, applicable across finance, medicine, networks, marketing, industry, and research. Categorical and mixed data analysis, distinct from numerical data, demands new methods for processing large datasets. This study focuses on outlier identification in databases, using Local Outlier Factor 𝐿𝑂𝐹, Self-Organizing Maps 𝑆𝑂𝑀, and Autoencoder algorithms. Its objective is to assess the effectiveness of these algorithms in detecting outliers in categorical data, aiming to determine their accuracy and provide support to experts. Additionally, innovative optimisation techniques, such as adjusting the "block size", have been introduced to enhance the efficiency of the 𝐿𝑂𝐹 algorithm. Six diverse datasets with about 2,000 to 285,000 samples featuring nominal, ordinal, and numerical features were analysed. These datasets included multiple variables, extensive samples, and complex, rare anomalies. This research contributes to categorical data analysis by comparing algorithmic approaches and enhancing the understanding of outlier detection. The results highlight the Autoencoder’s high effectiveness and the improved efficiency of 𝐿𝑂𝐹 through data partitioning. These findings are crucial for those analysing categorical data and seeking outlier patterns in databases, aiding knowledge discovery and decision-support systems.
26th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems. KES International , Verona, Italy, September 7-9, 2022
The research addresses the problem of outlier detection using Kohonen’s self-organising map (SOM) algorithm. Outliers can indicate something scientifically interesting, and we must analyse them deeply. In this research, we do not assume that after outliers are detected, they are removed. We hand them over to domain experts asking for further exploration. We have experience applying the LOF (local outlier factor) algorithm to detect outliers in real data. Now we wanted to check how a new algorithm, a SOM algorithm, will behave.
We compare the results of the outlier detection process using the LOF algorithm with the SOM algorithm results. We were interested in whether a data type significantly influences the efficiency of the process of outlier detection. Qualitative data is much more difficult to explore than quantitative data. We chose two various datasets (quantitive and qualitative). The research aims to change learning and clustering parameters and compare LOF and SOM algorithms to check which is more efficient for the outlier detection task.
The accuracy of the SOM algorithm for detecting outliers is satisfactory. Approximately (45%) forty-five per cent of the actual outliers have been successfully detected. This algorithm is faster than the LOF algorithm, the most popular algorithm for detecting outliers in the literature. The results show a strong negative correlation between the learning parameters and the learning time of the SOM map and quantisation error time calculations.
See full article
25th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems. KES International, Szczecin, Poland, September 8-10, 2021
The article concerns the detection of outliers in rule-based knowledge bases containing data on Covid 19 cases. The authors move from the automatic generation of a rule-based knowledge base from source data by clustering rules in the knowledge base to optimise inference processes and to detect unusual rules allowing for the optimal structure of rule groups.
The paper presents a two-phase procedure, wherein in the first phase, we look for the optimal structure of rule clusters when there are outlier rules in the knowledge base. We detect outliers in the rules in the second phase using the LOF (Local Outlier Factor) algorithm. Then we eliminate the unusual rules from the database and check whether the selected cluster quality measures responded positively to eliminating outliers, indicating that the rules were rightly considered outliers.
The performed experiments confirmed the effectiveness of the LOF algorithm and selected cluster quality measures in the context of detecting atypical rules. The detection of such rules can support knowledge engineers or domain experts in knowledge mining to improve the completeness of the knowledge base, which is usually the basis of the decision support system.
The article aims to analyse LOF, COF and KMEANS algorithms for outlier detection in rule-based knowledge bases. The subject of outlier mining is very important nowadays. Outliers in rules mean unusual rules that are rare compared to others and should be explored further by the domain expert.
In the research, the authors use outlier detection methods to find a given (1%, 5%, 10%) number of outliers in rules. Then, they analyse which of seven quality indices they used for all rules and, after removing selected outliers, improve the quality of rule clusters. In the experimental stage, the authors used six different knowledge bases. The results show that the optimal results were achieved for the COF outlier detection algorithm as the one for which, among all analysed quality indices, the cluster quality improved most frequently.
The article presents both clustering and outlier detection methods in complex data, such as rule-based knowledge bases. What distinguishes this work from others is the application of clustering algorithms to rules in domain knowledge bases and, secondly, the use of outlier detection algorithms to detect unusual rules in knowledge bases.
The paper aims to analyse four algorithms for outlier detection in rule-based knowledge bases: Local Outlier Factor (LOF), Connectivity-based Outlier Factor (COF), K-MEANS, and SMALL CLUSTERS. The subject of outlier mining is significant nowadays. Outliers in rules If-Then mean unusual rules, which are rare compared to others and should be explored by the domain expert as soon as possible.
In the research, the authors use the outlier detection methods to find a given number of outliers in rules (1%, 5%, 10%), while in small groups, the number of outliers covers no more than 5% of the rule cluster. Subsequently, the authors analyse which of seven quality indices they use for all rules and, after removing selected outliers, improve the quality of rule clusters. In the experimental stage, the authors use six different knowledge bases. The best results (the most often, the cluster quality was improved) are achieved for two outlier detection algorithms, LOF and COF.
The literature review proves that there are no tools to detect deviations in the dataset before the actual clustering. For that reason, the authors took up the challenge to create the SOaCRaport package, which allows for detecting outliers in any dataset using LOF and COF techniques, groups all data with and without outliers hierarchically, and compares the quality of created clusters.
February 2021 - September 2021
February 2023 - September 2023
Application dashboard link: https://studies.shinyapps.io/RulesAnalysis/
laboratory at the Faculty of Science and Technology during the academic years 2020/2021 - led by dr hab. Agnieszka Nowak-Brzezinska, co-conducting classes
©2024