Μελέτη και σύγκριση συναρτήσεων ενεργοποίησης σε νευρωνικά δίκτυα

Abstract

The flourishing of Artificial Intelligence (AI) in recent years, mainly due to the technological advancements resulting in high-performing hardware that boosted the field’s rise, has led to the research and development of many real-world applications. By extension, since the field’s theoretical beginnings are now applied to solve real-world problems, Machine Learning (ML), a subcategory of AI has been proven highly advantageous for computer vision tasks including image classification. This led to the development of various high-performing image classification Neural Networks (NN), each one with a different architectural approach. Through Transfer Learning (TL) these networks can be used for the development of real-world applications. However, such applications come with challenges that require an NN performing a task to be highly efficient, accurate, fast, stable generalized, and as less computational powerconsuming as possible. There is constant research to improve models by designing innovative architectures through various tools and techniques, including activation functions. This work focuses on improving popular pre-trained image classification NNs of high architecture and performance by altering the activation functions they use in their core. The models are trained for five datasets, each time with a different activation function in their entirety of architecture. Nine activation functions were chosen for testing. The experiments show optimistic results as improvements in performance in terms of accuracy or training time are possible and in many cases to a high extent.