paper-with-me

Papers

Evolution of Novel Activation Functions in Neural Network Training with Applications to Classification of Exoplanets

2019-06-01 · Snehanshu Saha, Nithin Nagaraj, Archana Mathur, Rahul Yedida

We present analytical exploration of novel activation functions as consequence of integration of several ideas leading to implementation and subsequent use in habitability classification of exoplanets. Neural networks, although a powerful engine in supervised methods, often require expensive tuning efforts for optimized performance. Habitability classes are hard to discriminate, especially when attributes used as hard markers of separation are removed from the data set. The solution is approached from the point of investigating analytical properties of the proposed activation functions. The theory of ordinary differential equations and fixed point are exploited to justify the "lack of tuning efforts" to achieve optimal performance compared to traditional activation functions. Additionally, the relationship between the proposed activation functions and the more popular ones is established through extensive analytical and empirical evidence. Finally, the activation functions have been implemented in plain vanilla feed-forward neural network to classify exoplanets.

📄 PDF Abstract BibTeX arXiv:1906.01975

Code (3)

sahamath/sym-netv1
tej-prash/GeneExpression
yrahul3910/symnet

Tasks

General Classification

Similar Papers 제목 키워드 기반

Activation Function Optimization Scheme for Image Classification

2024-09-07 · Abdur Rahman, Lu He, Haifeng Wang

Activation function has a significant impact on the dynamics, convergence, and performance of deep neural networks. The search for a consistent and high-performing activation function has always been a pursuit during dee…

Classificationimage-classificationImage Classification

Evolving Parsimonious Networks by Mixing Activation Functions

2017-03-21 · Alexander Hagg, Maximilian Mensing, Alexander Asteroth

Neuroevolution methods evolve the weights of a neural network, and in some cases the topology, but little work has been done to analyze the effect of evolving the activation functions of individual nodes on network size,…

Task-Specific Activation Functions for Neuroevolution using Grammatical Evolution

2025-03-13 · Benjamin David Winter, William John Teahan

Activation functions play a critical role in the performance and behaviour of neural networks, significantly impacting their ability to learn and generalise. Traditional activation functions, such as ReLU, sigmoid, and t…

Binary Classification

Evolutionary Optimization of Deep Learning Activation Functions

2020-02-17 · Garrett Bingham, William Macke, Risto Miikkulainen

The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, the Rectified Linear Unit (ReLU) remains …

Deep LearningEvolutionary Algorithms

Evolution of Activation Functions for Deep Learning-Based Image Classification

2022-06-24 · Raz Lapid, Moshe Sipper

Activation functions (AFs) play a pivotal role in the performance of neural networks. The Rectified Linear Unit (ReLU) is currently the most commonly used AF. Several replacements to ReLU have been suggested but improvem…

Deep Learningimage-classificationImage Classification