Squareplus: A Softplus-Like Algebraic Rectifier
We present squareplus, an activation function that resembles softplus, but which can be computed using only algebraic operations: addition, multiplication, and square-root. Because squareplus is ~6x faster to evaluate than softplus on a CPU and does not require access to transcendental functions, it may have practical value in resource-limited deep learning applications.
Code (1)
Tasks
CPUSimilar Papers 제목 키워드 기반
Implementation of Hyperelastic Physics-Augmented Neural Networks in the Explicit Finite Element Codes Simcenter Radioss and OpenRadioss with Applications to Impact Events
Data-driven material modeling techniques have gained significant attention due to their ability to capture complex constitutive behaviors beyond the limitations of classical material models. Physics-augmented neural netw…
Computational EfficiencyTowards Rigorous Understanding of Neural Networks via Semantics-preserving Transformations
In this paper we present an algebraic approach to the precise and global verification and explanation of Rectifier Neural Networks, a subclass of Piece-wise Linear Neural Networks (PLNNs), i.e., networks that semanticall…
DiagnosticOptimal Switching of Controlled Rectifiers
This paper discusses a linear programming approach for designing switching signals for controlled rectifiers to achieve a low input current & output voltage total harmonic distortions. The focus here is on fully controll…
Schauder Bases for $C[0, 1]$ Using ReLU, Softplus and Two Sigmoidal Functions
We construct four Schauder bases for the space $C[0,1]$, one using ReLU functions, another using Softplus functions, and two more using sigmoidal versions of the ReLU and Softplus functions. This establishes the existenc…
Unconstrained Multi-view Human Pose Estimation with Algebraic Priors
Recovering 3D human pose from multi-view imagery typically relies on precise camera calibration, which is often unavailable in real-world scenarios, thereby severely limiting the applicability of existing methods. To ove…
Pose Estimation