Softplus Function

📐 Definition

The softplus function is a smooth approximation of the hinge $\max(0,x)$ :

\operatorname{softplus}(x) = \log(1 + e^x)

It can also be written as a two-term log-sum-exp:

\operatorname{softplus}(x) = \operatorname{LSE}(0,x)

Domain and Codomain

Softplus is defined for all real inputs and returns positive real values: $\operatorname{softplus}:\mathbb{R}\to(0,\infty)$ .

⚙️ Key Properties

Derivative (Logistic)

\frac{d}{dx}\operatorname{softplus}(x) = \frac{1}{1 + e^{-x}}

Asymptotics

\lim_{x\to-\infty}\operatorname{softplus}(x) = 0, \qquad \lim_{x\to\infty}\big(\operatorname{softplus}(x) - x\big) = 0

Temperature Scaling

A sharper transition can be obtained by introducing an inverse-temperature $\beta>0$ :

\operatorname{softplus}_\beta(x) = \frac{1}{\beta}\log\!\left(1+e^{\beta x}\right)

🎯 Special Values and Limits

$x$	$\operatorname{softplus}(x)$
$0$	$\log 2$

As $\beta \to \infty$ , $\operatorname{softplus}_\beta(x)$ converges pointwise to $\max(0,x)$ .

Softplus is a log-sum-exp primitive and is tightly related to exponential/logarithmic stabilization. Hyperbolic tangent and soft clamping provide alternative smooth transitions with saturation.

Usage in Oakfield

Oakfield uses softplus-style expressions primarily for smooth clamping in operator logic:

Thermostat smooth clamps use a softplus-like function (log1p(exp(kx))/k) to apply smooth min/max limiting without hard kinks (see thermostat operator).
Softplus is also a convenient conceptual model for “soft” thresholds in UIs and parameter mappings, but it is not currently exposed as a standalone field operator.

Historical Foundations

📜 Smooth Approximations

Softplus is a modern smooth surrogate for the hinge/rectifier, closely connected to log-sum-exp and softmax constructions that became standard in numerical optimization and statistical modeling.

🌍 Modern Perspective

It is widely used as a differentiable “soft” hinge, with a tunable sharpness parameter via temperature scaling.

📚 References

Boyd & Vandenberghe, Convex Optimization
Cover & Thomas, Elements of Information Theory