All that You Need to Know About Sigmoid Function
The sigmoid function is a special case of a logistic function that has S-shaped characteristic and are used as an activation function in Neural Networks. In this article, we will briefly discuss logistic function, sigmoid function.
As in linear regression algorithms, we use linear function or equation to give the relationship between dependent and independent variables; similarly, in logistic regression algorithms, we use logistic function (or sigmoid function). A logistic function is an exponential function that has a characteristic S-shaped curve.
In logistic regression, we mainly use the sigmoid function, a particular Logistic Function case. This article will briefly discuss the logistic function and the Sigmoid Function.
Now, let’s dive deep to learn more about Logistic and sigmoid functions and how to use them.
Best-suited Machine Learning courses for you
Learn Machine Learning with these high-rated online courses
What is Logistic Function?
The logistic function is a particular case of the exponential function that is used to model the exponential growth of a population that is derived from the differential equation:
Now, to solve the above differential equation, substitute:
in the above equation, we get the following:
and on solving the above differential equation, we will get the following:
where:
x0: is the initial value at time t0
The above equation (eqn. – (iii)), is known as Logistic Function.
What is Sigmoid Function?
The sigmoid function is a particular case of logistic function (eqn. – (iii)) when the initial time (t0 = 0), x0 = ½, and r = 1.
So, substituting the above values (t0 = 0, and x0 = ½) in equation (iii), we will get:
The sigmoid function looks like this:
From, the above figure you can easily depict why sigmoid functions are also called S-shaped curve.
- The sigmoid function is the solution of the non-linear differential equation:
- Mainly used with a non-linear activation function.
- It is used with the model where we have to predict the probability as an output of the model.
- Since the probability value lies between 0 and 1.
Properties of Sigmoid Function
- Domain: (-infinite, +infinite)
- Range: (0, 1)
- x(0) = ½ = 0.5
- The sigmoid function is continuous and monotonically increasing everywhere.
- The function is differentiable everywhere.
- Maps feature space into probability function:
- When x-> + infinity, the value of the sigmoid function will be close to 1.
- When x-> -infinity, the value of the sigmoid function will be close to 0.
- For x = 0, the value of the sigmoid function will be ½.
Application of Sigmoid Function
- Used as an activation function in Neural Networks.
- Used to predict the probability of binary variables.
- It is used in the Logistic Regression Algorithm.
Also Read: ReLU and Sigmoid Activation Function
Advantages and Limitations of Sigmoid Function
- It gives a smooth gradient that prevents jumps in output values.
- One of the best-normalized functions.
- When used with a linear function, it will return a value between 0 and 1, which does not make the activation value disappear.
- Gradient values are only significant for the range -3 to 3.
- The graph will have minimal gradients for values greater than 3 or smaller than -3.
- The main disadvantage of the sigmoid function is that it suffers from the vanishing gradient problem.
- As the gradient value approaches zero, the network ceases to learn and suffers from a vanishing gradient descent problem.
- When sigmoid is used and:
- Inputs are non-zero centered; then saturation is a concern
- Saturation implies gradient will be zero.
- Inputs are non-zero centered; then saturation is a concern
- Extremely greater or smaller values are mapped to extremities to 0 and 1.
| Programming Online Courses and Certification | Python Online Courses and Certifications |
| Data Science Online Courses and Certifications | Machine Learning Online Courses and Certifications |
Implementation of Sigmoid Function in Python
Now, we will create the graph of sigmoid function in Python:
#import important libraries
import numpy as npimport mathfrom matplotlib import pyplot as plt
#use numpy.linspace (start, stop, num) that will return number spaces with respect to interval# num: Number of samples to generate
x = np.linspace(-20, 20, 500)
#define sigmoid functiony = 1/ (1 + np.exp(-x))
#plot the function
plt.plot(x, y)plt.xlabel("X")plt.ylabel("Sigmoid (X)")
plt.show()
Output
Conclusion
This article briefly discusses logistic function, sigmoid function, properties, application, and implementation of sigmoid function in python.
Hope you will like the article.
Top Trending Article
Top Online Python Compiler | How to Check if a Python String is Palindrome | Feature Selection Technique | Conditional Statement in Python | How to Find Armstrong Number in Python | Data Types in Python | How to Find Second Occurrence of Sub-String in Python String | For Loop in Python |Prime Number | Inheritance in Python | Validating Password using Python Regex | Python List |Market Basket Analysis in Python | Python Dictionary | Python While Loop | Python Split Function | Rock Paper Scissor Game in Python | Python String | How to Generate Random Number in Python | Python Program to Check Leap Year | Slicing in Python
Interview Questions
Data Science Interview Questions | Machine Learning Interview Questions | Statistics Interview Question | Coding Interview Questions | SQL Interview Questions | SQL Query Interview Questions | Data Engineering Interview Questions | Data Structure Interview Questions | Database Interview Questions | Data Modeling Interview Questions | Deep Learning Interview Questions |
FAQs
What is a Sigmoid Function?
A sigmoid function is a mathematical function that produces a characteristic S shaped curve or sigmoid curve. In data science, it's often used in logistic regression and artificial neural networks to map predicted values to probabilities.
Why are Sigmoid Functions Important in Machine Learning?
Sigmoid functions are vital in machine learning as they help in predicting the probability of an outcome in a binary classification problem, which can be essential in decision-making processes.
How Does a Sigmoid Function Work?
The sigmoid function takes any range of input and outputs it between 0 and 1, making it particularly useful in algorithms where the output needs to be a probability.
Can Sigmoid Functions be used in Multi-class Classification?
While sigmoid functions are primarily used for binary classification, they can be extended to multi-class classification problems using techniques like One-Vs-Rest.
What are the Limitations of Sigmoid Functions?
Sigmoid functions can sometimes suffer from the vanishing gradient problem, where the gradients become too small for the network to learn effectively. This can slow down the training process and make the model less accurate.
Vikram has a Postgraduate degree in Applied Mathematics, with a keen interest in Data Science and Machine Learning. He has experience of 2+ years in content creation in Mathematics, Statistics, Data Science, and Mac... Read Full Bio
Top Picks & New Arrivals
