3 Important Types of Vector Norm Used in Machine Learning
The length or the magnitude of the vector is known as vector norm or vector magnitude. In mathematics, a function is defined on a vector space that maps each vector to its magnitude (a scalar quantity). In this article, we will discuss different types of vector norm that are used in Machine Learning.
In the previous articles, we discussed how vectors are used in machine learning. In this article, we will discuss the norm of a vector. A norm is a way to measure the size of a vector, matrix, or tensor; it helps to quantify the magnitude of a vector. In simple terms, the length of the vector is referred to as the vector norm or the vector’s magnitude.
In machine learning, vector norms are used to evaluate the model’s error or reduce the model’s overall complexity.
This article will discuss what vector norm is, the types of vector norms, and how to calculate the vector norm using python.
Best-suited Machine Learning courses for you
Learn Machine Learning with these high-rated online courses
What is Vector Norm
The length or the magnitude of the vector is known as vector norm or vector magnitude. In mathematics, a function is defined on a vector space that maps each vector to its magnitude (a scalar quantity).
In Machine learning, we use them to measure the distance between multiple data points in different dimensions.
For example, the euclidean distance formula is the most often used formula to calculate the distance between any two points.
Let there be two points in 2-dimensional vector space:
P = (p1, p2)
Q = (q1, q2).
Then the distance between P and Q can be calculated by:
dist(P, Q) = [(p1 – q1)2 + (q1 – q2)2](½)
Example: Find the distance between P (2, 3) and Q (4, 5).
Answer: The distance between P and Q is:
dist(P, Q) = [(4 – 2)2 + (5 – 3)2]½
=> dist(P, Q) = [22 + 22]½
=> dist(P, Q) = [4 + 4]½
Hence, dist(P, Q) = 2.83 units
Notation
Let x be any vector, then the norm of x is given by: ||x||.
Properties of Vector Norm
- Positive Definiteness: The norm (or the magnitude) of a vector is always positive.
- The norm value is zero if and only if the vector is zero.
- i.e., ||x||>= 0, for all non-zero vector x.
- The norm value is zero if and only if the vector is zero.
- Triangle Inequality: The sum of the norm of two vectors is always less than the sum of their (individual) norm.
- If x and y are two vectors, then,
- ||x + y|| <= ||x|| + ||y||
- If x and y are two vectors, then,
- Homogeneity: Let k be any constant, and x be any vector then:
- ||kx|| = |k|||x||
| Set Theory | Differential Calculus |
| Maxima and Minima | Functions in Mathematics |
| Mean Value Theorem | Taylor Series |
Application of Norms of Vector in Machine Learning
- Used to evaluate the error in the model, i.e., the distance between the observed value and predicted value.
- Vector norms are used in regularization, which is a technique to prevent overfitting in the model.
Now, we will see the different types of Vector Norms.
Vector Norms in Machine Learning
There are different types of vector norms that can be used, such as L1, L2, Lp, L-infinity, Min-Max norm, etc.
But in machine learning, we mainly use L1, L2, and Min-Max Norm.
So, without further delay, let’s start with Lp Norm as all are special cases of Lp norm.
Lp Norm
It is defined as the absolute sum of the absolute value of the elements of the vector.
Note: L^p norm is the generalization of Euclidean Distance.
It is defined as the sum of the absolute value of the elements of the vector, raised to the pth power and then taking the pth root of the result. The Lp norm is written as ||x||p, where x is the vector and p is a positive real number.
Python Program to Calculate the Lp Norm:
#Import norm from linalg (linear algebra) module of NumPy Libraryimport numpy as npfrom numpy.linalg import norm #define a vector using numpy array v = np.array([1, 4, 9, 16, 25]) #find the L-p norm for p = 3p = 3vector_norm = norm(v, ord = p)print('The vector norm of v is:', vector_norm)
Output
The vector norm of v is: 27.375191994928183
L1-Norm
If p = 1, then the Lp norm is known as L1 – Norm. It is defined as the sum of the absolute value of the components of a given vector.
Let x be any vector, then the L1 norm of a vector is given by:
- L1-Norm is also known as Manhattan or the Taxicab Norm.
- It is typically the distance a taxi will have to cover from the origin.
- It is used in LASSO Regression.
L2-Norm
If p = 2, then the Lp norm is known as L2 – Norm. It is defined as the square root of the sum of the squared vector values.
Let x be any vector, then the L2 norm of a vector is given by:
- It is also known as Euclidean Norm.
- It is mainly used in KNN Algorithm and Ridge Regression.
Must Read: Ridge Regression vs Lasso Regression
L infinity-Norm
If p = infinity, then the Lp norm is known as the L infinity norm. It is defined as the absolute value of the largest component of the vector.
It is also called the Max Norm.
Conclusion
In this article, we have learned the following:
- What is Vector Norm
- Properties and Application of Vector Norm
- How to Calculate the Norm of a vector
- Lp Norm
Top Trending Article
Top Online Python Compiler | How to Check if a Python String is Palindrome | Feature Selection Technique | Conditional Statement in Python | How to Find Armstrong Number in Python | Data Types in Python | How to Find Second Occurrence of Sub-String in Python String | For Loop in Python |Prime Number | Inheritance in Python | Validating Password using Python Regex | Python List |Market Basket Analysis in Python | Python Dictionary | Python While Loop | Python Split Function | Rock Paper Scissor Game in Python | Python String | How to Generate Random Number in Python | Python Program to Check Leap Year | Slicing in Python
Interview Questions
Data Science Interview Questions | Machine Learning Interview Questions | Statistics Interview Question | Coding Interview Questions | SQL Interview Questions | SQL Query Interview Questions | Data Engineering Interview Questions | Data Structure Interview Questions | Database Interview Questions | Data Modeling Interview Questions | Deep Learning Interview Questions |
Vikram has a Postgraduate degree in Applied Mathematics, with a keen interest in Data Science and Machine Learning. He has experience of 2+ years in content creation in Mathematics, Statistics, Data Science, and Mac... Read Full Bio
Top Picks & New Arrivals
