How Vectors are Used in Machine Learning

Vectors are mathematical objects that contain both magnitude and direction, and they can be represented by the directed line segments (lines having directions) whose lengths are their magnitude. It is used to describe the movement of an object from one point to another. In this article, we will briefly discuss, what is vector, types of vectors, and vector arithmetic.

A vector is a data structure with at least two components: magnitude and direction. It is most commonly used in machine learning to represent the data in the most optimized and organized way.

In machine learning, while training any model, if our dataset contains the images and text, they are first converted into numbers and then stored in the form of vectors and matrices to represent these data, i.e., the very first step in building a machine learning model is vectorizing the data. Once the data is vectorized, you can easily use different linear algebra operations (or tools) to perform model training, data augmentation, etc.

One of the most common examples of vectors in machine learning is Support Vector Machines (a supervised machine learning algorithm that uses classification techniques to solve the problem). SVM analyzes the vectors across n-dimensional space to find the optimal hyperplane for a given dataset.

We don’t need a deep mathematical understanding of vectors to use vectors in machine learning. This article will discuss vectors, geometric representation, types of vectors, and the operations that can be performed on these vectors.

Without further delay, let’s dive deep to learn more about vectors.

Table of Content

• What is a Vector
  • Definition
  • Magnitude of Vectors
• Types of Vectors
  • Zero Vector
  • Unit Vector
  • Negative Vector
  • Parallel Vector
  • Equal Vector
  • Orthogonal Vector
• Vector Arithmetic
  • Addition
  • Subtraction
  • Scalar Multiplication of Vectors
  • Product
    • Dot Product
    • Cross Product

What is a Vector?

Definition

Vectors are mathematical objects that contain both magnitude and direction, and they can be represented by the directed line segments (lines having directions) whose lengths are their magnitude. It is used to describe the movement of an object from one point to another.

• The direction of the vector is from its tail to its head.
• Notation: The vector between two points (A & B) can be given by:
  • a or A or AB
  • Standard Form: A = ai +bj+ck, where a, b, c are real numbers, and i, j, and k are the unit vectors along the x, y, and z-axis.

Example: Velocity of a car.

In simple terms, a car’s velocity means the car’s speed is moving in a particular direction.

It can also be defined as a Tuple of one or more scalar values.

Example: V = (a, b, c); here, a, b, and c are scalars (real values).

Types of Matrix	Transpose of Matrix
Symmetric Matrix	Skew-Symmetric Matrix

Now, we will discuss how to find the magnitude of the vector.

Magnitude

The magnitude of any vector can be easily calculated by taking the square root of its component, i.e., if 

A = ai + bj + ck, then,

Magnitude of A = |A| = sqrt (a² + b² + c²)

Example: a = 3i + 4j – 7k. Find the magnitude of a.

Answer:

Note: In Machine learning, we use different vector norms (or magnitude) to find the distance, which are also called distance matrices:

• Manhattan Distance
• Euclidean Distance
• Minkowski Distance
• Hamming Distance

Types of Vectors

There are different types of vectors, but here we will discuss seven different types of vectors that are commonly used:

Zero Vector

A vector is said to be a zero vector if the magnitude of the vector is zero. 

• It is denoted by: O = (0, 0, 0). 
• Also known as Additive Identity
  • i.e., A + O = A = O + A

Unit Vector

A vector is said to be a unit vector if the magnitude of the vector is one.

Negative Vector

A vector is said to be a negative vector of a given vector if it has the same magnitude but points in the opposite direction.

• In simple terms, when we multiply any vector with the -1, it changes the direction of the vector.
  • i.e., (-1)v = -v

Parallel Vectors

Two vectors, a, and b are said to parallel if they have the same direction but not the same magnitude.

Equal Vectors

Two vectors (a and b) are said to be equal if they have the same magnitude and direction.

• If a = x₁i + y₁j + z₁k, and b = x₂i + y₂j + z₂k, then
  • a = b if and only if x₁ = x₂, y₁ = y₂, and z₁ = z₂
    • i.e., two vectors are equal if their corresponding components are equal.

Orthogonal Vectors

Two vectors, a, and b, are orthogonal if and only if they are perpendicular to each other.

• The angle between them is a right angle.
• Mathematically, vectors are orthogonal if the dot product of vectors is zero.

Programming Online Courses and Certification	Python Online Courses and Certifications
Data Science Online Courses and Certifications	Machine Learning Online Courses and Certifications

Until now, we have discussed what vectors are, types of vectors, and now it’s time to know how to perform arithmetic over these vectors.

Vector Arithmetic

Addition of Vectors

Two vectors, a, and b, are added using the Triangle Law of Addition.

If two vectors, a, and b, are represented as the side of a triangle with the magnitude and direction, then the third side of the triangle (magnitude and direction) will be the resultant vector.

• If a = a₁i + b₁j + c₁k, b = a₂i + b₂j + c₂k, then,
  • a+b = (a₁+a₂)i + (b₁+b₂) j + (c₁+c₂) k
• Vector addition follows: 
  • Commutative law, i.e., a + b = b + a
  • Associative law, i.e., a + (b + c) = (a + b) + c

Subtraction of two Vectors

Vector subtraction is similar to vector addition, i.e., if a and b are two vectors, such that a = a₁i + b₁j + c₁k, b = a₂i + b₂j + c₂k, then,

• a – b = a + (–b) = (a₁ – a₂) i + (b₁ – b₂) j + (c₁ – c₂) k

Scalar Multiplication of Vectors

If a = x1i + y1j + z1k and b = x2i + y2j + z3k are two vectors, and c is any scalar (or constant), then:

• ca = c (x₁i + y₁j + z₁k) = (cx₁) i + (cy₁) j + (cz₁) k
• c(a+b) = ca + CB
• (c+I) = ca + cI

Product

Two vectors, a, and b, can be multiplied in two ways:

Dot Product

A dot product (or a scalar product) is a mathematical operation that takes two products and returns a scalar product. It is calculated by multiplying the corresponding elements of the vectors.

• If a and b are two vectors, a = (a1, a2, a3, …, an), and b = (b1, b2, b3, …, bn), then
  • a · b = (a₁ * b₁) + (a₂ * b₂) + … + (a_n * b_n)
• If the angle between two vectors is given, then:
  • a · b =|a| · |b| cos (theta), 

where, 

• theta is the angle between a and b
• |a|, and |b| are the magnitude of a and b.

Must Read: Dot Product – All That You Need to Know

Cross Product

Cross product is a binary operation (multiplication) that is performed on two vectors, and the resultant vector is perpendicular to both the given vectors. It is calculated by using the determinants, i.e.,

• If a = a₁i + a₂j + a₃k, and b = b₁i + b₂j + b₃k, then
  • a x b = (a₂b₃ – a₃b₂)i – (a₁b₃ – a₃b₁)j + (a₁b₂ – a₂b₁)k
• If the angle between two vectors is given, then:

where,

|a| is the magnitude of a

|b| is the magnitude of b

theta: angle between a and b

n: unit vector at a right angle to both a and b

Must Read: Cross Product of Two Vectors

Also Read: Matrix Multiplication

Top Trending Article

Top Online Python Compiler | How to Check if a Python String is Palindrome | Feature Selection Technique | Conditional Statement in Python | How to Find Armstrong Number in Python | Data Types in Python | How to Find Second Occurrence of Sub-String in Python String | For Loop in Python |Prime Number | Inheritance in Python | Validating Password using Python Regex | Python List |Market Basket Analysis in Python | Python Dictionary | Python While Loop | Python Split Function | Rock Paper Scissor Game in Python | Python String | How to Generate Random Number in Python | Python Program to Check Leap Year | Slicing in Python

Interview Questions

Data Science Interview Questions | Machine Learning Interview Questions | Statistics Interview Question | Coding Interview Questions | SQL Interview Questions | SQL Query Interview Questions | Data Engineering Interview Questions | Data Structure Interview Questions | Database Interview Questions | Data Modeling Interview Questions | Deep Learning Interview Questions |