Vector - Matrix - Tensors

Ref - Link

Array is analogous to Tensors

Keep Exploring!!!

Day #66 - Maths behind backpropagation

Today it's mathematical learning for neural network fundamentals.

• Derivation of Backpropagation
• Crash Course in Partial Differentiation
• A Step by Step Backpropagation Example

Keynotes

• In Neural Network, Network forward propagates activation to produce output and it back propagates error to determine weight changes
• Partial Derivative - Derivative of one of the variables holding the rest constant
• Backpropagation uses gradient descent method, one needs to calculate the derivative of squared error function with respect to the weights of the network.

Happy Learning!!!

Day #22 - Data science - Maths Basics

Eigen Vector - Vector along which there is no change in direction

Eigen Value - Amount of Scaling factor defined by Eigen value

Eigen Value Decomposition - Only Square matrix can be performed Eigen Decomposition

Trace - Sum of Eigen Values

Rank of A - Number of Non-Zero Eigen Values

SVD - Singular Value Decomposition

• Swiss Army Knife of Linear Algebra
• SVD - for Stock market Prediction
• SVD - for Data Compression
• SVD - to model sentiments
• SVD is Greatest Gift of Linear Algebra to Data Science
• Square Root of (Eigen Values of AtA) - A Transpose A, becomes Singular Value of

Happy Learning!!! (Revise  - Relearn - Practice)

Day #21 - Data Science - Maths Basics - Vectors and Matrices

Matrix - Combination of rows and columns
Check for Linear Dependence - R2 = R2 - 2R1, When one of the rows is all zeros it is linearly dependent
Span - Linear combination of vectors
Rank - Linearly Independent set

Good Related Read - Span

Vector Space - Space of vectors, collection of many vectors
If V,W belong to space, V+W also belongs to space, multiplied vector will lie in R Square
If the determinant is non-zero, then the vectors are linearly independent. Otherwise, they are linearly dependent

Vector space properties

• Commutative  x+y = y+x
• Associative (x+y)+z = x+(y+z)
• Origin vector - Vector will all zeros, 0+x = x+0 = x
• Additive (Inverse) - For every X there exists -x such that x+(-x) = 0
• Distributivity of scalar sum, r(x+s) = rx+rs
• Distributivity of vector sum, r(x+s) = rx+rs
• Identity multiplication, 1*x = x

Subspace
Vector Space V, Subset W. W is called subspace of V
Properties
W is subspace in following conditions

• Zero vector belongs to W 
• if u and v are vectors, u+v is in W (closure under +)
• if v is any vector in W, and c is any real number, c.v is in W

Subset S belongs to V can be represnted as linear combination
 v = r1v1+ r2v2+... rkvk
v1,v2 distinct vectors from S, r belongs to R

Basis - Linearly Independent spanning set. Vector space is called basis if every vector in the vector space is a linear combination of set. All basis for vector V same cardinality

Null Space, Row Space, Column Space
Let A be m x n matrix

• Null Space - All solutions for Ax = 0, Null space of A, denoted by Null A, is set of all homogenous solution for Ax=0
• Row Space - Subspace of R power N spanned by row vectors is called Row Space
• Column Space -  Subspace of R power N spanned by column vector is called Column Space

Norms - Measure of length and magnitude

• For (1,-1,2), L1 Norm = Absolute value = 1+1+2 = 4
• L1 - Same Angle
• L2 - Plane
• L3 - Sum of vectors in 3D space
• L2 norm (5,2) = 5*5+2*2 = 29
• L infinity - Max of (5,2) = 5

Orthogonal - Dot product equals Zero
Orthogonality - Linearly Independent, perpendicular will be linearly independent
Orthogonal matrix will always have determinant +/-1

Map of Mathematics.

Enlarge the figure to see all the wonderful areas for exploration and imagination. Which topic might you find most fascinating?

By Dominic Walliman, @DominicWalliman, Source: https://t.co/mNu0hWzFGW, Used with permission. pic.twitter.com/kx1azWIhle

— Cliff Pickover (@pickover) August 22, 2022

Differential Equations - Notes - Link

Lectures - Link

Course Notes - Link

Happy Learning!!!

Day #9– Data Science Basics Maths Notes

Happy Learning!!!

Good Read on Taylor Seris

Two summary points

• A Taylor Series is an expansion of a function into an infinite sum of terms, like these ones
• A derivative gives you the slope of a function at any point

Detailed Notes in link
Taylor series Formula Compilation - link

Happy Learning!!!

Good Formula Lists Compilations

Good Formula Lists Compilations

• Integrals 
• Stats 
• Mathematics - Rotations 
• Partial Derivatives
• Differential Equations
• Laplace Transform
• Elementary Differential and Integral Calculus
• More Reads

Happy Learning!!!

Maths - Basics

Good Read
Link1

Happy Learning!!!

Variance and Standard Deviation Example

Basic Example for Variance, Standard Deviation Computation

Happy Learning!!!

R Notes

Matrices
Tip #1 
Declaration - matrix (0,3,4)
3 rows, 4 columns and values 0
print[1,1]

Tip #2
Vector a <- 1:12 (To fill Matrix)
matrix(a, 3, 4)

Tip #3

Assigning, Fetching & Printing values
plank <- 1:8 - Create Vector
dim(plank) <- c(2,4) - Assign Dimension
print(plank) - Print values
plank[1,4] <- 0 - Assign value
plank[2,] - Fetch all 2nd row values

Tip #4 - Plotting Matrix

elevation <- matrix(2,5,10)
contour(elevation) - 2D representation
persp(elevation) - 3D representation
persp(elevation, expand = 0.2)

Statistics
Tip #1 - Mean computation
a <- c(1,2,3,4,5)
mean(a)
barplot(a)
median(a)

Tip #2
Horizontal line across plots
abline(h=mean(a))
abline(h=median(a))

Tip #3 
Standard Deviation
deviation <- sd(a)

Factors
Tip #1 
R has special collection called factors
chests <- c('gold', 'silver', 'gems', 'gold', 'gems')
types <- factor(chests)
print(types)

Data Frame
Similar to Database
Tip #1 - Loading data from files
read.csv("data.csv")
read.table("a.txt",sep="\t")

Tip #2 - Merge Data Frames
data1 <- read.csv("data.csv")
data2 <- read.table("a.txt",sep="\t")
merge(x = data1, y = data2)

Real World Examples
Data1 <- read.csv("a.csv")
Data2 <- read.table("b.txt", sep="  ", header=TRUE)
TargetData <- merge(x = Data1, y = Data2)
plot(TargetData$Data1, TargetData$Data2)

More Reads
R Programming Examples
R Time Series

R Programming Examples
R Time Series

Happy Learning!!!

R Online Learning

I prefer to switch topics when I find it tricky to focus on one topic. I found R language easy, simple and great to get started. Codeschool has a beautiful self learning portal. This lists cheat sheet and fundamentals working with R. Capturing some of notes for my future reference.

Using R

Tip #1 - Assignment
x <- 42
y <- "Hello"

Tip #2 - Expression
5==10
5>10

Tip #3 - Arithmetic operations
2+2
3*3

Tip #4 - Functions
sum(1,2)
sum(1,2,3,4)

Help

Tip #5 - File I/O
list.files()
source(filename)

Vectors

Tip #1 - Vector
List of Values - vector represented by c(2,3,4)
List of Strings - vector represented by c('a','b','c')
List with multiple data types - vector represented by c(1,'a', TRUE)

Tip #2 - Sequence Vectors
Representing sequence of numbers m to b by m:n
 - seq(10,50)
 - seq(10,50,5) - With increment step 5
 - seq(50,10) - Reverse sequence representation

Tip #3 - Assigning Vectors (Single Quotes)
sentence <- c('walk', 'the', 'plank')
sentence[3]
a <- c (1,2,3)
b <- c (1,2,3)

Sample Vector Operations
a+1
a*2
a+b

x <- seq(1, 20, 0.1)
y <- sin(x)

Tip #4
c for combine vectors
c(1,2,3)

Tip #5
Plotting Vectors
vectorCoordinates <- c(4, 5, 1)
barplot(vectorCoordinates)
barplot(1:100)

x <- seq(1, 20, 0.1)
y <- sin(x)
plot(x,y)

Great Learning Sites
Statsmethod
Code School

Happy Learning!!!

Recommendation Algorithm Analysis

Item to Item Rating based on customer’s purchase of products

The formula for comparison is dot product divided by product of vector lengths
In the example for two sets Book and DVD

• Book – (1,1,1) – Set A consider it as (A1, A2, A3)
• DVD – (1,0,0) – Set B consider it as (B1, B2, B3)

Formula works as

• (A1.B1 + A2.B2 + A3.B3) /sqrt((A1 square + A2 Square + A3 Square)( B1 square + B2 Square + B3 Square))
• (1)/sqrt((3).sqrt(1)
• 1 / 1.732
• 0.577

     Item to Item Comparison based on customer ratings

The formula for comparison is dot product divided by product of vector lengths

In the example for two sets Book and DVD

• Book – (4,3,5) – Set A consider it as (A1, A2, A3)
• DVD – (1,0,0) – Set B consider it as (B1, B2, B3)

Formula works as 

• (A1.B1 + A2.B2 + A3.B3) /sqrt((A1 square + A2 Square + A3 Square)( B1 square + B2 Square + B3 Square))
• (4)/sqrt((16+9+25).sqrt(1)
• 4/7.07
• 0.565

Analysis - By comparing multiple items the items that yield the maximum value would be recommended to the customer

Happy Learning!!!

Inverse Matrix Computation

Matrix A Represented by

Reference - Link

Happy Learning!!!

Matrix and Determinants (Basics)

Back to School and basics. These posts are for my on-line references.

Matrix

• Rectangular Array of Numbers in rows and columns
• Example - [5,2,-3]
• Order of Matrix is represented as Rows X Columns

Types of Matrices

• Row matrix (1 Row, any number of columns)
• Column matrix (1 Column, Multiple Rows)
• Diagonal matrix (Square matrix, Except diagonal elements every other elements are 0)
• Scalar matrix (Square matrix & Diagonal matrix in which all diagonal elements are same)
• Identity matrix - Denoted by I - Diagonal Elements are 1 (Square & Diagonal Matrix)
• Transpose of matrix - Matrix where rows and columns are interchanged

        [ 1, 1, 1 ]

A  = [ 2, 2, 2 ]

        [ 3, 3, 3 ]

Transpose is

A' =   [ 1, 2, 3 ]

          [ 1, 2, 3 ]

         [ 1, 2, 3 ]

Addition of Matrices

• They should be of same order
• Add Corresponding elements

A =   [2,3,5]

         [5,7,-2]

        [5,3,0]

B =    [7,-1,5]

          [0,2,3]

         [7,5,2]

Result ( Add Corresponding elements in same positions)

A + B =     [9,2,10]

         [5,9,1]

               [12,8,2]

Subtraction (similar to addition)

• Order needs to match

A = [7,-2]

       [0,3]

B = [0,2]

[3,5]

A - B = [7,-4]

          [-3,-2]

Scalar Multiplication

• Multiplying constant with a Matrix
• Every Element of Matrix Multipled

c = [3,5]

[-2,-10]

-2c =  [-6,-10]

  [4,20]

Matrix Multiplication

• A (m x n)
• B (n x p)
• Columns in a (n) = Number of Rows in B (n)

A = [3,5]

[7,2]

[2,3]

B = [-2,5]

[3,7]

AB = Operate First Row (Operate) Multiple with First Coulmn

A = [3,5] --->

[7,2]

[2,3]

Select Column 

B = [-2,5] 

[3,7]  

  = [ 3 X -2 + 5 X 3,  3 X 5 + 5 X 7 ]

[ 7 X -2 + 2 X 3,  7 X 5 + 2 X 7 ]

[ 2 X -2 + 3 X 3,  2 X 5 + 3 X 7 ]

 = [9,50]

[-8,49]

[5,31]

Matrix Multiplication is not commutative

Determinant of Square Matrix

+  -

A = [5,6]

[3,-4]

|A| = [Multiply Principal Diagonal Elements ] - [Subtract the Next Diagonal Elements]

= (5 X -4) - (3 X 6)

      = -20 - 18

      = -38

Determinant of 3 X 3 Matrix

        +,-,+

A =  [3,-2,1]

       [2,3,4]

       [2,5,4]

= +3 X [3,4] - (-2) X [2,4] + 1 X [2,3]

                  [5,4]               [2,4]           [2,5]

= 3(12-20) + 2 (8-8) + 1(10-6)

       = -24 + 0 + 4

       = -20

References - Link

Happy Learning!!!