When we have large number of attributes in a dataset, it’s
hard to identify which of the variables are the most useful to use. Singular
Value Decomposition (SVD) algorithm helps us to reduce the dimensions of data.

Here we’ll learn how to implement SVD. But before that, we must know what does
Dimensional Reduction mean?

Dimensional
Reduction is the process of reducing the number of random variables under
consideration as feature selection and feature extraction. There are many ways
to do it. Here we will read about SVD, how it helps in Dimensional Reduction.

Before looking into implementation part, let’s have a brief
overview what SVD is.

Singular Value
Decomposition (SVD) is a matrix factorization method used in data mining.

In data mining, this algorithm is used to reduce the number
of attributes that are used in a data mining process. This reduction removes
unnecessary data that are linearly dependent in the point of view of Linear Algebra.
For example, imagine a database which contains a field that stores the water's
temperature on several samples and another that stores its state (solid, liquid
or gas). It’s easy to see that the second field is dependent from the first
and, therefore, SVD could easily show us that it is not important for the
analysis.

**Algorithm:**

SVD is the factorization of m x n matrix A, with m > n of
real or complex numbers:

A = U S V

^{T}
Where U is an orthogonal m x n matrix, S is a diagonal
matrix of singular values and V is an orthogonal n x n matrix &

U

^{T}U = V^{T}V = I
where I is an identity matrix.

**To compute SVD :**

- we find the eigen vectors and eigen values of A
^{T}A and AA^{T}. - The eigenvectors of A
^{T}A are the columns of V and the eigenvectors of AA^{T}are the columns of U. The singular values of A, in the diagonal of matrix S, are the square root of the common positive eigenvalues of AA^{T}and A^{T}A. - If AA
^{T}and A^{T}A have the same number of eigenvalues, then A is a square matrix, else eigenvalues of the matrix that have less eigen values are the eigenvalues of the matrix that has more. We can say, that singular values of A are the eigen values of the matrix, between AA^{T}and A^{T}A with less number of eigenvalues. - Singular values of matrix A is also known as rank of that matrix that specifies number of linearly independent rows or columns.
- Rank should not be greater than min(m,n)

Now we have clear idea of SVD, so now we’ll learn how to run
SVD in R.

**How to run SVD in R:**

Here is the example for how to run in R.

Suppose we have a 4x3 matrix.
hilbert <-
function(n) { i <- 1:n; 1 / outer(i - 1, i, "+") }

X <- hilbert(4)[, 1:3]

(s <- svd(X))

D <- diag(s$d)

s$u %*% D %* t(s$v)
# X = U D V'

t(s$u) %*% X %*% s$v
# D = U' X V

**d : a vector containing the singular values of x, of length min(n, p).**

**u : a matrix whose columns contain the left singular vectors of x.**

**v : a matrix whose columns contain the right singular vectors of x.**

**s$d**is a vector of singular values as below:

[1] 1.451914187
0.143312317 0.004228883

Using SVD,
we can remove noise and linear independent elements with most important
singular values. This is very useful in data mining.

Excellent blog

ReplyDeletepython interview questions

git interview questions

django interview questions

sap grc interview questions and answers

advanced excel training in bangalore

zend framework interview questions

apache kafka interview questions

uipath training in bangalore

Nice blog

ReplyDeleteuipath training in bangalore

angular4 interview questions

python interview questions

artificial intelligence interview questions

python online training

artificial intelligence online training

I got nice blog

ReplyDeletesap partner companies in bangalore

sap implementation companies in bangalore

sap partners in india

aws staffing

jquery interview questions

sql interview questions

very nice interview questions

ReplyDeletevlsi interview questions

extjs interview questions

laravel interview questions

sap bi/bw interview questions

pcb interview questions

unix shell scripting interview questions

relly awesome blog

ReplyDeletehr interview questions

hibernate interview questions

selenium interview questions

c interview questions c++ interview questions

linux interview questions

useful blog

ReplyDeletespring mvc interview questions

machine learning online training

servlet interview questions

mytectra.in