Introduction to Machine Learning (Part-3)

 Probability is a way to model uncertain events and Machine Learning is trying to make a process similar to real world process.


It is a basic model using machine learning.

It says if we have some missing data then through function we can retrieve the missing data.

Example : Apply for a bank loan is a real world process, it can be based on Age, income, Education, Marriage status, Defaulted ...etc. Here we have to apply Joint Probability Distribution. Here from function we predict probability to approximate actual probability.

Principle Components : These are the factors of the process , on which a process is based on.

Logistic Regression : It  is used where the scenario is that input is continuous variable and output is categorical variable.It can be represented as :

                      P(Y=1|X) = 1 / 1 + e(2Σi=1βixi)

Here βi  is the parameter. We have to learn the function which minimizes the error.

                   P(Y=0|X) = 1 - P(Y=1|X)                                                                                                             =  1 -  1 / 1 + e(2Σi=1βixi)

If we assume P(Y=1|X) > P(Y=0|X), here we can bet on 1 because here P(Y=1|X) is greater. If  P(Y=1|X) = P(Y=0|X) , then it will lead to decision boundary and this the proof that Logistic Regression can create a Linear decision boundary.

If we substitute the values in P(Y=1|X) = P(Y=0|X)

                               1 / 1 + e(2Σi=1βixi) =    e-(2Σi=1βixi) /  1 + e(2Σi=1βixi)

                                                        1 =   e(2Σi=1βixi)

                                                   By taking log on both sides

                                                 log 1 = log e(-Σβixi)

                                                  c = Σβixi

                                                  Σβixi = 0  

This is equation of plane.

In logistic regression also we try to fit linear decision boundary.

                                D maps P'(Y|X)

where here D is discriminative model and P'(Y|X) is conditional probability of Y given X, if it would be P'(XY) then it would have been generative model.

                                   P(Y|X) = P(XY)/P(X)

We get X by marginalization P(X) =  ΣY P(XY)

In Linear regression, input as well as output both are continuous.

Linear and Logistic Regression are not universal approximate , we cannot find f through them but can find the approximate by Mixture of Gaussian and Neural Network and can help to find approximate function.

 




Comments

Post a Comment

Popular posts from this blog

Supervised Learning(Part-5)

Image
  Decision Trees (Continued) Attribute Selection Measures If the dataset consists of N attributes then deciding which attribute to place at the root or at different levels of the tree as internal nodes is a complicated step. By just randomly selecting any node to be the root can’t solve the issue. If we follow a random approach, it may give us bad results with low accuracy. For solving this attribute selection problem, researchers worked and devised some solutions. They suggested using some criteria like : Entropy, Information gain, Gini index, Gain Ratio, Reduction in Variance Chi-Square These criteria will calculate values for every attribute. The values are sorted, and attributes are placed in the tree by following the order i.e, the attribute with a high value(in case of information gain) is placed at the root. While using Information Gain as a criterion, we assume attributes to be categorical, and for the Gini index, attributes are assumed to be continuous. Entropy Entropy is a me
Read more

Supervised Learning(Part-2)

Image
  Support Vector Machine A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data, the algorithm outputs an optimal hyperplane which categorizes new examples.  In two dimensional space this hyperplane is a line dividing a plane in two parts where in each class lay in either side. Suppose given plot of two label classes on graph as shown in image. You might have come up with something similar to following image (image B). It fairly separates the two classes. Any point that is left of line falls into black circle class and on right falls into blue square   In real world application, finding perfect class for millions of training data set takes lot of time. This is called regularization parameter.  Two terms regularization parameter and gamma are tuning parameters in SVM classifier. Varying those we can achieve considerable non linear classification line with more accuracy in reasonable amount of
Read more

Convolutional Neural Networks(Part-2)

Image
  Cross- Correlation Cross-correlation  is a  measure of similarity  of two series as a function of the displacement of one relative to the other. This is also known as a  sliding  dot product  or  sliding inner-product . It is commonly used for searching a long signal for a shorter, known feature. It has applications in  pattern recognition ,  single particle analysis ,  electron tomography ,  averaging ,  cryptanalysis , and  neurophysiology .  Steps followed in Cross- Correlation Take two matrices with same dimensions. Multiply them one by one, element by element (i.e , not the dot product, just the simple multiplication). Sum the elements together. VGG16 VGG16 is a convolutional neural network model proposed by K. Simonyan and A. Zisserman from the University of Oxford in the paper “Very Deep Convolutional Networks for Large-Scale Image Recognition”.  The model achieves 92.7% top-5 test accuracy in ImageNet,  which is a dataset of over 14 million images belonging to 1000 classes. A
Read more