Introduction to Machine Learning (part-2)

 Sample is the representative of underlying population. Probability of a sample is a good reflection of probability of population. 

Actual Probability distribution of a sample is hidden, that's why we try to predict the same by making some assumptions.Let us consider Sample as m where m maps to  ∞ (infinity) where P' is the estimated probability, where it may be possible that P'(predicted probability) = P(Actual Probability) for an unbiased sample.

Variables : Variables can be categorical (Nominal) or numeric (continuous). For categorical, it can have n unique values where it's probability can be :   P(x1) =  θ1 , P(X2) =  θ2 ,..........., P(X1k) = θ1k.

  where  θ = x/m where x is number of times the variables occur and m is the size of data set, which is also called Frequentistic Approximation. 

If there are k parameters and let us consider there are k-1 parameters known then to get the probability of last parameter that is k is

                             θ1k = 1-  k-1Σi=1 θ1i

  For continuous : In continuous , we don't follow nominal because due to large dataset, it's probability can be assigned to Zero by x/m,also it include decimals. So to avoid this we follow this normal or gaussian distribution.


It is parametric because it is based on some parameters.
Here,   P(X) = 1/(√2π)*σ[(e-((xi-mean)2)/2σ2)] 

It is the hypothesis space equation of normal distribution. Here we reduced our learning task like estimate joint probability distribution. The parameters considered in this distribution are mean and σ (Standard deviation). To find mean and σ we have to search hypothesis space.



Here each dot in hypothesis space is a model which has different values of mean and standard deviation. For search in the Hypothesis space we use one of the algorithm known as Gradient Descent.

Gradient Descent starts at some random point in hypothesis space and evaluate how good the model is.  Error = observed value - predicted value.

Cost function by sum of squared errors between actual and predicted value.
         Cost fn = 1/m(mΣi=1 (yi-(β1xi+β0)2))

Feature Engineering :  It transform original data to some feature space f1and y to make it linear and make it work.

X,Y can be transformed to √x,3√x,x2,x3,x4,......,Y .
Linear regression : y = β0 + mΣi=1 β1xi

Here , Linear regression means the equation is not linear but here xi  is linear with parameter which help them to fit.

Gaussian distribution can be a mixture of components of Gaussian.

In mixture of gaussian we apply Joint Probability Distribution on various components which combine mixture of gaussians. Continuous distribution can be turned in gaussian mixture distribution. Any probability density function can be approximated by gaussian mixture model.

Maximum Likelihood Principle : It says if we have sample m , it maximizes the probability of generating sample S.
                                    P(Data) = mπi=1(P(xi))

where π mean product.

Comments

Post a Comment

Popular posts from this blog

Model Evaluation and Selection

Image
Model Process :    In decision maker process, we follow f : x  → y and Real world process (RWP) tends to the population. After building a model, we need to know, how accurate the model is expected to be on population. For that, here comes the Confusion Matrix Here we are getting 88 out of 100 as correct, therefore the                        Accuracy = 88/100 = 0.88 we need to check whether the accuracy will be fine for model, for that we check Expected Accuracy.                                E[x] =  Σ x P(x)X                                h(x(vector)) = y (Actual) Goal to be achieved here is    h(x(vector)) = f(x(vector)) for each x belongs to population.   h(x(vector)) = y , E [ h(x(vector)) = f (x(vector)) ] for each x belongs...
Read more

Convolutional Neural Networks(Part-4)

Image
  AlexNet AlexNet is considered to be the first paper/ model which rose the interest in CNNs when it won the ImageNet challenge in 2012. AlexNet is a deep CNN trained on ImageNet and outperformed all the entries that year. It was a  major improvement  with the next best entry getting only 26.2% top 5 test error rate. Compared to modern architectures, a relatively simple layout was used in this paper. ZFNet ZFNet is a modified version of AlexNet which gives a better accuracy.  One major difference in the approaches was that ZFNet used 7x7 sized filters whereas AlexNet used 11x11 filters. The intuition behind this is that by using bigger filters we were losing a lot of pixel information, which we can retain by having smaller filter sizes in the earlier conv layers. The number of filters increase as we go deeper. This network also used ReLUs for their activation and trained using batch stochastic gradient descent. GoogLeNet The GoogLeNet architecture is very different f...
Read more

Introduction to Machine Learning(Part-4)

Image
Training Data is the subset of population, where population can be of n variables x 1 ,x 2 ,........,x n . Let us discuss it by example , assume we are taking to factors to differentiate individuals that is Eye color and Hair color. Eye color can be Blue,Green  and Brown. Hair color can be Black, red, blond and grey. So the total combinations we will be working be 3*4 = 12 Here, we can make prediction of function by Joint Probability Distribution.We assume that eye color and hair color does not tell us much about the individuals.Let us add an attribute Milk allergy which has binary values Yes(y) or No(N), now the combinations on which we can work will be 3*4*2 = 24. If we add an another attribute income which is continuous then the combinations to learn is  3*4*2* ∞ =  ∞. So , there are two ways by which we can learn this problem: Discretize / Bucket : In this we divide the attribute income in interval like: k maps to [1,100k] ,[100k,500k,.......] Probability Density Func...
Read more