Introduction to Machine Learning (Part-6)

 Real world process is generated from Population where sample is the sub set of population. Sample can be of two type : 

  • Generative : It says if there are n variables x1,x2,.....,xn and we need to find the probability of these n variables then we apply Joint probability distribution on it as it also follows joint probability distribution then P(x) = P(x1,x2,....,xn)
  • Discriminative : It learns from the conditional probability, which can be defined as  if f : x(vector) maps to y then P(y|x(vector)), where it simply learn by reducing parameters. In hypothesis set, each Hi is not equal to Hj  because each parameter is different in this set. Here, h* = argmin C(y,y'(x(vector))) where y is actual value and y' is the predicted value.
Constraints on  which we get ability to learn are :
  • Hypothesis set chosen
  • Search algorithm
Parameters grown exponentially with respect to number of variables :
  • For binary : 2n-1
  • For non-binary : kn-1
Logistic regression is used when y is categorical whereas  linear regression is used when y is numeric. We can simply learn by reducing parameters by Independent assumptions or Parametric models.

In linear regression, we usually get noise in the measurement.
         y = mx + c + e
where m is the slope, c in the intercept and e  is the unexplained variance.

Bernoulli Experiment : In this , we have only two values , it can be 0 or 1 , yes or no etc...If there are n transaction in an experiment then we predict its value as expected value , where expected value is defined as :
if P(x =1) , then expected value = θx(1-θ)1-x
    P(D) where there are n transactions can be done as :
                          P(D) =nπi=1 P(xi)
                                  = θ#x(1-θ)#(1-x)
                                  = θr(1-θ)n-r
  
θ represents Accuracy

                  if we take log on both sides :
                log(P(D)) = r log θ + (n-r) log θ
This leads to maximum log likelihood

Binomial Distribution : It tells what is the probability of getting correct prediction of n. In this we are interested in set of outcome rather than one outcome. The probability that n Bernoulli trails leads to k states.
      P(y=k|θ)  = θk(1-θ)n-k

Mean = nq  where q = θ
Variance = nq(1-q) where q = θ

Multinomial Distribution :  In this we have categorical trials. The probability is 
 
P(y1,y2,..yk | θ ) = (n!/y1!y2!...yk!) kπi=1 yiθi 

n = kΣi=1 yk



Comments

Post a Comment

Popular posts from this blog

Supervised Learning(Part-5)

Image
  Decision Trees (Continued) Attribute Selection Measures If the dataset consists of N attributes then deciding which attribute to place at the root or at different levels of the tree as internal nodes is a complicated step. By just randomly selecting any node to be the root can’t solve the issue. If we follow a random approach, it may give us bad results with low accuracy. For solving this attribute selection problem, researchers worked and devised some solutions. They suggested using some criteria like : Entropy, Information gain, Gini index, Gain Ratio, Reduction in Variance Chi-Square These criteria will calculate values for every attribute. The values are sorted, and attributes are placed in the tree by following the order i.e, the attribute with a high value(in case of information gain) is placed at the root. While using Information Gain as a criterion, we assume attributes to be categorical, and for the Gini index, attributes are assumed to be continuous. Entropy Entropy is a me
Read more

Supervised Learning(Part-2)

Image
  Support Vector Machine A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data, the algorithm outputs an optimal hyperplane which categorizes new examples.  In two dimensional space this hyperplane is a line dividing a plane in two parts where in each class lay in either side. Suppose given plot of two label classes on graph as shown in image. You might have come up with something similar to following image (image B). It fairly separates the two classes. Any point that is left of line falls into black circle class and on right falls into blue square   In real world application, finding perfect class for millions of training data set takes lot of time. This is called regularization parameter.  Two terms regularization parameter and gamma are tuning parameters in SVM classifier. Varying those we can achieve considerable non linear classification line with more accuracy in reasonable amount of
Read more

Text Analysis (Part - 4)

Image
  The article covers a very important concepts of Text Analysis from Bayesian's i.e the  Latent Dirichlet Allocation  which is the part of topic modelling . What is Topic Modelling? Topic modelling is a type of statistical modelling for discovering the abstract “topics” that occur in a collection of documents. This can be useful for search engines, customer service automation, and any other instance where knowing the topics of documents is important . Thus, Topic Models , in a nutshell, are a type of statistical language models used for uncovering hidden structure in a collection of texts. There are multiple methods of for topic modelling and one of them includes Latent Dirichlet Allocation (LDA) . Latent Dirichlet Allocation Latent Dirichlet Allocation (LDA) is an example of topic model and is used to classify text in a document to a particular topic. It builds a topic per document model and words per topic model, modelled as Dirichlet distributions. Basic Idea LDA is a  genera
Read more