Data Science

  Regularization : Regularization of a model can be done by avoiding overfitting in the model. It adds a penalty term in the cost function based on the parameters. Cost func   θ* = arg θ max logP(S;θ)= Σ m j=1 logP(y (f) |x j ;θ)-λΣ n i=1 β i 2 λ is a constant (Hyper-Parameter) that determine the strength of the penalty term. For Linear Regression :                     θ* = arg θ min =  Σ m i=1 (y i -y i ') 2 +λΣ n j=1 β j 2 Way to minimize this is by minimizing individual terms. Here, for all j = 1 ,  β = 0. In linear regression using L 2 penalty term Σ n j=1 β j 2 results in Ridge regression and using L 1 penalty term Σ n i=1 |β j | results in Lasso Regression. In Linear regression remove co related independent variables . Overfitting :  Try to keep the model simple by damping  β to avoid complexity. Samples and Estimation :  Sample is the subset of Population, training data is used to...

  Probabilistic Inference :  The computation from observed evidence, of posterior probabilities for query  Propositions. It leads to the Joint Probability Distribution.                If P(A|B) = P(B|A)*P(A) /P(B)  so here P(B|A) is the evidence , P(A) is the prior probability and P(B) is the posterior Probability. General Inference Procedure :  Let X be the query value which is a dependent variable. Let E be the evidence variables and e be the observed values and is specific for them. Let Y be the unobserved Variables.     P(X|e) =  α P(X,e) =  α  Σ y P(X,e,Y) where  α is the normalization constant. Bayesian  Belief Network :   It is a probabilistic Graphical Model. It follows casuality, means   there will be the reason for something.Way to reduce its parameters is by making sum of some independent variables. Product Rule : Applicable when there are two variables pres...

 Real world process is generated from Population where sample is the sub set of population. Sample can be of two type :  Generative : It says if there are n variables x 1 ,x 2 ,.....,x n and we need to find the probability of these n variables then we apply Joint probability distribution on it as it also follows joint probability distribution then P(x) = P(x 1 ,x 2 ,....,x n ) Discriminative : It learns from the conditional probability, which can be defined as  if f : x(vector) maps to y then P(y|x(vector)), where it simply learn by reducing parameters. In hypothesis set, each H i is not equal to H j   because each parameter is different in this set. Here, h* = argmin C(y,y'(x(vector))) where y is actual value and y' is the predicted value. Constraints on  which we get ability to learn are : Hypothesis set chosen Search algorithm Parameters grown exponentially with respect to number of variables : For binary : 2 n -1 For non-binary : k n -1 Logistic regressio...

Binomial Distribution :                                       As the name suggests , in this distribution only two outcomes are possible like 0 or 1 , Success or Failure , Yes or No etc. These type of distributions are known as binomial distribution. It is applicable when an event is occurring multiple times for example we toss coin 20 times, here the possibility of outcomes of coin is either head or tail and it occurs 20 times so here, we will use binomial distribution.  Bernoulli Distribution  is also same as that of binomial distribution, it also has two outcomes possible. Multinomial Distribution :                                         In this probability distribution, we get the outcomes from the multiple experiments for the occurrence of the particular event. As the name sugges...

Probability :  Probability starts with a sample space that describes possible outcomes in an experiment. Probability consists of sample space which contain all the elements and from these elements we select the outcome required for the experiment to get the probability. Examples :  1. Coin Tossing :                                    S = {H,T} it gives us idea for the one toss where S is the Sample space and {H,T} are the outcomes. If the coin is tossed twice the possible sample  space can be :                               S = {HH,HT,TH,TT} For multiple tosses it could be :                              S = {HH .........H, HH........HT , HH.........HTH,.........,T......T} 2. Roll Die :               ...