A regression is a statistical technique that relates a dependent variable to one or more independent (explanatory) variables. You predict a continuous value, rather than discrete classes such as in Classification.

We can use L1 and L2 distance to solve regression problems. In Stanford CS231n, it was a Classification problem, where you used the SVM loss to predict the class.

Linear Regression

Method 1: Using MLE

Simple linear regression model: Alternate Formulation

We use data to estimate

The Likelihood Function is given by ?? i am too lazy to put this

We come up with the line of best fit using MLEs. We get the following results (derivation is at page 402) for the estimates of the parameters :

The line of best fit is given by

has no predictive power for .

Method 2: Least Squares

I don’t think the teacher went too in depth for this… They both end up with the same final equation.

We are making the Gauss-Markov Theorem

We want to ask if if then has no predictive power for Suppose H_0\beta = 0H_1\beta \neq 0$

You do hypothesis testing, where it is given by