Statistical learning Week 1

I recently started the course 'Statistical learning' by Rob Tibshirani and Trevor Hastie. This course is based on the book 'Introduction to Statistical Learning' (2nd ed) and the book is available online for free. Following are the notes from the first week of this course -

Examples of Statistical Learning -

• Identifying the risk factors for prostate cancer
• Predicting whether someone has a heart attack based on demographic features
• identifying the numbers in a handwritten zip code

**Supervised learning -**

The training data has 2 components: X (features) and y (outcome). 'X' represents features a.k.a. inputs, regressors, covariates and independent variables. The outcome variable 'y' is also known as dependent variable, response and target. Every observation of the traning data is represented by (x₁, y₁) ... (x_N, y_N).

The two types of supervised learning problems are -

1. Classification problem - Predicting whether the patient died or survived, whether the patient has malignant or benign cancer
2. Regression Problem - Estimating the price of a house given demographic information

Ojectives of supervised learning -

• To accurately predict y for unseen test cases
• To understand which features affect y and how
• To assess the quality of the predictions and inferences

**Unsupervised learning -**

The data is unlabelled for unsupervised learning. The purpose of the model is to assess the samples and group them based on common features.

An example of unsupervised learning is the netflix prize competition. Participants were provided with a matrix of 18,000 movies and 400,000 customers, each rated between 1 and 5. The data was very sparse - 98% of the data being missing! The objective was to predict the rating for 1 million customer movie pairs that were missing in the training data. This is a typical recommender system problem.

In unsupervised learning, it is hard to know how well the model is performing. There is no gold standard. For example, an algorithm can classify the positive vs. negative reviews for movies. But it takes human intervention to actually label the group containing positive and negative reviews. It is more expensive to label the data, but it is much easier to collect unlabelled data.

Statistical learning vs. machine learning -

There is a high overlap between these 2 terms. But machine learning is concerned more about pure prediction, whereas statistical learning, while also being concerned about prediction, also tries to come up with methods that can be interpreted by scientists more easily.