Classification, Regression, Error Functions and Optimization

In this lecture, the following concepts were introduced.

An overview of the idea of classification. Including
- Understanding a basic classification algorithm like the perceptron algorithm
- Understanding what a feature matrix is.
- Understand what the data labels are.
- The concept of a learning rate
- The concept of linear separability
An overview of the idea of regression. Including
- Basis functions can be used to make a linear regression non-linear.
- An example of a commonly used basis set (like polynomials or radial basis functions).
- A commonly used error (or objective) function such as the sum of squared errors.
The difference between a model and an algorithm
The concept of generalization
The idea of a training set
The use of the error function (also known as an objective function)
The importance of the mathematical concepts of
- vectors
- differentiation
- minimum
The idea behind the optimization approach of steepest descent.
How stochastic gradient descent differs from steepest descent and why this is important.