# Multilayer Perceptron (MLP)

## Structure

* Multiple linear models (perceptrons);
* TanH activation in hidden layers over sigmoid (logistic);
* Linear (identity) activation for output layer for regression;
* TanH activation for output layer for binary classification;
* Small learning rate can lead to local minimum;
* The higher the dimension of the loss, the lower the chance is to fall into a local minimum;
* SGD will always lead to the global minimum of the loss;
* Initialize weights between -0.01 and 0.01. Small weights with TanH require less iterations;
* High learning rates can lead to NaN values (number greater than bit limits);