Normalization

Normalization is the process of transforming data so that it fits within a specific range or format.

Max-min normalization $x_{normalized}=\frac{x-\min (x)}{\max (x)-\min (x)}$

• This scales everything between $[0,1]$

This does not guarantee \sum x = 1

This normalization ensures that all values are between 0 and 1, but it doesn’t ensure that the sum of the normalized values is 1. This is needed when we want a Probability Distribution over our data.

• For that, use a Softmax

Also see Standardization.

Other Ideas of normalization

There is a simpler way to normalize, which is $x=\frac{x_i}{\sum x_i}$

Danger

This doesn’t handle well mixed positive and negative values. Consider X = [-0.3, 0.3]. The sum would 0, resulting in division by 0. Use Softmax!

I have also seen from here $x_{normalized}=\frac{x}{\max (x)}$

Multinomial Distribution → Multinomial probability distribution