Vectorization

A big part of the CS231N course (actually idk, for the beginning for sure) was practicing optimizing by vectorizing all of the operations.

The term vectorization describes the use of optimized, pre-compiled code written in a low-level language (e.g. C) to perform mathematical operations over a sequence of data. This is done in place of an explicit iteration written in the native language code (e.g. a “for-loop” written in Python).

There is a very BIG speedup when you vectorize operations.

I really sucked at these vectorized operations. Need more practice.

To see how Vectorization is implemented, see the Vectorization is implemented, see the NumPy page.

Sometimes, you can’t vectorize

For example, when you do the for loops, because the updates are dependent on each other, you can vectorize it:

• You can’t vectorize the for loop, because each iteration is updating state. Vectorization is primarily used when the calculation can be done such that each iteration is calculating an independent (in some sense) result.

For example, if you want to do Value Iteration on a particular state, you need to do the update.

It’s the same idea on why you can’t do full Vectorization for Dynamic Programming, because present values depend on past values between each loop.