Mixed Precision This is pretty cool, where they mix FP16 and FP32 depending on the needs. Resources https://arxiv.org/abs/1710.03740 https://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html https://www.tensorflow.org/guide/mixed_precision