Weight Decay Decay is just a general idea, there is also learning rate decay for Neural Network training, see Learning Rate