In order to implement a learning rate schedule, as described in this paper, I believe I need a way to update the learning rate during training, each batch, by a value calcuated from the label distribution of the true labels in the batch (y_true
as it's typically denoted in keras/tensorflow)
哪里... x模型的输出 y相应的地面真相标签 Βm个样本的小批量(例如64个) ny地面真相标签y的整个训练样本量 ny-1逆标签频率
我关注的公式部分是α和Δθ之间的部分
我可以在自定义损失函数中轻松实现这一目标,但是我不知道如何提高损失率的学习率(如果可以的话)。
def loss(y_true, y_pred):
y = math_ops.argmax(y_true, axis=1)
freqs = tf.gather(lf, y) # equal to lf[y] if `lf` and `y` were numpy array's
inv_freqs = math_ops.pow(freqs, -1)
E = 1 / math_ops.reduce_sum(inv_freqs) # value to use when updating learning rate
where ...
lf the sample frequencies for each class. e.g. 2 classes, c0 = 10 examples, c1 = 100 -->
lf == [10, 100]
有什么花哨的方法可以更新优化器的学习率,例如可以通过CallBack进行哪些操作?
def on_batch_begin(self, batch, log):
# note: batch is just an incremented value to indicate batch index
self.model.optimizer.lr # learning rate, can be modified from callback
在此先感谢您的帮助!
You can use LearningRateScheduler from callbacks to update learning rate during training