DeepMind recently introduced DataRater, a meta-learning approach that rates the value of training data points using meta-gradients. It promises more efficient training by filtering/re-weighting low-value data instead of relying on heuristics.

Sharing an implementation of the DataRater where I tried to reproduce the results on MNIST!

This result is representative of the idea where the DataRater learned how to down-weight corrupted images: https://github.com/rishabhranawat/DataRater/blob/main/mnist_...

The inspiration was Figure 3 in the paper.

Paper link: https://arxiv.org/abs/2505.17895 Authors: Calian et. al., Google Deepmind