Data labelling requires raters to make binary decisions when things are often not that simple. What can experiments tell us about the annotation process?
We propose the asymmetric certified robustness problem, which requires certified robustness for only one class and reflects real-world adversarial scenarios.