To gradient or not to gradient?