Multiple Optimality Guarantees In Statistical Learning