We introduce Accumulative Regularization (AR), a unified high-order framework that closes the gap between fast function-value residual convergence and slow gradient norm convergence in convex optimization. AR achieves optimal gradient norm rates for composite convex problems and extends to uniformly convex settings with parameter-free, inexact algorithms.