SageMaker container caching cut GenAI scale-out latency 51% (2026 guide)
AWS's June 2026 container caching removes the ECR image pull from SageMaker scale-out. In AWS's own test a GenAI endpoint's cold start fell from 525 to 258 seconds. The benchmarks and when to use it.
