3 articles tagged
CPU-based autoscaling scales GPU inference too late. A 2026 config guide to HPA, KEDA, external metrics and Karpenter, with YAML for queue-depth and scale-to-zero scaling of LLM inference on Kubernetes.
AWS's June 2026 container caching removes the ECR image pull from SageMaker scale-out. In AWS's own test a GenAI endpoint's cold start fell from 525 to 258 seconds. The benchmarks and when to use it.
In-place pod resize graduated to stable in Kubernetes v1.35, and pod-level resize reached beta in v1.36. What changed, what still forces a restart, and how to wire it to the Vertical Pod Autoscaler.
