On this page · 12 sections
- What actually shipped
- The two-step claim
- The KubeRay version nobody pinned
- Quota is charged per RayCluster, not per job
- What the observability line costs
- Who this affects, and how to tell if it is you
- Access, tokens and the six-hour ceiling
- India-specific considerations
- What is still unknown
- FAQ
- How eCorpIT can help
- References
Summary. Amazon SageMaker HyperPod added managed Ray support on 24 August 2026, and the getting-started page states that a HyperPod cluster on Amazon EKS plus the KubeRay operator are "all you need to run Ray on HyperPod". The four capabilities the announcement leads with need five further installs: the SageMaker AI Spaces add-on, the HyperPod Ray Endpoint Operator, the EKS Pod Identity Agent, the Observability add-on at version 1.0.6 or later, and HyperPod Tiered Storage. The KubeRay install command AWS gives you is unpinned, while the upstream page it links to pins --version 1.6.0, a chart released 19 March 2026 against KubeRay 1.7.0 from 20 August 2026. Dashboard sessions expire after 6 hours, JWT signing keys rotate every 30 minutes, HyperPod runs in 18 AWS Regions, and the Grafana half of the observability story bills separately at $9 per editor and $5 per viewer per month in us-east-1.
What actually shipped
The AWS What's New entry, posted 24 August 2026, describes built-in observability, resilient training, accelerated inference and managed development environments for Ray on HyperPod. Ray is the open-source framework for distributed Python; HyperPod is the managed GPU cluster service that AWS orchestrates through Amazon EKS. The pitch is that a data scientist creates a Ray cluster in Amazon SageMaker Studio, attaches JupyterLab, Code Editor or a local IDE, and iterates against cluster-scale compute without writing Kubernetes manifests.
None of that is wrong. It is the setup cost that the announcement does not carry.
The two-step claim
The HyperPod documentation for Ray opens its getting-started page with a plain sentence: "These two steps are all you need to run Ray on HyperPod." The two steps are creating a HyperPod cluster orchestrated by Amazon EKS, and installing the KubeRay operator on it.
That is true for running a RayCluster with kubectl. It is not true for any of the four things the announcement puts in its first paragraph. Each one has its own prerequisite chain, documented on a different page.
| Announced capability | What the docs also require | Where it is stated |
|---|---|---|
| Attach JupyterLab, Code Editor or a local IDE | SageMaker AI Spaces add-on, a SageMaker AI domain, and 5 named cluster-access policies | Setting up Studio for Ray |
| One-click authenticated Ray Dashboard link | HyperPod Ray Endpoint Operator, which itself requires the Spaces add-on Active with web browser access, plus the Amazon EKS Pod Identity Agent add-on | Installing the HyperPod Ray Endpoint Operator |
| Grafana dashboards with Ray metrics | HyperPod Observability add-on at version 1.0.6 or later, plus a separate dashboard upgrade in the SageMaker AI console | Setting up Ray metrics collection |
| Managed tiered KV cache for Ray Serve | HyperPod Tiered Storage enabled, LMCache with the HyperPod storage backend, and worker-manifest edits for /dev/shm and NODE_IP |
Setting up tiered KV cache |
| Deploy a SageMaker JumpStart model with Ray Serve | The toolkit-for-ray-on-sagemaker-ai Python library, installed with pip |
Accelerated inference |
Five installs, one of which depends on another, and one of which needs a minimum version. Call the setup two steps only if you are running Ray by hand.
The KubeRay version nobody pinned
The install page gives this command:
helm install kuberay-operator kuberay/kuberay-operator
No --version. Helm resolves that to the newest chart in the repository. The same AWS page then links to the KubeRay operator installation page in the Ray documentation "for more information about operator installation options", and that page's recommended command is:
helm install kuberay-operator kuberay/kuberay-operator --version 1.6.0
The two pages disagree in the only place it matters. KubeRay v1.6.0 was published on 19 March 2026. Since then the project has shipped v1.6.1 on 23 April 2026, v1.6.2 on 18 June 2026, and v1.7.0 on 20 August 2026, four days before the HyperPod announcement. A team that copies the AWS command today installs v1.7.0. A team that follows the link AWS gave them installs a chart five months old. Two engineers on the same cluster can end up on different operators, and neither of them chose a version.
Pin it yourself. Decide which KubeRay release your Ray images are tested against, put the version in the Helm command, and record it. AWS states that HyperPod "runs the upstream, open source Ray and KubeRay without changes", so the compatibility question is yours to answer, not the service's.
The upgrade instruction on the same page has the same shape. It tells you to run helm repo update and helm upgrade kuberay-operator kuberay/kuberay-operator, again unpinned, which moves you to whatever the latest chart is on the day you run it.
Quota is charged per RayCluster, not per job
The announcement says task governance "improves compute utilization through quotas, priorities, and preemption", and names the problem it solves as "low GPU utilization from static team allocations".
The quota page is more specific. Task Governance accounts quota at RayCluster granularity. A cluster reserves quota for its full declared size when it is admitted, and holds that quota for its whole lifetime regardless of load. The doc says it directly: "A long-lived cluster that sits idle still counts against your team quota until you delete it."
That is a problem for the workflow the launch leads with. The interactive development experience is a Ray cluster you keep running so you can attach an IDE to it and iterate. Keep it up overnight and it is a static allocation, which is the thing the announcement says HyperPod fixes. The FAQ acknowledges this and offers the fix: size a persistent cluster to the capacity you intend to reserve, and use RayJob for work that should acquire and release capacity.
Two more scheduling behaviours are worth knowing before you plan around this. Admission is gang-scheduled: a RayCluster is not admitted until quota exists for the entire declared size, so an over-sized manifest sits unadmitted rather than starting partially. And preemption deletes the whole cluster, head node included, when higher-priority work arrives. An interactive session preempted mid-afternoon does not shrink; it disappears.
What the observability line costs
"HyperPod provisions Grafana dashboards with metrics in Amazon Managed Service for Prometheus", says the announcement, "giving you visibility into your workloads from the first run". The delivery mechanism is Amazon Managed Grafana and Amazon Managed Service for Prometheus, and both are separately metered services.
Prices below are from the AWS Price List API for us-east-1, retrieved 25 August 2026.
| Meter | Price, us-east-1 | What drives it |
|---|---|---|
| Amazon Managed Grafana editor | $9 per user per month | Anyone who edits a Ray dashboard |
| Amazon Managed Grafana viewer | $5 per user per month | Anyone who only reads them |
| Prometheus metric samples, first 2B | $0.90 per 10M samples above the first 40M | Scrape interval times series count |
| Prometheus metric samples, next 250B | $0.35 per 10M samples | Sustained large-cluster collection |
| Prometheus metric storage | $0.03 per GB-month above 10 GB free | Retention |
| Prometheus query samples | $0.10 per 1B samples processed | Dashboard refreshes and alerts |
A 10-person team on editor seats is $90 a month before a single metric lands. That is not a large number next to a GPU bill, and it is not the point. The point is that "from the first run" describes a newly created cluster, where the console turns Ray metrics on by default. On an existing cluster you install or update the add-on to 1.0.6 or later, then separately upgrade the Grafana dashboards from the SageMaker AI console, and AWS warns that upgrading "overwrites the Ray Core, Ray Data, Ray Train, and Ray Serve dashboards, so copy any customizations first".
Who this affects, and how to tell if it is you
You are affected if you already run HyperPod on Amazon EKS and were planning to turn Ray support on this week. Check three things in this order.
First, run helm list -A and see whether a kuberay-operator release already exists and what chart version it is on. The HyperPod FAQ says you can keep an existing KubeRay installation, so the risk is not that you must reinstall; it is that an unpinned helm upgrade moves it.
Second, check the Observability add-on version in the console. Anything below 1.0.6 runs, and Ray metrics silently do not appear. AWS documents that as the cause of the empty-dashboard symptom.
Third, look at your Task Governance allocations per namespace. A Ray workload submitted to a namespace with no compute allocation is held unadmitted with no pods created, and the symptom looks like a scheduling bug rather than a quota gap.
Two smaller traps are documented in the same troubleshooting page and both cost an afternoon. Head and worker pods stay Pending when requested vCPU or memory exceeds Kubernetes allocatable capacity, which is always lower than the instance nameplate because kube-system reservations and DaemonSets take a share. And ray.init() fails when the Ray version in your client image does not match spec.rayVersion in the cluster manifest.
Access, tokens and the six-hour ceiling
The authenticated dashboard link is not a permanent URL. The HyperPod CLI command hyp create ray-dashboard-connection returns a short-lived presigned URL; the browser exchanges its token for a session cookie valid for up to six hours, after which you generate a new URL. The Endpoint Operator signs those JWTs with keys that rotate every 30 minutes, and AWS recommends configuring an AWS KMS key so that disabling the key revokes all sessions and CloudTrail records each key-generation event. Without a KMS key the rotator generates signing material locally and you lose both the revocation path and the audit trail.
Under the private access strategy the dashboard is scoped to the identity that created the connection, so a colleague opening your link gets an authorization error rather than the dashboard. That is correct behaviour and it will still be filed as a bug by someone on your team in the first week.
India-specific considerations
HyperPod is available in 18 AWS Regions, including ap-south-1 (Mumbai) and ap-south-2 (Hyderabad), so an Indian team can keep training data in-country. The Ray documentation narrows the availability line to "all AWS Regions where SageMaker HyperPod supports Amazon EKS orchestration" and then points at the general HyperPod Region list, which does not break out EKS orchestration separately. Confirm your Region in the console before you plan a migration rather than trusting either sentence.
For teams handling personal data under the Digital Personal Data Protection Act 2023, the KMS decision above is the one to make early. A locally generated signing key gives you no CloudTrail record of key generation, and a dashboard that reaches training data is an access path an auditor will ask about. The pricing above is in US dollars for us-east-1; at roughly Rs 88 to the dollar, ten Grafana editor seats is about Rs 7,900 a month, which is rounding error against a single p5 node and still a line someone has to approve.
What is still unknown
AWS has not published a supported KubeRay version range for HyperPod, only the statement that upstream KubeRay runs unmodified. It has not published a Ray version matrix either; the guidance is to match spec.rayVersion to your image. There is no stated price for the HyperPod Observability add-on itself, and no documented quota for how many RayDashboardConnection resources a namespace may hold. Until those exist, treat version selection and seat count as decisions you own.
FAQ
How eCorpIT can help
Most of the work in this launch is platform work, not model work: pinning an operator version, sizing a persistent cluster against a team quota, deciding who gets a Grafana editor seat, and wiring KMS-backed signing before a dashboard reaches training data. eCorpIT builds and runs that layer for teams on Amazon EKS, and our engineering organisation is CMMI Level 5 and ISO 27001:2022 certified. If you are turning Ray support on this quarter, book a HyperPod platform review and we will go through your add-on versions and quota allocations with you. Teams that need the capacity in-house can also hire AI engineers to own it.
References
- Amazon SageMaker HyperPod enhances support for Ray, AWS What's New, 24 August 2026.
- Ray on SageMaker HyperPod, Amazon SageMaker AI Developer Guide.
- Getting started with Ray on HyperPod, Amazon SageMaker AI Developer Guide.
- Installing KubeRay on HyperPod Amazon EKS, Amazon SageMaker AI Developer Guide.
- KubeRay Operator Installation, Ray documentation.
- KubeRay releases, ray-project/kuberay on GitHub.
- Quota and scheduling behavior for Ray workloads, Amazon SageMaker AI Developer Guide.
- Setting up Ray metrics collection, Amazon SageMaker AI Developer Guide.
- Installing the HyperPod Ray Endpoint Operator, Amazon SageMaker AI Developer Guide.
- Generating a dashboard connection URL, Amazon SageMaker AI Developer Guide.
- Setting up tiered KV cache, Amazon SageMaker AI Developer Guide.
- Troubleshooting Ray on HyperPod, Amazon SageMaker AI Developer Guide.
- Ray on HyperPod FAQs, Amazon SageMaker AI Developer Guide.
- Setting up Studio for Ray, Amazon SageMaker AI Developer Guide.
- AWS Price List API, Amazon Managed Grafana and Amazon Managed Service for Prometheus offer files, retrieved 25 August 2026.
Related reading: AI compute capacity planning, EKS control plane parameters and provisioned lock-in, SageMaker container caching and cold starts, and cloud FinOps for Indian teams.
Last updated: 25 August 2026.
