1---
2title: HyperPod
3description: Provision SageMaker HyperPod clusters — Slurm or EKS orchestrated — and run ML workloads on them with sbatch, raw manifests, or effectful Jobs with task governance.
4---
5
6**SageMaker HyperPod** is AWS's persistent fleet for ML training
7and inference: a cluster of accelerated instances with automatic
8health checks, faulty-node replacement, and deep health checks
9for GPUs. Where ECS and EKS run application containers, HyperPod
10runs distributed training — and stays up between jobs.
11
12The HyperPod primitives:
13
14- A **Cluster** is the fleet. It is orchestrated by **Slurm**
15 (the default) or by an **EKS** cluster you attach it to.
16- An **instance group** names a set of identical instances — a
17 controller group, a worker group — each with an instance type,
18 count, execution role, and lifecycle script.
19- A **lifecycle script** (`on_create.sh` in S3) runs on every
20 node when it boots. This is where real clusters install
21 schedulers, mount FSx, and wire observability.
22- **Task governance** (EKS orchestration only) arbitrates the
23 fleet between teams: a scheduler policy sets priority classes,
24 and per-team compute quotas reserve capacity.
25
26Alchemy models these with
27[`Cluster`](/providers/aws/sagemaker/cluster),
28[`ClusterSchedulerConfig`](/providers/aws/sagemaker/clusterschedulerconfig),
29and [`ComputeQuota`](/providers/aws/sagemaker/computequota), and
30Kubernetes workloads opt onto HyperPod nodes by referencing their
31attributes (the group's `nodeSelector`, the quota's governed
32`namespace` and `queueName`) from plain
33[`Kubernetes.Deployment`](/providers/kubernetes/deployment) and
34[`Kubernetes.Job`](/providers/kubernetes/job) props.
35
36## Choose an orchestrator
37
38| | Slurm (default) | EKS (`orchestrator: { Eks }`) |
39| --------------- | ------------------------------------- | ---------------------------------------------------- |
40| Workloads | `sbatch` on the login node (over SSM) | Kubernetes objects — `Deployment`, `Job`, `Manifest` |
41| High-level DX | — | `Kubernetes.Job` / `Deployment` via HyperPod attributes |
42| Governance | Slurm accounting | `ClusterSchedulerConfig` + `ComputeQuota` (Kueue) |
43| Extra plumbing | Lifecycle bucket + script | EKS cluster + the HyperPod dependencies Helm chart |
44
45Pick Slurm for classic HPC-style training queues. Pick EKS when
46your platform is Kubernetes or you want typed workloads and task
47governance.
48
49## Create a Slurm cluster
50
51A Slurm cluster needs three things: an S3 bucket holding the
52lifecycle script, an execution role the nodes assume, and the
53cluster itself. The script must exist in S3 before the cluster
54creates — a deploy-time `Action` uploads it, and the data flow
55(bucket → script → cluster) orders the deploy:
56
57```typescript
58const bucket = yield* AWS.S3.Bucket("LifecycleScripts", {
59 forceDestroy: true,
60});
61
62const script = yield* UploadLifecycleScript({
63 bucketName: bucket.bucketName,
64});
65
66const role = yield* AWS.IAM.Role("HyperPodInstanceRole", {
67 assumeRolePolicyDocument: {
68 Version: "2012-10-17",
69 Statement: [
70 {
71 Effect: "Allow",
72 Principal: { Service: "sagemaker.amazonaws.com" },
73 Action: ["sts:AssumeRole"],
74 },
75 ],
76 },
77 managedPolicyArns: [
78 "arn:aws:iam::aws:policy/AmazonSageMakerClusterInstanceRolePolicy",
79 ],
80});
81
82const cluster = yield* AWS.SageMaker.Cluster("TrainingCluster", {
83 instanceGroups: {
84 controller: {
85 InstanceType: "ml.t3.medium",
86 InstanceCount: 1,
87 ExecutionRole: role.roleArn,
88 LifeCycleConfig: {
89 SourceS3Uri: script.sourceS3Uri,
90 OnCreate: script.onCreate,
91 },
92 },
93 },
94});
95```
96
97Instance groups reconcile in place: change a count or add a group
98and the cluster updates; remove a group from `instanceGroups` and
99it is deleted from the cluster. Changing the VPC replaces the
100cluster. Provisioning takes ~5 minutes for small CPU groups and
10110–25 minutes for large GPU groups.
102
103The full working stack is
104[`examples/aws-hyperpod`](https://github.com/alchemy-run/alchemy/tree/main/examples/aws-hyperpod).
105
106## Submit Slurm jobs
107
108Slurm has no remote submission API — jobs are submitted on the
109cluster. Every node is an SSM target named
110`sagemaker-cluster:<cluster-id>_<instance-group>-<instance-id>`:
111
112```sh
113aws sagemaker list-cluster-nodes --cluster-name <clusterName>
114aws ssm start-session \
115 --target sagemaker-cluster:<cluster-id>_controller-<instance-id>
116# then, on the node:
117sbatch --nodes=4 train.sbatch
118```
119
120## Create an EKS-orchestrated cluster
121
122Under EKS orchestration, HyperPod nodes join an EKS cluster you
123provide and workloads are ordinary Kubernetes objects. The API
124enforces a few constraints, all encoded in the example:
125
126- The EKS cluster must use the **`API`** (or `API_AND_CONFIG_MAP`)
127 authentication mode — EKS's own CONFIG_MAP default is rejected.
128- HyperPod trails the newest Kubernetes version; pin one it
129 supports (1.28–1.35 today).
130- `LifeCycleConfig` is required for EKS-orchestrated instance
131 groups too.
132- The **HyperPod dependencies Helm chart**
133 ([aws/sagemaker-hyperpod-cli](https://github.com/aws/sagemaker-hyperpod-cli))
134 must be installed on the EKS cluster before the HyperPod
135 cluster attaches — SageMaker validates it. The example fetches
136 the chart with an `Action` and applies it with
137 [`Kubernetes.HelmChart`](/providers/kubernetes/helmchart).
138
139```typescript
140const eks = yield* AWS.EKS.Cluster("Orchestrator", {
141 roleArn: eksRole.roleArn,
142 version: "1.34",
143 resourcesVpcConfig: { subnetIds: network.privateSubnetIds },
144 accessConfig: {
145 authenticationMode: "API",
146 bootstrapClusterCreatorAdminPermissions: true,
147 },
148});
149
150const hyperpod = yield* AWS.SageMaker.Cluster("HyperPod", {
151 orchestrator: { Eks: { ClusterArn: eks.clusterArn } },
152 vpcConfig: {
153 SecurityGroupIds: [clusterSecurityGroupId],
154 Subnets: network.privateSubnetIds,
155 },
156 instanceGroups: {
157 workers: {
158 InstanceType: "ml.t3.medium",
159 InstanceCount: 1,
160 ExecutionRole: role.roleArn,
161 LifeCycleConfig: {
162 SourceS3Uri: script.sourceS3Uri,
163 OnCreate: script.onCreate,
164 },
165 },
166 },
167});
168```
169
170HyperPod nodes must live in **private subnets** — the
171[`Network`](/providers/aws/ec2/network) helper with `nat:
172"single"` builds a suitable VPC.
173
174## Run workloads on HyperPod nodes
175
176HyperPod nodes are ordinary EKS nodes carrying well-known labels.
177[`Kubernetes.Deployment`](/providers/kubernetes/deployment) and
178[`Kubernetes.Job`](/providers/kubernetes/job) opt onto them with
179each instance group's `nodeSelector` attribute — health-checked
180nodes of that group. The instance-group keys carry through to the
181cluster's attributes as types, so the reference is typed per key
182(a typo'd group name is a compile error) and the workload is
183connected to the fleet through the resource graph:
184
185```typescript
186const train = yield* Kubernetes.Job(
187 "Train",
188 {
189 cluster: eks,
190 main: import.meta.url,
191 podTemplate: {
192 spec: {
193 nodeSelector: hyperpod.instanceGroups.workers.nodeSelector,
194 },
195 },
196 },
197 Effect.gen(function* () {
198 return {
199 run: Effect.gen(function* () {
200 // training / eval logic; bindings land IAM on pod identity
201 }),
202 };
203 }),
204);
205```
206
207For anything the typed surface doesn't cover — a Kubeflow
208`PyTorchJob`, a custom operator — apply a raw
209[`Kubernetes.Manifest`](/providers/kubernetes/manifest) pinned by
210the node labels directly:
211
212```typescript
213nodeSelector: {
214 "sagemaker.amazonaws.com/node-health-status": "Schedulable",
215 "sagemaker.amazonaws.com/instance-group-name": "workers",
216},
217```
218
219## Task governance
220
221Task governance arbitrates the fleet between teams. It ships as
222the `amazon-sagemaker-hyperpod-taskgovernance` EKS add-on (Kueue
223under the hood), a **scheduler policy** with priority classes,
224and per-team **compute quotas**:
225
226```typescript
227yield* AWS.EKS.Addon("TaskGovernance", {
228 clusterName: eks.clusterName,
229 addonName: "amazon-sagemaker-hyperpod-taskgovernance",
230});
231
232// One policy per cluster — a second create fails with the typed
233// ClusterSchedulerConfigAlreadyExists error.
234const policy = yield* AWS.SageMaker.ClusterSchedulerConfig("Scheduler", {
235 clusterArn: hyperpod.clusterArn,
236 schedulerConfig: {
237 PriorityClasses: [
238 { Name: "inference", Weight: 100 },
239 { Name: "training", Weight: 75 },
240 ],
241 FairShare: "Enabled",
242 },
243});
244
245// Creates the hyperpod-ns-research namespace + its Kueue queue.
246const quota = yield* AWS.SageMaker.ComputeQuota("ResearchQuota", {
247 clusterArn: hyperpod.clusterArn,
248 computeQuotaTarget: { TeamName: "research", FairShareWeight: 10 },
249 computeQuotaConfig: {
250 ComputeQuotaResources: [{ InstanceType: "ml.t3.medium", Count: 1 }],
251 },
252});
253```
254
255A workload submits through governance by referencing the quota's
256attributes — the governed namespace and the team's Kueue queue —
257which also orders the workload after the quota:
258
259```typescript
260namespace: quota.namespace, // hyperpod-ns-research
261labels: {
262 [AWS.SageMaker.KUEUE_QUEUE_NAME_LABEL]: quota.queueName,
263 [AWS.SageMaker.KUEUE_PRIORITY_CLASS_LABEL]: "training-priority",
264},
265```
266
267## Where next
268
269- [`examples/aws-hyperpod`](https://github.com/alchemy-run/alchemy/tree/main/examples/aws-hyperpod)
270 — both orchestrators, every workload tier, task governance
271- [EKS](/aws/compute/eks) — the Kubernetes surface HyperPod
272 workloads ride on
273- [`Cluster`](/providers/aws/sagemaker/cluster),
274 [`ClusterSchedulerConfig`](/providers/aws/sagemaker/clusterschedulerconfig),
275 [`ComputeQuota`](/providers/aws/sagemaker/computequota) — API
276 reference