Cluster settings
You create a Ray cluster with the Cluster Launcher dialog in the EugoHPC Manager. This page goes through every setting in that dialog, in the order the dialog shows them, with what each one does, its default and its limits.
This page covers the settings only. For how your clusters are organized, how a cluster scales and how to connect your session to one, see EugoHPC Manager. For the Python packages your code needs on the cluster's workers, see Packages on the cluster.
Note: Two items depend on your plan. The Enterprise GPUs (ECU-E) worker pool can be selected on the Enterprise plan only; on other plans its card is shown but greyed out, with the hint "Enterprise plan required". The Eugo Runtime Image choice appears only on the Enterprise plan.
The settings at a glanceโ
| Setting | Where in the dialog | Default | Limits |
|---|---|---|---|
| Name | Top of the dialog | hpc-cluster- followed by the date and time (UTC) | 1 to 128 characters: lowercase letters, numbers and hyphens |
| Auto Shutdown After Idle | Idle Shutdown Policy | 30 Minutes (Recommended) | 5 Minutes, 15 Minutes, 30 Minutes, 1 Hour or Never |
| Head node Instance Size | Head Node Configuration | Follows the size of the first worker pool you selected | large to 48xlarge |
| Head node Storage | Head Node Configuration | 128 | 128 to 65,536 (64 TiB) |
| Allow Tasks on Head Node | Head Node Configuration | On | Off requires at least one worker pool |
| Worker types | Worker Pool Configuration | High-Performance CPUs (ECU) | Any mix of the three types |
| Pool Instance Size | Each worker pool | 2xlarge for ECU; for ECU-G, choose a size yourself (the list starts at 2xlarge) | See the size tables below |
| Min Nodes / Max Nodes | Each worker pool | ECU 0 to 20, ECU-G 0 to 4 | Max Nodes up to 100 (ECU) or 20 (ECU-G) |
| Pool Storage | Each worker pool | 128 | 128 to 65,536 (64 TiB) |
| Shutdown After Idle | Each worker pool | The cluster's value | Same choices as the cluster |
| Enable Cluster-Wide Storage | Advanced Configuration | On | Not offered for GB200 clusters, which always include shared storage |
| Eugo Runtime Image | Advanced Configuration (Enterprise plan) | Standard | Standard or Extended (Full Package Set) |
Nameโ
The Name field is required. The dialog fills it in for you with hpc-cluster-YYYYMMDD-HHMMSS, the date and time (UTC) you opened it, and you can replace it with your own name (the field's placeholder is my-cluster).
- Use only lowercase letters, numbers and hyphens. Anything else shows "Name must contain only lowercase letters, numbers, and hyphens" and the cluster cannot be launched until you fix it.
- The name is 1 to 128 characters long.
Idle Shutdown Policyโ
Auto Shutdown After Idle sets how long a node may sit idle before it is released. The choices are:
- 5 Minutes
- 15 Minutes
- 30 Minutes (Recommended), the default
- 1 Hour
- Never (Manual termination only)
A shorter time costs less, because idle nodes are released sooner, but a new job may then have to wait for nodes to start again.
Note: This setting releases idle worker nodes. A worker node that has run nothing for the chosen time is released, down to its pool's Min Nodes. The head node is not released: it, and with it your cluster, keeps running until you terminate the cluster from the EugoHPC Manager (see Cluster lifecycle). With Never, worker nodes are not released for being idle. Terminate a cluster you no longer need.
Each worker pool can override this value with its own Shutdown After Idle setting.
For a cluster with a GB200 pool the setting is fixed and greyed out: the prepaid capacity block runs for its full reserved window whatever the idle time.
Head Node Configurationโ
The head node coordinates the cluster, and by default it runs your tasks too. Its section is collapsed when the dialog opens; click its title to open it.
Node Type, CPU Family and Instance Sizeโ
- Node Type: the head node is always a High-Performance CPU (ECU) node. The GPU types are marked "(workers only)".
- CPU Family: AWS Graviton5.
- Instance Size: from large to 48xlarge, the same sizes as the ECU worker pool. Until you choose one, the head node follows the size of the first worker pool you selected, so it changes when you change that pool's size.
When your cluster includes a GB200 pool, the head node is raised to at least 8xlarge and the whole Head Node Configuration section becomes read-only. Sizes below 8xlarge are marked "(requires non-ECU-E cluster)".
Storageโ
The head node's disk is gp3 (General Purpose SSD) block storage. Enter its Size in the field next to it:
- Default and minimum: 128.
- Maximum: 65,536 (64 TiB).
The field is labelled GB; the numbers are binary gigabytes (GiB).
Allow Tasks on Head Nodeโ
Under Head Node Tasks, Allow Tasks on Head Node is on by default: the head node runs your tasks and actors as well as coordinating the cluster. Turn it off to keep the head node for coordination only; the cluster then needs at least one worker pool. What it does and when to turn it off is covered in Allow Tasks on Head Node. It cannot be changed after launch.
Worker Pool Configurationโ
Choosing the worker typesโ
Each worker type is a card. Click a card to add a pool of that type to your cluster, and click it again to remove the pool. You can combine types in one cluster.
| Card | Hardware | Shown for |
|---|---|---|
| High-Performance CPUs (ECU) | AWS Graviton5 | Data processing |
| General Purpose GPUs (ECU-G) | AWS Graviton2 CPU + NVIDIA Tesla T4G GPU | AI/ML Dev & โค40B Inference |
| Enterprise GPUs (ECU-E) | NVIDIA Grace-Blackwell Superchip (GB200) | AI/ML Training, โค1T Inference, & HPC |
A new cluster starts with the ECU card selected. You can also remove every pool: with Allow Tasks on Head Node on, the head node alone then runs your work. The ECU-E card needs the Enterprise plan and a reserved capacity block; see GB200 capacity blocks.
Each selected card opens its own pool settings, described below.
Instance Sizeโ
Instance Size sets the size of every node in the pool. A larger size has more vCPUs and memory (and, for GPU sizes, more GPUs). Each option reads as a number of units followed by the size: for example 2 ECUs (2xlarge) in an ECU pool and 2 ECU-Gs (2xlarge) in an ECU-G pool. Below the choice, the panel adds up the pool's CPUs, memory, GPUs and GPU memory.
The panel labels memory in GB; the figures below are in binary gigabytes (GiB).
High-Performance CPUs (ECU). The default is 2xlarge.
| Size | ECUs | vCPUs | Memory (GiB) |
|---|---|---|---|
| large | 1/2 | 2 | 8 |
| xlarge | 1 | 4 | 16 |
| 2xlarge | 2 | 8 | 32 |
| 4xlarge | 4 | 16 | 64 |
| 8xlarge | 8 | 32 | 128 |
| 12xlarge | 12 | 48 | 192 |
| 16xlarge | 16 | 64 | 256 |
| 24xlarge | 24 | 96 | 384 |
| 48xlarge | 48 | 192 | 768 |
General Purpose GPUs (ECU-G). Each node has NVIDIA T4G GPUs with 16 GiB of memory each. When you add an ECU-G pool, choose its size yourself from the list, which starts at 2xlarge. Then, in the Review Configuration summary, check the ECU-G pool's size: if it is not the size you chose, go Back, select a different size and then the one you want.
| Size | vCPUs | Memory (GiB) | GPUs | GPU memory (GiB) |
|---|---|---|---|---|
| 2xlarge | 8 | 16 | 1 | 16 |
| 4xlarge | 16 | 32 | 1 | 16 |
| 8xlarge | 32 | 64 | 1 | 16 |
| 16xlarge | 64 | 128 | 2 | 32 |
Enterprise GPUs (ECU-E). A GB200 pool is one whole capacity block, and its size is the size of the block you choose. The figures are for the whole block.
| Size | Nodes | CPUs | Memory (GiB) | GPUs | GPU memory (GiB) |
|---|---|---|---|---|---|
| 9xlarge | 9 | 1,296 | 8,640 | 36 | 6,660 |
| 18xlarge | 18 | 2,592 | 17,280 | 72 | 13,320 |
Scalingโ
Each ECU and ECU-G pool has two numbers, the range Ray's autoscaler may scale the pool between:
| Pool | Min Nodes default | Max Nodes default | Max Nodes limit |
|---|---|---|---|
| ECU | 0 | 20 | 100 |
| ECU-G | 0 | 4 | 20 |
- Min Nodes starts at 0 and can be no more than Max Nodes. Nodes up to this number stay running even when idle.
- Max Nodes is the most nodes the pool can grow to under load.
With a Min Nodes of 0, a pool has no nodes until work arrives, and the first job waits for nodes to start. How the autoscaler grows and shrinks a pool is described in Cluster lifecycle.
A GB200 pool shows Nodes instead: the node count is set by the size of your capacity block, and it cannot be changed or scaled. In a cluster that mixes GB200 with ECU or ECU-G pools, the autoscaler scales only the ECU and ECU-G pools.
Worker storageโ
Each worker node gets its own gp3 (General Purpose SSD) disk. Storage sets its Size for every node in the pool, and the panel shows the pool's total:
- Default and minimum: 128.
- Maximum: 65,536 (64 TiB).
As for the head node, the field is labelled GB and the numbers are GiB.
Shutdown After Idleโ
Each worker pool has its own Shutdown After Idle, with the same choices as the cluster's Auto Shutdown After Idle. It starts at the cluster's value and follows it until you change it for this pool. Once you have, Reset timeouts to whole cluster value puts the pool back on the cluster's value.
For a GB200 pool this setting is read-only. GB200 pools show Nodes, Storage and Shutdown After Idle only once a capacity block is chosen.
GB200 capacity blocksโ
A GB200 pool runs on a capacity block: GB200 capacity you reserve and prepay for a fixed time window. Selecting the Enterprise GPUs (ECU-E) card opens these settings.
Choosing a capacity blockโ
The Capacity block list shows the blocks your organization has reserved, for example 9xlarge ยท 9 nodes / 36 GPUs with its length in hours. Pick the block this cluster should use. If you have none, the list reads "โ no reserved blocks โ reserve new below". A block marked "held for review, contact support" cannot be chosen; contact Eugo support about it.
Reserving a new blockโ
Below the list, the reserve form asks for:
- Instance size:
9xlarge ยท 9 nodes / 36 GPUsor18xlarge ยท 18 nodes / 72 GPUs. - Duration: 1 to 14 days, then whole weeks from 3 weeks up to 26 weeks (182 days). The default is 7 days. Lengths between those steps are not offered.
- Start date: from today up to 8 weeks ahead.
- Get quote shows the windows on offer with their prices. If none is offered, try a later start date or a smaller size.
- Pick a window and tick the box that reads: "I understand this is an irrevocable, non-refundable purchase and authorize the charge. The total shown (block price plus tax) is deducted from your organization's credits immediately at purchase."
- Type
purchase ecu-ein the confirmation field, then click Reserve & Purchase.
A capacity block is a purchase that cannot be undone or refunded, so check the size, duration and start date before you confirm.
Starting ECU and ECU-G workers earlyโ
When you pick a capacity block and the cluster also has ECU or ECU-G pools, one more checkbox appears, off by default:
Start my on-demand (ECU / ECU-G) workers now, before the GB200 block is delivered. They begin billing immediately and cannot run GB200 work until the block arrives. Leave unticked to start everything together on the delivery date.
Leave it off to start the whole cluster together when the block is delivered.
Advanced Configurationโ
Advanced Configuration is collapsed when the dialog opens; click Show to open it.
Cluster-Wide Storageโ
Enable Cluster-Wide Storage is on by default. It creates shared storage (Amazon EFS) with your cluster and mounts it at /mnt/ss on every node, so all nodes see the same files there. The storage is elastic, so there is no size to choose. Terminating the cluster removes this storage with it (see Security).
A cluster with a GB200 pool shows Cluster-Wide Storage (Included) instead, already ticked and not changeable: an FSx for Lustre file system is created with the cluster and mounted at /mnt/ss on every node, the head node and all workers, including any ECU and ECU-G pools. There is no size to choose here either.
Eugo Runtime Imageโ
On the Enterprise plan, a Eugo Dev Tools part appears with the note "Eugo Dev Tools are in Early Access. APIs may change without notice and documentation may be incomplete." Its one setting, Eugo Runtime Image, picks the Eugo image every node runs:
- Standard covers most uses.
- Extended (Full Package Set) carries the full Eugo package set, including build-time dependencies, for advanced development work.
It defaults to the image your own session runs, and to Standard otherwise.
Review and launchโ
Review Configuration shows a summary of everything you chose; Cancel closes the dialog without creating anything. From the summary, Back returns to the settings and Launch Cluster creates the cluster. For a GB200 cluster, the capacity block must be chosen first.
Changing a running clusterโ
On a running cluster's card in the EugoHPC Manager, Scale cluster opens Scale Cluster, followed by the cluster's name. There you can change each worker pool's Min Compute Nodes and Max Compute Nodes, within the same limits as at launch, and save with Apply Changes. The head node, storage, shared storage and the other settings are shown read-only, and a GB200 pool's node count stays fixed by its capacity block. To change anything else, launch a new cluster.
Relatedโ
- EugoHPC Manager: how workspaces and clusters are organized, the cluster lifecycle, Allow Tasks on Head Node, and connecting your session.
- Installing Python packages: what is preinstalled, and how to get extra packages onto the cluster's workers.