Skip to main content

Cluster settings

You create a Ray cluster with the Cluster Launcher dialog in the EugoHPC Manager. This page goes through every setting in that dialog, in the order the dialog shows them, with what each one does, its default and its limits.

This page covers the settings only. For how your clusters are organized, how a cluster scales and how to connect your session to one, see EugoHPC Manager. For the Python packages your code needs on the cluster's workers, see Packages on the cluster.

Note: Two items depend on your plan. The Enterprise GPUs (ECU-E) worker pool can be selected on the Enterprise plan only; on other plans its card is shown but greyed out, with the hint "Enterprise plan required". The Eugo Runtime Image choice appears only on the Enterprise plan.

The settings at a glanceโ€‹

SettingWhere in the dialogDefaultLimits
NameTop of the dialoghpc-cluster- followed by the date and time (UTC)1 to 128 characters: lowercase letters, numbers and hyphens
Auto Shutdown After IdleIdle Shutdown Policy30 Minutes (Recommended)5 Minutes, 15 Minutes, 30 Minutes, 1 Hour or Never
Head node Instance SizeHead Node ConfigurationFollows the size of the first worker pool you selectedlarge to 48xlarge
Head node StorageHead Node Configuration128128 to 65,536 (64 TiB)
Allow Tasks on Head NodeHead Node ConfigurationOnOff requires at least one worker pool
Worker typesWorker Pool ConfigurationHigh-Performance CPUs (ECU)Any mix of the three types
Pool Instance SizeEach worker pool2xlarge for ECU; for ECU-G, choose a size yourself (the list starts at 2xlarge)See the size tables below
Min Nodes / Max NodesEach worker poolECU 0 to 20, ECU-G 0 to 4Max Nodes up to 100 (ECU) or 20 (ECU-G)
Pool StorageEach worker pool128128 to 65,536 (64 TiB)
Shutdown After IdleEach worker poolThe cluster's valueSame choices as the cluster
Enable Cluster-Wide StorageAdvanced ConfigurationOnNot offered for GB200 clusters, which always include shared storage
Eugo Runtime ImageAdvanced Configuration (Enterprise plan)StandardStandard or Extended (Full Package Set)

Nameโ€‹

The Name field is required. The dialog fills it in for you with hpc-cluster-YYYYMMDD-HHMMSS, the date and time (UTC) you opened it, and you can replace it with your own name (the field's placeholder is my-cluster).

  • Use only lowercase letters, numbers and hyphens. Anything else shows "Name must contain only lowercase letters, numbers, and hyphens" and the cluster cannot be launched until you fix it.
  • The name is 1 to 128 characters long.

Idle Shutdown Policyโ€‹

Auto Shutdown After Idle sets how long a node may sit idle before it is released. The choices are:

  • 5 Minutes
  • 15 Minutes
  • 30 Minutes (Recommended), the default
  • 1 Hour
  • Never (Manual termination only)

A shorter time costs less, because idle nodes are released sooner, but a new job may then have to wait for nodes to start again.

Note: This setting releases idle worker nodes. A worker node that has run nothing for the chosen time is released, down to its pool's Min Nodes. The head node is not released: it, and with it your cluster, keeps running until you terminate the cluster from the EugoHPC Manager (see Cluster lifecycle). With Never, worker nodes are not released for being idle. Terminate a cluster you no longer need.

Each worker pool can override this value with its own Shutdown After Idle setting.

For a cluster with a GB200 pool the setting is fixed and greyed out: the prepaid capacity block runs for its full reserved window whatever the idle time.

Head Node Configurationโ€‹

The head node coordinates the cluster, and by default it runs your tasks too. Its section is collapsed when the dialog opens; click its title to open it.

Node Type, CPU Family and Instance Sizeโ€‹

  • Node Type: the head node is always a High-Performance CPU (ECU) node. The GPU types are marked "(workers only)".
  • CPU Family: AWS Graviton5.
  • Instance Size: from large to 48xlarge, the same sizes as the ECU worker pool. Until you choose one, the head node follows the size of the first worker pool you selected, so it changes when you change that pool's size.

When your cluster includes a GB200 pool, the head node is raised to at least 8xlarge and the whole Head Node Configuration section becomes read-only. Sizes below 8xlarge are marked "(requires non-ECU-E cluster)".

Storageโ€‹

The head node's disk is gp3 (General Purpose SSD) block storage. Enter its Size in the field next to it:

  • Default and minimum: 128.
  • Maximum: 65,536 (64 TiB).

The field is labelled GB; the numbers are binary gigabytes (GiB).

Allow Tasks on Head Nodeโ€‹

Under Head Node Tasks, Allow Tasks on Head Node is on by default: the head node runs your tasks and actors as well as coordinating the cluster. Turn it off to keep the head node for coordination only; the cluster then needs at least one worker pool. What it does and when to turn it off is covered in Allow Tasks on Head Node. It cannot be changed after launch.

Worker Pool Configurationโ€‹

Choosing the worker typesโ€‹

Each worker type is a card. Click a card to add a pool of that type to your cluster, and click it again to remove the pool. You can combine types in one cluster.

CardHardwareShown for
High-Performance CPUs (ECU)AWS Graviton5Data processing
General Purpose GPUs (ECU-G)AWS Graviton2 CPU + NVIDIA Tesla T4G GPUAI/ML Dev & โ‰ค40B Inference
Enterprise GPUs (ECU-E)NVIDIA Grace-Blackwell Superchip (GB200)AI/ML Training, โ‰ค1T Inference, & HPC

A new cluster starts with the ECU card selected. You can also remove every pool: with Allow Tasks on Head Node on, the head node alone then runs your work. The ECU-E card needs the Enterprise plan and a reserved capacity block; see GB200 capacity blocks.

Each selected card opens its own pool settings, described below.

Instance Sizeโ€‹

Instance Size sets the size of every node in the pool. A larger size has more vCPUs and memory (and, for GPU sizes, more GPUs). Each option reads as a number of units followed by the size: for example 2 ECUs (2xlarge) in an ECU pool and 2 ECU-Gs (2xlarge) in an ECU-G pool. Below the choice, the panel adds up the pool's CPUs, memory, GPUs and GPU memory.

The panel labels memory in GB; the figures below are in binary gigabytes (GiB).

High-Performance CPUs (ECU). The default is 2xlarge.

SizeECUsvCPUsMemory (GiB)
large1/228
xlarge1416
2xlarge2832
4xlarge41664
8xlarge832128
12xlarge1248192
16xlarge1664256
24xlarge2496384
48xlarge48192768

General Purpose GPUs (ECU-G). Each node has NVIDIA T4G GPUs with 16 GiB of memory each. When you add an ECU-G pool, choose its size yourself from the list, which starts at 2xlarge. Then, in the Review Configuration summary, check the ECU-G pool's size: if it is not the size you chose, go Back, select a different size and then the one you want.

SizevCPUsMemory (GiB)GPUsGPU memory (GiB)
2xlarge816116
4xlarge1632116
8xlarge3264116
16xlarge64128232

Enterprise GPUs (ECU-E). A GB200 pool is one whole capacity block, and its size is the size of the block you choose. The figures are for the whole block.

SizeNodesCPUsMemory (GiB)GPUsGPU memory (GiB)
9xlarge91,2968,640366,660
18xlarge182,59217,2807213,320

Scalingโ€‹

Each ECU and ECU-G pool has two numbers, the range Ray's autoscaler may scale the pool between:

PoolMin Nodes defaultMax Nodes defaultMax Nodes limit
ECU020100
ECU-G0420
  • Min Nodes starts at 0 and can be no more than Max Nodes. Nodes up to this number stay running even when idle.
  • Max Nodes is the most nodes the pool can grow to under load.

With a Min Nodes of 0, a pool has no nodes until work arrives, and the first job waits for nodes to start. How the autoscaler grows and shrinks a pool is described in Cluster lifecycle.

A GB200 pool shows Nodes instead: the node count is set by the size of your capacity block, and it cannot be changed or scaled. In a cluster that mixes GB200 with ECU or ECU-G pools, the autoscaler scales only the ECU and ECU-G pools.

Worker storageโ€‹

Each worker node gets its own gp3 (General Purpose SSD) disk. Storage sets its Size for every node in the pool, and the panel shows the pool's total:

  • Default and minimum: 128.
  • Maximum: 65,536 (64 TiB).

As for the head node, the field is labelled GB and the numbers are GiB.

Shutdown After Idleโ€‹

Each worker pool has its own Shutdown After Idle, with the same choices as the cluster's Auto Shutdown After Idle. It starts at the cluster's value and follows it until you change it for this pool. Once you have, Reset timeouts to whole cluster value puts the pool back on the cluster's value.

For a GB200 pool this setting is read-only. GB200 pools show Nodes, Storage and Shutdown After Idle only once a capacity block is chosen.

GB200 capacity blocksโ€‹

A GB200 pool runs on a capacity block: GB200 capacity you reserve and prepay for a fixed time window. Selecting the Enterprise GPUs (ECU-E) card opens these settings.

Choosing a capacity blockโ€‹

The Capacity block list shows the blocks your organization has reserved, for example 9xlarge ยท 9 nodes / 36 GPUs with its length in hours. Pick the block this cluster should use. If you have none, the list reads "โ€” no reserved blocks โ€” reserve new below". A block marked "held for review, contact support" cannot be chosen; contact Eugo support about it.

Reserving a new blockโ€‹

Below the list, the reserve form asks for:

  1. Instance size: 9xlarge ยท 9 nodes / 36 GPUs or 18xlarge ยท 18 nodes / 72 GPUs.
  2. Duration: 1 to 14 days, then whole weeks from 3 weeks up to 26 weeks (182 days). The default is 7 days. Lengths between those steps are not offered.
  3. Start date: from today up to 8 weeks ahead.
  4. Get quote shows the windows on offer with their prices. If none is offered, try a later start date or a smaller size.
  5. Pick a window and tick the box that reads: "I understand this is an irrevocable, non-refundable purchase and authorize the charge. The total shown (block price plus tax) is deducted from your organization's credits immediately at purchase."
  6. Type purchase ecu-e in the confirmation field, then click Reserve & Purchase.

A capacity block is a purchase that cannot be undone or refunded, so check the size, duration and start date before you confirm.

Starting ECU and ECU-G workers earlyโ€‹

When you pick a capacity block and the cluster also has ECU or ECU-G pools, one more checkbox appears, off by default:

Start my on-demand (ECU / ECU-G) workers now, before the GB200 block is delivered. They begin billing immediately and cannot run GB200 work until the block arrives. Leave unticked to start everything together on the delivery date.

Leave it off to start the whole cluster together when the block is delivered.

Advanced Configurationโ€‹

Advanced Configuration is collapsed when the dialog opens; click Show to open it.

Cluster-Wide Storageโ€‹

Enable Cluster-Wide Storage is on by default. It creates shared storage (Amazon EFS) with your cluster and mounts it at /mnt/ss on every node, so all nodes see the same files there. The storage is elastic, so there is no size to choose. Terminating the cluster removes this storage with it (see Security).

A cluster with a GB200 pool shows Cluster-Wide Storage (Included) instead, already ticked and not changeable: an FSx for Lustre file system is created with the cluster and mounted at /mnt/ss on every node, the head node and all workers, including any ECU and ECU-G pools. There is no size to choose here either.

Eugo Runtime Imageโ€‹

On the Enterprise plan, a Eugo Dev Tools part appears with the note "Eugo Dev Tools are in Early Access. APIs may change without notice and documentation may be incomplete." Its one setting, Eugo Runtime Image, picks the Eugo image every node runs:

  • Standard covers most uses.
  • Extended (Full Package Set) carries the full Eugo package set, including build-time dependencies, for advanced development work.

It defaults to the image your own session runs, and to Standard otherwise.

Review and launchโ€‹

Review Configuration shows a summary of everything you chose; Cancel closes the dialog without creating anything. From the summary, Back returns to the settings and Launch Cluster creates the cluster. For a GB200 cluster, the capacity block must be chosen first.

Changing a running clusterโ€‹

On a running cluster's card in the EugoHPC Manager, Scale cluster opens Scale Cluster, followed by the cluster's name. There you can change each worker pool's Min Compute Nodes and Max Compute Nodes, within the same limits as at launch, and save with Apply Changes. The head node, storage, shared storage and the other settings are shown read-only, and a GB200 pool's node count stays fixed by its capacity block. To change anything else, launch a new cluster.

  • EugoHPC Manager: how workspaces and clusters are organized, the cluster lifecycle, Allow Tasks on Head Node, and connecting your session.
  • Installing Python packages: what is preinstalled, and how to get extra packages onto the cluster's workers.