Running on KISTI¶
KISTI-6 has 4× GH200 (aarch64) per node, like Clariden, but it isn’t a CSCS system. It has no FirecREST, no CSCS container plugin (EDF), no /capstor, and no route to the OpenTela mesh. sml runs there with the slurm launcher from a login node, stock pyxis for containers, and OpenTela and metrics disabled.
Only
sml advancedis KISTI-aware for now. The interactivesmlflow still assumes CSCS (model paths, OpenTela, no--comment) and gets refused.
Site rules¶
KISTI’s submit filter (cli_filter) refuses jobs that break either of these rules:
| Rule | How sml handles it |
|---|---|
Jobs must run from /scratch or /appsdata, not $HOME |
SML_WORKING_DIR=/scratch/$USER/sml |
Every job carries --comment="field=<field>;appl=<appl>" (showappl) |
--sbatch-arg="--comment=field=nlp;appl=vllm" |
Two more rules don’t trigger a refusal, but the job won’t work without them:
- GPUs are only visible if requested: the example passes
--sbatch-arg=--gres=gpu:4. - The driver is 570 (CUDA ≤ 12.8): use
*-cu129images (they run via minor-version compatibility), not the default CUDA 13 builds.
One-time setup¶
On a login node (ssh <user>@hangang.ksc.re.kr). The system python3 is 3.6, so use cray-python:
cd /appsdata/$USER # /scratch purges files unaccessed for 15 days
git clone https://github.com/swiss-ai/model-launch && cd model-launch
module load cray-python/3.12.12
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
sml init # choose the "slurm" launcher; the API key is only used by the TUI health check
# Import the vLLM image once (~4 min). The tag must match vllm_kisti.toml.
mkdir -p /appsdata/$USER/sml/images
enroot import -o /appsdata/$USER/sml/images/vllm-v0.32.0-cu129-arm64.sqsh \
docker://vllm/vllm-openai:v0.32.0-cu129
Launching¶
cd /appsdata/$USER/model-launch && module load cray-python/3.12.12 && source .venv/bin/activate
export SML_ACCOUNT=<your project> # e.g. in0138
export SML_WORKING_DIR=/scratch/$USER/sml
./examples/kisti/cli/qwen/Qwen3-0.6B-vllm.sh # partition intTest; override with SML_PARTITION=normal
The example is an ordinary sml advanced call. To serve another model, copy it and change --framework-args. Models are pulled from Hugging Face at job start (compute nodes have internet) into HF_HOME=/appsdata/$USER/sml/hf. The flags that make it KISTI-specific:
| Flag | Why |
|---|---|
--environment …/vllm_kisti.toml |
Image and HF_HOME on /appsdata (not purged), both mounted |
--container-runtime pyxis |
srun --container-image instead of CSCS --environment |
--sbatch-arg="--comment=field=nlp;appl=vllm" |
Required by cli_filter |
--sbatch-arg=--gres=gpu:4 |
GPUs are only visible when requested |
--disable-opentela --disable-metrics |
Their binaries and endpoints live on CSCS |
--pre-launch-cmds "pip uninstall -y -q torchcodec \|\| true" |
vllm-openai:*-cu129 ships a CUDA 13 torchcodec that crashes the import |
SML_SBATCH_ARGS and SML_CONTAINER_RUNTIME can replace the two --sbatch-args and --container-runtime if you’d rather set them once in your shell.
Reaching the model¶
Without OpenTela there is no gateway, so you talk to the node directly on port 8080:
squeue --me # NODELIST, e.g. node1895
curl http://node1895:8080/v1/models
curl http://node1895:8080/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"model":"<user>/Qwen/Qwen3-0.6B","messages":[{"role":"user","content":"hi"}]}'
The served name is namespaced with your username (see served model names). From your laptop, tunnel through the login node:
Logs¶
Everything lives under $SML_WORKING_DIR/logs/<jobid>/ (on /scratch, so purged after 15 days):
log.out/log.err: the batch script (node list, replica URLs)replica_<r>.out/.err: the framework itselfreplica_health.json: the in-job health checker’s last report
Cancel with scancel <jobid>.
How it works¶
--container-runtime pyxis makes sml read the env toml where the job is rendered (so it must be a local path) and translate it for stock pyxis:
| EDF toml key | pyxis |
|---|---|
image |
--container-image ({arch} resolved on the node) |
mounts |
--container-mounts (bare /path becomes /path:/path) |
workdir |
--container-workdir |
[env] |
exported on the batch node and named in --container-env (else the image’s value wins) |
annotations |
ignored (CSCS hooks) |
Values are expanded by bash on the batch node, so $USER works. The same mechanism should carry over to other clusters with stock pyxis (e.g. JSC).