26TH INTERNATIONAL CONFERENCE ON COMPUTING IN HIGH ENERGY & NUCLEAR PHYSICS (CHEP2023)

A web workbench system for the Slurm cluster at IHEP

Not scheduled

1h

Hampton Roads Ballroom and Foyer Area (Norfolk Waterside Marriott)

Poster Poster Poster Session

Speaker

Fu, Shiyuan ( Institute of High Energy Physics, CAS)

Description

Slurm REST APIs are released since version 20.02. With those REST APIs one can interact with slurmctld and slurmdbd daemons in a RESTful way. As a result, job submission and cluster status query can be achieved with a web system. To take advantage of Slurm REST APIs, a web workbench system is developed for the Slurm cluster at IHEP.
The workbench system consists with four subsystems: dashboard, tomato, jasmine and cosmos. The dashboard subsystem is used to display cluster status including nodes and jobs. The tomato subsystem is developed to submit special HTCondor glidein jobs in the Slurm cluster. The jasmine system is used to generate and submit batch jobs based on workload parameters. The cosmos subsystem is an accounting system, which not only generates statistical charts but also provides REST APIs to query jobs. Besides, Slurm asks for user JWT Tokens to call Slurm REST APIs, therefore, the workbench system provides a token REST API to provide user JWT Tokens for the four subsystems. This token REST API can also be used by other web systems to interact with Slurm cluster.

Peer reviewing

Paper

Accepted on Dec 15, 2023 by Arne Wiebalck

Paper files:

• 2023_workbench.pdf
• public_copyright_slurm_workbench_scan.pdf