๐งฐ ezpz CLIโ๏ธ
Once installed, ezpz provides a CLI with a few useful utilities to help launch
distributed PyTorch applications.
Explicitly, these are ezpz <command>:
- ๐
ezpz launch: Launch commands with automatic job scheduler detection (PBS, Slurm)- ๐ฏ
ezpz test: Run simple distributed smoke test1. - ๐
ezpz benchmark: Run all examples and generate a report - ๐ฎ
ezpz submit: Submit jobs to PBS (qsub) or SLURM (sbatch); generates job scripts automatically
- ๐ฏ
- ๐
ezpz status: Inspect job status locally or over SSH/IRI (fanout, filters, JSON) - ๐ซ
ezpz cancel: Cancel a queued or running job (local or remote) - โฑ๏ธ
ezpz extend/ezpz update-job: Extend or retarget a queued job โ walltime, queue, account - ๐
ezpz fetch: Read a job's stdout without opening an interactive SSH session - ๐
ezpz logs: Browse per-example benchmark logs for a run - ๐ฅ๏ธ
ezpz resources: List IRI compute resources (Aurora, Polaris, Crux, โฆ) - ๐จ
ezpz incidents: List IRI incidents โ maintenance windows, outages, facility events - ๐ฆ
ezpz yeet: Distribute files (envs, models, datasets, etc.) to all worker nodes via parallel rsync- ๐๏ธ
ezpz tar-env: Package current Python environment as a tarball
- ๐๏ธ
- ๐ฉบ
ezpz doctor: Health check your environment - ๐
ezpz kill: Kill ezpz-launched python processes (local node or--all-nodes) - โฌ๏ธ
ezpz update: Upgradeezpzin the active environment and refreshutils.sh - ๐ค
ezpz export-amsc: Export a finished run as one AmSC at-scale benchmarks CSV row - โจ๏ธ
ezpz completion: Emit shell completion forbash/zsh/fish -
๐
ezpz.examples: Collection of distributed training examples (DDP, FSDP, ViT, FSDP+TP, diffusion, HF, HF Trainer, inference) -
ezpz --helpTo see the list of available commands, run:
$ ezpz --help Usage: ezpz [OPTIONS] COMMAND [ARGS]... ezpz distributed utilities. Options: --version Show the version and exit. -h, --help Show this message and exit. Commands: benchmark Run all ezpz examples sequentially and generate a report. cancel Cancel a queued or running job. completion Emit shell completion for bash/zsh/fish. doctor Inspect the environment for ezpz launch readiness. export-amsc Export a finished run as one AmSC at-scale benchmarks row. extend Extend or retarget a queued job (IRI PUT / SSH qalter). fetch Read a job's stdout without opening an interactive SSH session. incidents List IRI incidents โ maintenance windows, outages, facility events. kill Kill ezpz-launched python processes (or any matching pattern). launch Launch a command across the active scheduler. logs Browse per-example benchmark logs. resources List IRI compute resources. status Show job status (local or remote). submit Submit a job to the active scheduler (PBS/SLURM). tar-env Create (or locate) a tarball for the current environment. test Run the distributed smoke test. update Upgrade ezpz in the active environment and refresh utils.sh. update-job Alias for extend. yeet Distribute files (envs, models, datasets, etc.) to worker nodes.
-
This is really just a wrapper around:
↩