Skip to content

๐Ÿงฐ ezpz CLIโš“๏ธŽ

Once installed, ezpz provides a CLI with a few useful utilities to help launch distributed PyTorch applications.

Explicitly, these are ezpz <command>:

  • ๐Ÿš€ ezpz launch: Launch commands with automatic job scheduler detection (PBS, Slurm)
    • ๐Ÿ’ฏ ezpz test: Run simple distributed smoke test1.
    • ๐Ÿ“Š ezpz benchmark: Run all examples and generate a report
    • ๐Ÿ“ฎ ezpz submit: Submit jobs to PBS (qsub) or SLURM (sbatch); generates job scripts automatically
  • ๐Ÿ“Š ezpz status: Inspect job status locally or over SSH/IRI (fanout, filters, JSON)
  • ๐Ÿšซ ezpz cancel: Cancel a queued or running job (local or remote)
  • โฑ๏ธ ezpz extend / ezpz update-job: Extend or retarget a queued job โ€” walltime, queue, account
  • ๐Ÿ“„ ezpz fetch: Read a job's stdout without opening an interactive SSH session
  • ๐Ÿ“ ezpz logs: Browse per-example benchmark logs for a run
  • ๐Ÿ–ฅ๏ธ ezpz resources: List IRI compute resources (Aurora, Polaris, Crux, โ€ฆ)
  • ๐Ÿšจ ezpz incidents: List IRI incidents โ€” maintenance windows, outages, facility events
  • ๐Ÿ“ฆ ezpz yeet: Distribute files (envs, models, datasets, etc.) to all worker nodes via parallel rsync
    • ๐Ÿ—œ๏ธ ezpz tar-env: Package current Python environment as a tarball
  • ๐Ÿฉบ ezpz doctor: Health check your environment
  • ๐Ÿ’€ ezpz kill: Kill ezpz-launched python processes (local node or --all-nodes)
  • โฌ†๏ธ ezpz update: Upgrade ezpz in the active environment and refresh utils.sh
  • ๐Ÿ“ค ezpz export-amsc: Export a finished run as one AmSC at-scale benchmarks CSV row
  • โŒจ๏ธ ezpz completion: Emit shell completion for bash/zsh/fish
  • ๐Ÿ“ ezpz.examples: Collection of distributed training examples (DDP, FSDP, ViT, FSDP+TP, diffusion, HF, HF Trainer, inference)

  • ezpz --help

    To see the list of available commands, run:

    $ ezpz --help
    Usage: ezpz [OPTIONS] COMMAND [ARGS]...
    
    ezpz distributed utilities.
    
    Options:
    --version   Show the version and exit.
    -h, --help  Show this message and exit.
    
    Commands:
    benchmark    Run all ezpz examples sequentially and generate a report.
    cancel       Cancel a queued or running job.
    completion   Emit shell completion for bash/zsh/fish.
    doctor       Inspect the environment for ezpz launch readiness.
    export-amsc  Export a finished run as one AmSC at-scale benchmarks row.
    extend       Extend or retarget a queued job (IRI PUT / SSH qalter).
    fetch        Read a job's stdout without opening an interactive SSH session.
    incidents    List IRI incidents โ€” maintenance windows, outages, facility events.
    kill         Kill ezpz-launched python processes (or any matching pattern).
    launch       Launch a command across the active scheduler.
    logs         Browse per-example benchmark logs.
    resources    List IRI compute resources.
    status       Show job status (local or remote).
    submit       Submit a job to the active scheduler (PBS/SLURM).
    tar-env      Create (or locate) a tarball for the current environment.
    test         Run the distributed smoke test.
    update       Upgrade ezpz in the active environment and refresh utils.sh.
    update-job   Alias for extend.
    yeet         Distribute files (envs, models, datasets, etc.) to worker nodes.
    

  1. This is really just a wrapper around:

    ezpz launch python3 -m ezpz.examples.test
    
     ↩