Configuration¶

Configuration profiles collect reusable scheduler options, job defaults, retry rules, and retention policy.

Default locations¶

molq uses the MolCrafts configuration root:

Set MOLCRAFTS_HOME to relocate both:

export MOLCRAFTS_HOME=/srv/molcrafts

The project path is then derived below that root. CLI commands accept --config when only the profile file needs an override.

Minimal profile¶

profiles:
  gpu:
    scheduler: slurm
    cluster_name: dardel

Load it in Python:

import molq as mq

cluster = mq.Cluster.from_profile("gpu")
queue = mq.Submitor.from_profile("gpu", target=cluster)

Or from the CLI:

molq submit slurm --profile gpu python train.py

Full profile¶

profiles:
  gpu:
    scheduler: slurm
    cluster_name: dardel
    jobs_dir: /scratch/alice/.molq/jobs

    scheduler_options:
      sbatch_path: sbatch
      squeue_path: squeue
      scancel_path: scancel
      sacct_path: sacct
      extra_sbatch_flags: ["--clusters=gpu"]

    defaults:
      resources:
        cpu_count: 8
        memory: "34359738368"
        gpu_count: 1
        time_limit: 14400
      scheduling:
        partition: gpu
        account: project123
        qos: normal
      execution:
        cwd: /scratch/alice/project
        job_name: molq-job

    retry:
      max_attempts: 3
      retry_on_states: [failed, timed_out]
      retry_on_exit_codes: [1, 137]
      backoff:
        mode: exponential
        initial_seconds: 10
        maximum_seconds: 120
        factor: 2

    retention:
      keep_job_dirs_for_days: 14
      keep_terminal_records_for_days: 90
      keep_failed_job_dirs: true

plugins:
  nerve:
    enabled: true
    expand_threshold: 8
    debounce_seconds: 0.3
    ingest_url: "http://127.0.0.1:17890"
    show_members: attention

Units in the config file¶

Profile values use the serialized storage representation:

The Python API and CLI accept friendlier constructors and strings: Memory.gb(32), Duration.hours(4), --mem 32G, and --time 4h.

Profile fields¶

The plugins mapping is top-level rather than nested below a profile. They apply to CLI sessions opened from the same config file. molq daemon defaults to the official Nerve plugin only when no plugin table exists; see Plugins and Nerve.

Precedence¶

For job fields:

submit_job(...) value
        overrides
profile / Submitor default
        overrides
dataclass default

Merging happens field by field. A submission can override cpu_count while retaining profile memory and time limits.

Scheduler options¶

Each scheduler accepts only its matching option table:

Using options for the wrong scheduler raises an error.

Remote profiles¶

Set host to make the profile describe a remote destination. It accepts anything ssh accepts, including a ~/.ssh/config alias:

profiles:
  gpu:
    scheduler: slurm
    cluster_name: dardel
    host: dardel
    defaults:
      resources:
        cpu_count: 8
        memory: "32G"

Both halves of the profile then load together — destination and lifecycle:

import molq as mq

with mq.Submitor.from_profile("gpu") as queue:
    job = queue.submit_job(argv=["python", "train.py"])

Omit host and the profile runs on the current machine.

To apply a profile's resource defaults to a destination you build yourself, pass an explicit target — it wins over the profile's host:

cluster = mq.Cluster("dardel", "slurm", host="dardel-backup")

with mq.Submitor.from_profile("gpu", target=cluster) as queue:
    job = queue.submit_job(argv=["python", "train.py"])

The CLI picks the host up automatically:

molq submit slurm --profile gpu python train.py

Use an isolated database¶

Applications and tests can construct a store explicitly:

from molq import Cluster, Submitor
from molq.store import JobStore

store = JobStore("./state/jobs.db")
queue = Submitor(
    target=Cluster("test", "local"),
    store=store,
)

JobStore requires a path. Submitor(store=None) is the convenience path that opens the canonical MolCrafts database automatically.