Python API¶

How you run a model. The model itself is the YAML file — SPEC is what it may contain and what it means; this page is the nineteen names that load, check, build, solve and read one back. The surface is pinned by a test and the reasoning behind its size is ARCHITECTURE.

Six verbs — check, load_model, build, solve, solve_over, write — and the exception tree rooted at LpspecError: LanguageError (with SchemaError, DimensionError, PiecewiseExpansionError) for the model, DataError for what was bound to it.

import lpspec as lps

lps.check('model.yaml')  # parse → validate → lower, no data bound
model = lps.load_model('model.yaml')  # Model — the declared math
model.to_dict()  # ...and back out, as data
model.to_yaml()  # ...or as the file a reviewer reads

result = lps.solve('model.yaml', sources, solver_options={'time_limit': 60})
# ...or solver_name='gurobi', the other solver sink — same model either way
result.status, result.termination_condition, result.objective
result.is_ok  # rolled-up verdict: not an error, abort or refusal
result.has_primal  # narrower: are there values to read
result.primal('p')  # tidy frame (dims…, value) — the native shape
result.dual('power_balance')  # shadow prices, the same shape and the same join
result.to_pandas('p')  # the same, as a DataFrame
result.to_dataarray('p')  # the same, labelled: .sel / resample / plot
result.to_dataset()  # every variable by default; names for a subset
result.to_parquet(directory)  # streamed to disk, never through this process

lps.write('model.yaml', sources, 'model.lp')  # sink chosen by the suffix

Nothing has to be released. The built model is frames this process owns, so primal and the to_* readers stay valid for as long as the Result does. close() and the context-manager protocol exist to hand a large model back early, not because forgetting them breaks anything. lps.build returns the executor when one build should feed more than one sink:

ex = lps.build('model.yaml', sources)
ex.write('model.lp')
ex.omissions()  # rows a constraint declared but did not build, and why it matters
result = ex.solve()

What sources accepts is SPEC §8. Nothing on this path imports linopy, and primal returns a polars.DataFrame — Arrow-backed, so it exports the same protocol the loader recognises. to_pandas and to_dataarray are the bridges out and need pandas / xarray, which ship with the [linopy] extra. The only build knob is coords; solver_options is not a build knob and is forwarded verbatim to the solver.

Every verb takes the model four ways: a path, a str, a dict, or a Model. check, build, solve and write share one first argument, so a framework that emits declarations never writes a temporary file to run them:

model = {'dimensions': ..., 'variables': ..., 'constraints': ..., 'objectives': ...}

lps.solve(model, sources)  # a dict runs like a file
checked = lps.load_model(model)  # ...or validate once and keep it
checked.to_yaml()  # the review copy — a dict-built model still gets a file
lps.solve(checked, sources)  # a Model is passed through, not revalidated

This is the supported path for a framework, and it is the one closing #29 and #30 chose: a library composing optional features emits data, not YAML text, and never merges files. What keeps it honest is the last two lines — a generated model that cannot show you a file is the failure mode hard rule 5 exists to prevent, so to_yaml() is not a convenience here, it is the condition. Hand-written math still starts as a file; nothing about this path asks it not to.

A Model goes back out two ways, and they agree. to_dict() is the model as data; to_yaml() is that dict as the file hard rule 5 says you review and diff — which a model built as a dict, the way a framework emits one, would otherwise never have. tests/test_roundtrip.py holds load → out → load for both forms over every example and every port, holds that the two forms match, and holds that dumping twice gives the same bytes, since a review copy that changes per run is a diff nobody can read.

Every value is written; only what is absent is dropped — a null, an infinite bound, or a mapping that declares nothing. One mechanical rule, on purpose: omitting defaults reads better but needs a list of which ones are consequential, and that list is a second copy of the schema. An empty list stays, because a list carries cardinality here and zero is one of its values — foreach: [] is a scalar declaration.

An infinite bound is in that list because it is not a bound — it is the unbounded side, which is exactly what omitting the bound already means. That also makes JSON lossless: JSON has no infinity, so anything reaching model_dump_json as inf came back as null and read as absent regardless.

The rule lives on the model's serializer, so model_dump, model_dump_json, to_dict and to_yaml all give the same content — a helper beside them would have left pydantic's own methods describing the model differently from the file.

Which solver is a caller's choice, not the file's. solver_name is highs (ships with the package) or gurobi (needs the [gurobi] extra), and nothing in the YAML names one — the same file means the same model whichever takes it. Options travel in the chosen solver's own vocabulary, {'time_limit': 60} for HiGHS against {'TimeLimit': 60} for Gurobi, because forwarding verbatim is the contract and translating names would mean holding an opinion about every option either one has. A name outside the two is an error listing them, never a quiet fallback to the default.

Gurobi's remote and licensing options travel the same way, so Compute Server, Instant Cloud and WLS need nothing from this package:

options = {'ComputeServer': 'srv:61000', 'ServerPassword': '…'}
lps.solve('model.yaml', sources, solver_name='gurobi', solver_options=options)

They are applied when Gurobi's environment is created, which is what ComputeServer, TokenServer and WLSAccessID require.

Reading a result:

Rule
`is_ok` is not `has_primal`	`is_ok` rolls up the termination condition; `has_primal` adds the solver's verdict on whether an incumbent exists, and is what every reader gates on. A MIP that hits `time_limit` before finding a feasible point is `ok` with nothing to read
reading anyway	`NoSolutionError`; `objective` is `nan`
`dual` raises rather than zero-filling	no values at all is `NoSolutionError`; values but no duals — any integer or binary variable makes them undefined — is `LpspecError`, because only this quantity is missing
duals exist only where a solver ran	either solver sink hands them back through the same join; a model written to LP and solved elsewhere never passes back through here. Reduced costs and slacks ride that join too and are not exposed yet
`to_dataset` costs what it says	each variable arrives dense over its own dims — name a subset, or use `to_parquet`
`write`	the suffix picks the writer — `.lp` today, `.mps` a `NotImplementedError` naming it as planned, anything else a `ValueError` listing both sets. Checked before the build, so a format nothing can write costs no model

Solving one model many times¶

solve_over runs the same model once per slice and folds the answers. It is a driver over solve, not a second engine: a plan cannot contain a loop; a process may loop over plans (the ceiling). Scenarios, rolling horizons and myopic pathways are all the same fold.

runs = lps.solve_over('model.yaml', sources, lps.EachCoordinate('scenario'), executor=ProcessPoolExecutor(4))
runs.objective  # (scenario, status, termination_condition, objective)
runs.primal('p')  # (scenario, snapshot, generator, value)

runs = lps.solve_over(
    'window.yaml',
    sources,
    lps.EachWindow('snapshot', length=48, step=24, into='t'),
    carry={'soc_initial': ('soc', 23)},
)
runs.primal('soc')  # (snapshot_start, t, value) — the window, and the index inside it

Runs reads like Result, one dimension wider — primal, dual, to_pandas, to_dataarray, to_dataset, to_parquet, under the same names and with the slice key prepended. That extra dimension is named by you, not by the library: EachCoordinate('scenario') keys on scenario and EachCoordinate('draw') on draw, a window on <dim>_start, and key_name= overrides either. So runs.to_dataarray('p') on a scenario sweep is (scenario, snapshot, generator), which is what a sweep is for: .sel one scenario, take a spread across them, plot the band.

stitch= is how you ask for the answer over real coordinates, and it is a keyword on the readers rather than a reader of its own:

runs.primal('soc')  # (snapshot_start, t, value) — keyed by slice
runs.primal('soc', stitch=True)  # (snapshot, value)          — the answer
runs.dual('balance', stitch=True)  # the same, for a price

A flag rather than a method because what has to be undone is a property of the axis, not of the quantity — so duals get it for free, and a name that is both a variable and a constraint (which the language permits) is never dispatched on. to_pandas and to_dataarray take it too.

Every axis answers it. For EachWindow it is the answer a rolling horizon is for: the overlap dropped and the global coordinate restored, each window contributing the step coordinates it owns — the final one included, which can hold no more and so keeps all of it. snapshot_start + t would not do, because a window spans coordinates rather than values and there is nothing to add for a datetime or a string axis. For EachCoordinate and a hand-built axis nothing was re-indexed, the key column already is a coordinate, and the frame comes back unchanged.

Keyed is the default, because stitching is lossy: it keeps only what each window owns and drops the lookahead rows the sweep solved. A default that discarded computed answers would also key differently from objective, which is one row per slice always, and the two would stop joining. For the same reason to_dataset and to_parquet have no stitch — a bulk export of what the sweep holds is the wrong place to lose rows.

Per slice is a partition of a frame you already have, so there is no reader for it: runs.primal('p').partition_by(runs.key_name, as_dict=True).

Rule
a partition is a filter on the sources	not a narrower `coords` — the containment check refuses parameter rows outside the declared coordinates, by design. The axis rewrites the sources and supplies the matching `coords` together
everything a slice produced is kept	every variable's primals and every constraint's duals, read back through `runs.primal(name)` and `runs.dual(name)`. It is still a fold — each slice's model is released as the loop goes, so build peak stays at one slice however many there are, and what accumulates is the answer. Narrowing that is a later addition and an easy one; it is absent because it would need two keywords, a constraint being allowed to carry a variable's name
duals are keyed, never combined	`runs.dual(name)` is `runs.primal(name)`'s shape. Averaging window prices, taking the last, and reading one slice alone are all defensible, so the reduction is the caller's. A slice whose model had an integer variable contributes none, and `runs.objective` says which
no aggregate objective	`objective` is a frame keyed by slice. Scenarios are a distribution, not a sum; summing window objectives double-counts whatever the overlap discards
duals are not exposed	a window's shadow price is that window's. Concatenating them into a price curve is wrong in a way nothing complains about
`carry` is a copy, never arithmetic	`{parameter: (variable, index)}`. Accumulation — `existing += built` — is a derived variable in the YAML, where the math is reviewable
the two declarations say what is copied	whichever dimension the variable has and the parameter does not is the one the carry collapses, and `index` names a coordinate of it. Everything else rides along. So `soc` over `(t, storage)` into `soc_initial` over `(storage)` drops `t` and hands both stores forward, and `total` over `(generator)` into `existing` over `(generator)` drops nothing and needs no index — pass `None`
the carry index is explicit	with `EachWindow(…, 48, 24, …)` the state to carry is at coordinate 23 of `into`, not 47. An implicit "last" is correct until overlap is introduced and silently wrong after
a carry is checked before anything is read	the dims come from the YAML, so a carry that cannot line up — collapsing two dimensions at once, a parameter over more than the variable is, an index where the sides already match — raises before the axis has scanned a single source, never mind solved a slice. `check` cannot answer this for you: `carry` is an argument to the call, not part of the model
a hand-built axis names its own key	the class axes derive the key column — `EachCoordinate('scenario')` keys on `scenario`, a window on `<dim>_start` — but a plain list of cuts cannot say what its keys are coordinates of, so it must pass `key_name='draw'`. `'slice'` would be this library naming somebody else's axis, which is the same reason `into` has no default. `key_name` overrides the derived name anywhere, and is refused only when it collides with a dimension a kept variable already carries
the model is parsed once	`solve_over` validates it up front and hands every slice the schema, so a model outside the streaming language fails before the data is touched and no worker re-reads the YAML
a window keys as `<dim>_start`	`EachWindow('snapshot', …)` drops `snapshot` and re-indexes to `into`, so the key column is `snapshot_start` and holds where each window began. Naming it `snapshot` would put window starts under the name of the coordinate they are not, and join cleanly against real data
a slice that did not solve	contributes no `primal` rows, so that frame can be shorter than the sweep. `objective` is one row per slice always, and is the record of which slices those were
the lookahead is `t >= step`	overlapping windows return every row they solved, including the tail the next window recomputes. Keeping only what each window owns is one clause and no special case — `runs.primal('soc').filter(pl.col('t') < step)` — because the final window can never hold more than `step` rows
a sweep's memory grows with its answer	each slice's model is released as the fold goes, so build peak stays at one slice — but the extracted frames accumulate, and nothing bounds them. `to_parquet` copies out frames already in memory: a bridge, not a bound. Whether to bound it, and how, is #610
a window spans coordinates, not values	`length=48` is forty-eight snapshots however they are numbered, so the dimension only has to be orderable — datetimes, strings and gapped integers all work. `into` is a dense `0..n-1` local index, which is what keeps the seam's `where: "t == 0"` matching, and it has no default because the name belongs to the model
non-positional grouping	"each calendar month" has unequal groups, so it is a precomputed column plus `EachCoordinate`. What `EachWindow` uniquely offers is overlap
`carry` excludes `executor`	a carried value makes slice i+1 depend on slice i, so the slices cannot run concurrently. Refused rather than one silently winning
`workers_share_fs`	whether the executor's workers can read your paths. Inferred from the pool — a stdlib `ProcessPoolExecutor` runs here and reads what is here, anything else is assumed remote — and only path sources are affected

Choosing an executor¶

executor is any concurrent.futures.Executor — a submit returning a Future, and nothing else. That is deliberate: this package ships no remote transport and no vendor integration, so the executor is the only extension point there is, and it has to be one anybody can implement. tests/test_strategy.py runs the whole sweep through a nine-line class to keep that honest.

	use it when	notes
`None` (default)	always, until measurement says otherwise	sequential. Nothing is serialised, because nothing crosses a boundary
`ThreadPoolExecutor`	rarely	works, and sources are not encoded — but polars is already multithreaded, so slices contend with its pool, and threads share an address space so peak is additive rather than per-worker
`ProcessPoolExecutor`	genuine local parallelism	must not use `fork` — see below. Sources cross as parquet
anything remote	a cluster you already run	dask's `Client`, ray's wrappers, loky. Assumed not to share your filesystem, so paths travel as bytes; pass `workers_share_fs=True` if the workers really do mount it

A forked worker hangs. polars' thread pool does not survive fork, and the failure is a hang rather than an error — indistinguishable from a slow solve, which makes it the worst shape a failure can take. Measured: fork never returns where spawn and forkserver both do. It cannot be enforced from inside solve_over, because a remote executor has no start method to inspect, so pass the context yourself:

import multiprocessing
from concurrent.futures import ProcessPoolExecutor


def main():
    ctx = multiprocessing.get_context('spawn')  # or 'forkserver'
    with ProcessPoolExecutor(4, mp_context=ctx) as pool:
        runs = lps.solve_over('model.yaml', sources, lps.EachCoordinate('scenario'), executor=pool)


if __name__ == '__main__':  # spawn re-imports your module; without this it recurses
    main()

Parallel is N × peak. Each worker holds its own slice's model, so a four-way pool wants four times the memory of one slice. That is a machine decision, and the reason None is the default rather than a pool sized for you.

Sources cross a process boundary as parquet, never as pickled frames — smaller and faster on every shape it was measured against. A path the workers can reach stays a path; one they cannot travels as its own bytes untouched, because decoding and re-encoding a parquet file produces identical output at a large multiple of the CPU. Either way a source no slice rewrote is encoded once for the whole sweep rather than once per slice. Pass paths or frames, whichever you already have — there is nothing to tune. (df.lazy() is not an optimisation: an eager frame is embedded in the plan, so it pickles larger than the frame. Only scan_parquet is a reference.)

The linopy shim (lpspec.linopy.build / .extend, [linopy] extra) puts the same YAML math on a linopy.Model that already exists in memory. It is documented with everything else about that relationship in docs/design/linopy.md.