Routing and placement

Describe requirements, not machines

A worker declares the models it needs by alias. A model name pins that model exactly; a requirement such as {"kind": "agent", "intelligence": "medium"} states a floor that placement resolves against what outposts serve. Intelligence tiers are declared by the operator who serves a model, never guessed from its weights. On one outpost a warm model beats a cold one, then the lowest satisfying tier wins.

Eligibility comes first: an outpost must be among the allowed set, be fresh in the registry, run or be able to load the model, advertise an executor for the operation’s kind and a peer endpoint, and be able to carry the output back in one reply. Eligible outposts are then ranked by your preferred list, then warm models, then advertised capacity per attempt already in flight, then local before remote.

Capacity is what each receiver advertises in its heartbeat, per operation kind, and the receiver’s admission remains authoritative. A detector attempt goes first to an outpost already running that detector, then to one that would load it and has the GPU memory free; a detector loaded for an attempt is unloaded after 15 idle minutes.

Preview before you submit

estimate(), and .explain() on a dataset, describe the scheduler’s plan without reserving capacity, moving inputs or running the worker: record and partition counts, the eligible outposts in rank order, a recommended concurrency, transfer bytes, the assumptions made and why any outpost was blocked. Durations are not estimated. Use preferred for a locality hint and allowed when execution must stay within named outposts.

1
2
3
4
5
6
7
8
9
10
11
12
13import drkm

drkm.connect()
worker = drkm.package("worker.py:run", ["worker.py"])
records = [{"id": "sensor-a", "data": {"temperature": 72}}]

plan = drkm.estimate(worker, inputs=records, preferred=["self"])
print(plan["partitions"], plan["eligible_outposts"], plan["blocked"])

# Restrict to named outposts by ID
fleet = drkm.outposts()
ids = [o["outpost_id"] for o in fleet if not o["stale"]]
plan = drkm.estimate(worker, inputs=records, allowed=ids)

Design for a changing link

The console and the agent talk only to the outpost’s API, and local models, local detectors and the job system need no route out; the map’s imagery and address search still reach their provider, and a hosted model reaches its provider. After a power cycle an outpost comes back to what it served: its cluster, API, model engines, catalog detectors and running guards restart on their own, and only an explicit operator stop, or a start that still fails once the host is up, stops them from coming back. What cannot resume is reported: an in-flight chat turn fails, a background run ends as lost and wakes its conversation once, a job attempt is marked unknown and its partition is retried or failed.

In the mesh, a member that cannot reach the authority keeps its membership and its last blocklist until it reconnects, and the Outposts page marks a peer stale after about five missed heartbeats and stops asking it for work. Keep inputs bounded, ship only the context a worker needs, and read coverage when a link changes.