Local inference
Optional Tier-1 local model execution behind the InferenceBackend trait seam — a local model file by path, Metal on Apple, CUDA cloud-only.
Local model execution is opt-in (Tier 1). The default kx install is FFI-free and does not include it; enable it when you want a model running on your own machine.
What it is
- A local inference backend behind the
InferenceBackendtrait seam — the runtime talks to an interface, never to a specific engine, so the backend is swappable. - A model file by path — point the runtime at a model on your own disk.
- A small local agent model is the reference configuration.
Build requirements
The local backend compiles a native library, so it needs a C++ toolchain. The FFI-free default build does not.
GPU offload
- Metal on Apple silicon.
- CUDA is cloud-only (not part of the local build).
Swapping the backend
Because inference sits behind the InferenceBackend seam, you can implement your own backend (a different local engine, or a remote one) without touching the rest of the runtime. See Extending.
Frontier on demand
Local-first does not mean local-only. The runtime is designed to reach for hosted frontier capacity when a task needs the depth, while keeping the default path local and FFI-free.