Inference, compiled for your constraints.
Nebula compiles AI models into inference programs optimized for your hardware, workload, and numerical requirements.
- HardwareH100 SXM5
- Workloadbatch 1
- NumericsFP32 · RN
- Retained alternatives
- Selected route
- Across planes
From a model to an inspectable program.
Five stages. Each one hands the next something you can open.
Stage contract- 01ModelAn authored program: ops, shapes, dtypes.
- 02SearchKernels that could run each op, kept side by side.
- 03SelectionOne route elected, under your conditions.
- 04LoweringMade concrete: launch, memory, rounding.
- 05ProgramEvery row decided, every choice recorded.
- Retained alternatives
- Selected route
- Handed to the next stage
The model supplies the computation to preserve. The search is over implementations of that program, not over what the program should compute.Alternative implementations are represented together, so shared structure is never duplicated. Context narrows what is explored; it never justifies an equivalence on its own.A candidate is selected for one hardware target, workload, and numerical contract. The highlighted route describes a choice; it is not a claim of global optimality.The chosen plan is translated into target-level operations, keeping its correspondence to the selected representation inspectable rather than presenting a code snippet as proof.The program carries its selection context, revisions, and available evidence. A receipt indexes that evidence; it is never a substitute for it.
One model, three requests.
The same model, compiled for three requests. Each elects its own route through the candidate space, so each gets its own program. Schematic: measured runs appear on Results once accepted.
Request · H100 SXM5 · batch 1 · FP32 · RN
- Hardware
- H100 SXM5
- Workload
- batch 1
- Numerics
- FP32 · RN
What the program does
- One launch per expert; decode is bound by memory, not math.
- Q, K and V projections fused into one kernel.
- FP32 throughout, rounded to nearest.
Request · A100 SXM4 · batch 4 · BF16
- Hardware
- A100 SXM4
- Workload
- batch 4
- Numerics
- BF16
What the program does
- Experts grouped into one launch across the batch.
- Tiles resized for the smaller shared memory.
- BF16 storage, FP32 accumulation.
Request · H100 SXM5 · batch 4 · FP8
- Hardware
- H100 SXM5
- Workload
- batch 4
- Numerics
- FP8
What the program does
- FP8 matrix multiplies with per-tensor scales.
- Experts grouped and padded to the tile.
- Admitted only if its error against an exact reference is no worse than the authored form’s.
- Elected for this request
- Elected for the first request, where it differs