Compressing Onboard Language Models for Space Robots

THE CENTRAL IDEA

Do not compress onboard, and do not accept the implicit behavioral drift of generic post-training quantization. Treat compression as a ground-segment step that emits one static artifact: assign each linear module to the quantizer family—AWQ, GPTQ or AutoRound—whose substitution keeps the final next-token distribution closest to the BF16 model validated on the ground.

Compressing Onboard Language Models Under a Fidelity Requirement

The challenge: flight rules for onboard language models

Language models are being explored as the interface through which space robots interpret operator commands and select among verified skills. Running them onboard means low-bit compression—and flight use adds requirements that generic post-training quantization leaves implicit: a fixed memory budget and storage format decided before launch, a static artifact placed under configuration control, and behavior that stays close to the full-precision model validated on the ground.

These rules turn compression from an accuracy-versus-size trade-off into a verification problem. A compressed model that drifts from the validated reference in uncommon contexts is not a smaller model—it is a different model, and one that has never been reviewed. The paper's framing is precise: for flight, behavioral fidelity to the reference is a requirement, not a metric.

Compression as a ground-segment step

The authors move the entire compression pipeline to the ground segment. Nothing adapts onboard: the artifact uploaded to the spacecraft is one static low-bit model, produced before flight, frozen under configuration control, and evaluated against the BF16 reference with the same discipline as any other flight software delivery. This separation matters operationally—onboard computing is spent on running the model, never on deciding what the model should be.

A ground-segment view also clarifies who signs off. Because the artifact is static and the reference is the ground-validated model, the acceptance question becomes measurable: does every accepted substitution move the final output distribution closer to the reference, or at least not farther? That question can be answered exhaustively before flight, module by module, without touching the onboard stack.

Three mechanisms of QSelect

The method, QSelect, combines three mechanisms. First, module-level composition: starting from compatible AWQ, GPTQ and AutoRound checkpoints of the same base model, each linear module is assigned to exactly one quantizer family, so the final artifact mixes families layer by layer rather than committing to a single one. Second, output-level scoring: every candidate substitution is scored at the final next-token distribution, using the forward KL divergence from the BF16 reference. Third, fidelity-gated acceptance: in the default fidelity mode—no labels, no tuned thresholds—a substitution is accepted only when it decreases that KL divergence; an optional target-adapted mode adds answer likelihood on a small labeled set for task-specific tuning.

The acceptance gate is what makes the scheme flight-compatible. Each substitution must argue for itself against the reference; anything that cannot is refused. The result is a composed model whose every module carries a documented, distribution-level reason for being there—exactly the kind of evidence chain a configuration-control board can review.

accept(m → q) ⇔ KL( P_BF16 ‖ Q_new ) < KL( P_BF16 ‖ Q_cur )

Why score at the final output

A natural alternative is to score substitutions by similarity of intermediate representations—cheaper and smoother. In a matched scaffold, the paper shows the final-output criterion is strictly better where it counts: scoring at the next-token distribution gives lower held-out KL than representation-similarity scoring in all 12 paired runs. Hidden-state similarity can look reassuring while the logits—the part the operator actually reads—drift.

For teams building similar pipelines, the transferable lesson is the scoring location, not the quantizers. Whatever compression families are on the table, acceptance should be decided where behavior is observed—at the model's output distribution—because that is the only level at which "close to the validated reference" has an operational meaning.

Results: composed models win all six settings

On DeepSeek-R1-Distill-Qwen-7B, Llama-3.1-8B and Llama-3.2-3B at 3 and 4 bits, composed models reach lower mean held-out perplexity than every single-family model in all six model-and-bitwidth settings, and the fidelity mode lowers perplexity and KL further on top. With labels drawn from the training or development splits of the evaluated task families, the target-adapted mode raises 3-bit pooled accuracy on the two Llama models by 4.3 points.

The pattern of results matches the mechanism: no single quantizer family is best everywhere, so module-level selection harvests complementary strengths; and because acceptance is gated by fidelity to the reference, the composed model's aggregate gains do not come at the price of unexplained local regressions.

Where the method applies—and its boundaries

The method presumes compatible checkpoints of the same base model across quantizer families—without them, module-level composition has nothing to select from. The fidelity gate certifies distribution closeness, not task correctness: a composed model can be close to a reference that is itself wrong on some task family, which is why the optional target-adapted mode and task-level evaluation remain necessary before flight. And the largest headroom sits at aggressive bit-widths: at 3 bits the choice of quantizer matters most, which is where the composed approach shows its clearest advantage.

The authors are careful about the claim boundary: the evidence supports composed compression under a fidelity requirement on the evaluated models and task families, not a universal guarantee for arbitrary base models or quantizers. For adoption, that honesty tells reviewers exactly which parts of the pipeline must be re-run—checkpoint generation, fidelity gating and task evaluation—when the base model changes.

Engineering mapping and evaluation boundaries

Mapped to a flight program, QSelect sits entirely in the ground segment: it consumes the validated BF16 reference and the candidate checkpoints, and emits one static low-bit artifact plus its acceptance log—the KL delta of every admitted substitution. Onboard, the model loads into the fixed memory budget and storage format decided at design time; runtime assurance reduces to monitoring inputs and outputs, because the artifact itself never changes.

A concrete evaluation plan for adoption: regenerate the quantizer-family checkpoints for the target base model; run fidelity-gated composition with frozen settings; then evaluate the artifact on a task matrix that deliberately oversamples the command-and-skill-selection contexts expected in orbit, reporting perplexity, KL and task accuracy side by side. Only when the artifact reproduces the reference's behavior on contexts the composition process never saw should it enter configuration control as a flight candidate.

SOURCE PAPER & REFERENCE MATERIAL