Foundation Models for Space Robots

Knowledge must cross a physical interface

A foundation model can interpret an instruction, recognize an object, or reuse a learned behavior. For a space robot, those abilities become useful only when the output reaches an execution system with the right coordinates, units, timing, and physical assumptions. The review follows this path from perception and prediction through planning and action to validation.

Several interfaces are possible. A model can issue numerical targets, select existing skills, generate constraints for a planner, predict action sequences, or represent possible future states. Each places physical responsibility somewhere different. A skill library inherits the behavior of its controllers; a learned action policy inherits the dynamics and action conventions represented in its training data.

A moving arm also moves its spacecraft

On a fixed terrestrial robot, a workbench or floor supplies a familiar support condition. On a free-floating spacecraft, arm motion reacts on the base. Replaying a terrestrial end-effector trajectory changes spacecraft motion and control-resource demand. The action can have the same numerical representation and a different physical consequence.

Capture changes the system again. After contact, the client becomes part of a coupled mechanical interaction. Contact may dissipate kinetic energy, but internal forces cannot remove total momentum. Reaching a grasp pose therefore differs from retaining the grasp and stabilizing the combined system.

The paper identifies three places for adaptation: condition the model on relevant base and relative-motion state; adapt training to changed inertia, momentum, and contact; or modify the external controller that interprets proposed actions. These routes can be combined, but their effects should be tested separately before attributing a gain to the model.

More actions per second do not guarantee fresher feedback

Action chunking generates multiple future commands in one inference call. It can raise action throughput while the policy continues to execute from an older observation. Query latency, command rate, and the interval before fresh observations influence the policy are distinct measurements.

The review discusses methods that overlap inference and execution, and methods that use a faster decoder with fresh tactile or force feedback within a chunk. These mechanisms address different parts of the delay. They still require an interrupt or recovery path when a contact assumption fails.

For an approaching spacecraft, clearance must accommodate pose error, motion during sensing and computation, tracking error, and braking distance. A higher model throughput helps only if that improvement reaches this physical timing budget. A local controller may close a fast loop even while the high-level model updates slowly.

Event sensing and spiking computation need system tests

Event cameras report brightness changes and can offer timely observations in fast motion or strong illumination contrast. Spiking neural networks offer a route to sparse computation. The two are complementary design choices, but neither automatically establishes a better navigation or manipulation system.

Encoding, preprocessing, memory traffic, inference, and command execution determine whether a sensing advantage survives. Estimated arithmetic energy differs from measured task energy. A model evaluated with synthetic events or ideal target crops also answers a narrower question than a complete detector and estimator operating with physical sensors.

The comparison should keep tasks, priors, adaptation budgets, and computing boundaries aligned. Changes in model capacity or pretrained encoders can otherwise be mistaken for an advantage of the neural representation itself.

Separate generalization from adaptation

A frozen model solving an unseen task tests generalization under its available interfaces. Updating a policy, revising task instructions, or tuning a controller using target-domain interaction tests adaptation. Both can be valuable, but a report should identify what changed, how much interaction was used, and what knowledge was reused.

The proposed sim-to-ground-to-space pathway uses physical tests and telemetry to diagnose discrepancies, followed by closed-loop revalidation. A crossed evaluation can vary task novelty and physical novelty independently. It then measures whether performance survives new inertia, contact conditions, or task combinations within a stated adaptation budget.

A complete capture test records approach, contact, retention, stabilization, and failed attempts. It also records operator interventions, feedback timing, total energy, and sustained thermal behavior. That evidence connects reusable model knowledge to the work a space robot can actually complete.