Layer-Wise Knowledge Analysis: Identifying Critical and Redundant Computation

The challenge: preserve visual evidence under compute pressure

A robot examining a spacecraft panel needs more than a plausible description. Its perception system must distinguish a visible latch from an expected latch, an open interface from an occluded one, and a nearby tool from an attached component. These distinctions matter because language becomes a bridge from observation to action. When a model changes an object label or spatial relation late in inference, the downstream executive receives a different world description even though the camera image has not changed.

At the same time, running every layer for every output token consumes resources that may also be needed for tracking and planning. The engineering question is therefore twofold: where does useful visual information become available, and which subsequent computation changes the decision? A layer-wise view creates a way to investigate both questions together. The goal is to retain the transformations that matter for physical interpretation while identifying computation that can be simplified with measurable consequences.

Paper evidence: critical and mutation layers

Using LLaVA-1.5 with a 32-layer Vicuna backend on AMBER and MSCOCO, the paper examines intermediate token probabilities, adjacent-layer Jensen–Shannon divergence and hidden-feature projections. It identifies rapid evolution, stabilization and mutation phases. Some late prediction changes accompany hallucinated entities. Skip-layer experiments preserve meaning in selected intervals, while more aggressive skipping can degrade descriptions.

An intermediate readout can be understood as asking a repeated question: if this representation were decoded now, which token would it favor? A probability curve then records when a candidate enters contention and whether it remains competitive. Reading only the final winner discards this history. For an engineering investigation, the history can distinguish a weak visual hypothesis that never formed from a plausible hypothesis that was later displaced.

Interpreting distribution change

Jensen–Shannon divergence compares two complete probability distributions. A small value means the distributions are similar, not that either distribution is correct. Likewise, an unchanged top token can coexist with a large redistribution among alternatives. It is therefore useful to record three signals together: distribution divergence, the margin between leading candidates, and whether the semantically important prediction changed. These signals answer different questions and should not be reduced to one confidence number.

Tokenization introduces another practical issue. A component name may occupy several tokens, while punctuation may need only one. For a proposed diagnostic dashboard, group tokens into semantic fields such as object identity, attachment state and relative direction. Compare equivalent fields across layers and examples. This prevents a large number of stable function words from overwhelming a small number of unstable terms that actually determine the robot’s next action.

Proposed diagnostic workflow for spacecraft inspection

Build a scene suite in which the camera pose, illumination and occlusion can be varied independently. Each scene should carry a structured reference: visible objects, visible interfaces, attachment relationships and uncertain regions. Ask short, targeted questions before requesting a long description. A question such as “Is the handle attached to this panel?” produces a more focused trace than an unrestricted scene narrative and makes disagreement easier to associate with a visual condition.

For each question, capture selected hidden states, candidate probabilities and the final structured answer. Replay the same image with controlled changes in the prompt, then replay changed images with the same prompt. These paired interventions separate sensitivity to wording from sensitivity to the evidence. A useful report shows the image, the layer trajectory and the semantic field together, enabling a reviewer to locate exactly where the interpretation diverged.

Proposed depth selection: measure before removing

Use the traces to nominate candidate layer intervals for removal, rather than making a global assumption that all late layers are interchangeable. Evaluate each candidate checkpoint with the same scene suite and output contract. A layer interval that appears redundant for object names may still contribute to spatial relations or negation. Score these fields separately, and require acceptable results on the most consequential fields before accepting a faster model.

Static depth reduction and dynamic early exit lead to different implementations. A statically shortened model offers a consistent graph and simpler timing analysis. A dynamic scheme can allocate more work to difficult inputs, but also needs reliable exit criteria and a compatible attention-cache strategy during autoregressive decoding. Compare both as system designs, including branch overhead, memory residency and worst-case latency. The objective is a better complete pipeline, not simply a smaller count of transformer blocks.

Proposed runtime response to semantic disagreement

A compact monitor can focus on a small set of action-relevant fields instead of storing every hidden tensor. If an attachment state changes between selected readouts, the task executive can request a closer view or an additional observation before committing to the corresponding manipulation. The monitor supplies a reason for acquiring information: a specific field changed, and a particular observation can help resolve it. This creates a direct link between internal model diagnostics and embodied information gathering.

Thresholds should reflect the cost of the decision. A descriptive log entry and a grasp target need not use the same escalation rule. Record the observation timestamp alongside the proposed answer, so a stable prediction is not confused with a current prediction. In a moving scene, additional deliberation can make the input stale; a refresh of visual evidence may be more valuable than further language computation on an old frame.

Evaluation: connect internal traces to task outcomes

The proposed scorecard combines semantic field accuracy, disagreement detection, additional observations, end-to-end latency and task completion. Measure how often the monitor identifies a wrong decision, how often it interrupts a correct one, and whether the extra observation resolves the ambiguity. Compare the full model, a statically reduced model and a monitored model on identical episodes. This turns interpretability into an actionable design instrument: evidence about where computation helps, where it changes a judgment, and what the robot should do next.