A bigger context window does not remove the decision of what to put in it; it raises the cost of deciding badly. Sparse Evidence Attention (SEA) is the layer that chooses which durable records become part of a context pack. It is deliberately not a single similarity sort — it scores eight heterogeneous criteria so that a record which is strong on one axis but weak on another is caught.
RESEARCH · June 4, 2026 · 7 min read
How Sparse Evidence Attention Decides What the Model Sees
Before a model reasons, something must choose which records reach the context window. SEA scores eight criteria, then packs greedily under a hard token budget.
By MLAI Research · Abbey
Eight criteria, one weighted score
Each candidate c is scored on semantic similarity, keyword overlap, metadata fit, recency, source authority, graph connectivity, an explicit contradiction flag, and task-fit. The eight scores combine under a fixed weight vector, clamped to the unit interval — most mass on semantics, but never enough for one criterion to dominate:
\mathrm{score}(c) \;=\; \mathrm{clamp}_{[0,1]}\!\Big(\textstyle\sum_{i} w_i\, s_i(c)\Big)
w \;=\; (0.30,\ 0.15,\ 0.15,\ 0.10,\ 0.10,\ 0.10,\ 0.05,\ 0.05)
Packing under a hard budget
Scoring ranks the candidates; selection respects the window. SEA admits records greedily by score, but rejects any that would exceed the token budget B or violate a per-cluster diversity cap. Token cost is estimated from length, and the budget is a hard ceiling — the pack never overflows:
\mathrm{tok}(x) = \max\!\big(1,\ \lceil |x|/4 \rceil\big), \qquad \sum_{c \in S} \mathrm{tok}(c) \;\le\; B
Auditable by construction
SEA returns the rejected set and the reason each candidate was dropped — budget, diversity, or score — alongside the selection. That turns context assembly into something an operator can review: you can see why a record that 'should' have been included was not. Naive top-k stuffing cannot answer that, because it never made the decision explicit.