Section 4 — Failure Modes
Attention degradation
As a session grows, each token's attention budget is spread across more competitors. The signal on any one meaningful relationship shrinks; noise from irrelevant context crowds...
Matt Pocock
As a session grows, each token's attention budget is spread across more competitors. The signal on any one meaningful relationship shrinks; noise from irrelevant context crowds in. Same model, same parameters — just more mouths to feed from the same plate. Cause of the smart zone / dumb zone effect.
Usage:
"It's deep in the dumb zone — inventing generics that aren't in the type file."
"Attention degradation. The type definitions are still in context, but the signal on them is buried under everything we've added since. Clear and reload."
Share