Emergent Communication
Definition
Emergent communication is the development of a signaling protocol by interacting agents rather than by direct hand specification.
Why It Matters
Learned communication can help agents coordinate under partial observability, but it also creates an interpretation problem: high task reward may not mean that agents use signals with shared meanings. This is especially important for hybrid human-AI populations and safety benchmarks that use coordination success as a proxy for understanding.
Formalism / Key Objects
- A common setup is a signaling game: a Sender observes a state
s, emits a signalc, and a Receiver chooses an actiona. - Agents update signal-state and signal-action utilities, for example
V(c,s)for a Sender role andV(c,a)for a Receiver role. - Task reward measures coordination, while separate semantic metrics can compare whether different agents assign signals to the same states or actions.
- In Successful Misunderstandings, shared interpretation is not inferred from reward alone; it is tested through alignment, Sender intent, and robustness to new interaction partners.
Connections
- Successful Misunderstandings names the failure mode where coordination succeeds despite misaligned signal interpretations.
- Multi-Agent Non-Stationarity is adjacent because communication protocols can change while agents adapt.
- Ad Hoc Teamwork raises the transfer problem: a useful protocol for one partner population may not be interpretable to unseen teammates.
- Strategic Reasoning treats communication as a population-level institution, not merely an individual policy output.
- Explainable AI matters when one wants signal use to be interpretable rather than only reward-improving.
Common Confusions
- Emergent communication does not guarantee natural-language-like semantics.
- Coordination success does not prove that agents understand signals in the same way.
- A communication protocol can be stable for one partner pairing but brittle under population changes.