Value Decomposition Networks
Definition
Value Decomposition Networks are cooperative-MARL value learners that approximate a shared team action-value function as an additive sum of per-agent value functions.
Why It Matters
VDN is one of the simplest centralized-training/decentralized-execution baselines. Its additive structure makes decentralized greedy action selection straightforward, and later methods can be read as attempts to relax, generalize, or outperform that factorization.
Formalism / Key Objects
For agents with histories h_i and actions a_i, VDN approximates the team value by
Training uses a shared TD loss on the team value, while execution lets each agent greedily choose from its local \tilde{Q}_i. Cooperative MARL Baseline Formalisms records this additive assumption beside the shared-reward DEC-POMDP and MAPPO/IPPO comparison setup.
Connections
- Cooperative Multi-Agent Reinforcement Learning supplies the shared-reward setting in which VDN is normally used.
- Temporal Difference Learning supplies the bootstrapped value-learning machinery.
- Multi-Agent PPO is a contrasting on-policy actor-critic baseline that Yu et al. show can be surprisingly strong in the same benchmark families.
- Regularized TD Stability and PQN uses a VDN-style additive extension, PQN-VDN, to test regularized TD learning in cooperative tasks.
- Multi-Agent Non-Stationarity remains relevant because each agent’s value target depends on the changing policy population.
Common Confusions
- VDN’s additivity is a structural assumption, not a theorem that all cooperative tasks decompose cleanly.
- Per-agent value heads do not mean each agent has its own reward; the training signal is the shared team return.
- VDN is simpler than monotonic mixing methods such as QMIX; it does not learn a state-conditioned nonlinear mixer.