Strategic Reasoning
Overview
This area tracks multi-agent interaction, incentives, game-theoretic structure, and reasoning about other agents. It differs from Safe Multi-Agent Reinforcement Learning by centering incentive, equilibrium, information, and adaptation structure even when no safety intervention is being designed.
Active Questions
- How do safety constraints alter strategic equilibria or coordination patterns?
- When does a safety result depend on strategic structure, and when should that dependency be represented in Safe Multi-Agent Reinforcement Learning instead?
- When should we treat norms, rewards, or constraints as socially negotiated rather than fixed?
- How should agents choose strategies when multiple values can fail for reasons partly outside their control?
- Which abstractions best capture strategic state for learning and verification?
- How do adaptive agents shape each other’s future learning updates?
- How do strategic abilities shift when agents can change game rules or acquire information under resource bounds?
- When do coupled safety constraints change the relevant equilibrium notion or invalidate single-agent duality intuitions?
- How should agents reason when other agents are not fixed opponents but concurrently updating learners?
- When can satisfaction thresholds or aspiration updates stabilize cooperation without explicit opponent modeling?
- When do individually safe or well-coordinated strategies fail after being embedded in a larger agent population?
- What communication structures make shared interpretation a strategic property rather than an individual representation?
- How do descriptive moral-cognition models compress multi-agent responsibility, patienthood, and harm into simpler strategic frames?
- How should agents infer subtasks, roles, or teammate types when communication and prior coordination are absent?
- When does direct incentive-giving become cooperation support, and when does it become deception or manipulation?
- What benchmark evidence is needed before treating a coordination mechanism as broadly strategic rather than task-specific?
- Which equilibrium guarantees come from the stationary stochastic-game model itself, and which require additional learning, constraint, or potential-game assumptions?
- When does an agent’s strategic incentive to preserve, ignore, or corrupt feedback channels change the safety analysis?
- How should intervention and counterfactual questions be represented when agents can adapt their strategies to the changed game?
Key Concepts
- Causal Games
- Multi-Agent Influence Diagrams
- Causal Game Intervention Semantics
- Goal Alignment
- Reward Tampering
- Corrigibility
- Aspiration-Based Reinforcement Learning
- Multi-Agent Off-Switch Game
- Emergent Communication
- Successful Misunderstandings
- Opponent Shaping
- Stochastic Games
- Constrained Markov Potential Games
- Multi-Agent Non-Stationarity
- Multi-Agent Coordination
- Ad Hoc Teamwork
- Social Learning in MARL
- Cooperative MARL Benchmarking
- Safe Reinforcement Learning
- Probabilistic Strategic Timed CTL
- Alternating-Time Temporal Logic
- Dynamic Epistemic Logic
- Responsibility Anticipation
- Dyadic Morality
- Decision Theory
- Causality
Key Sources
- MacKinlay2026 - Opponent Shaping as a Model for Manipulation and Cooperation
- Fox2024 - Causality and Strategic Reasoning
- Everitt2019 - Towards Safe Artificial General Intelligence
- Bendor2001 - Aspiration-Based Reinforcement Learning in Repeated Interaction Games
- Fink1964 - Equilibrium in a Stochastic n-Person Game
- Ghasemi2025 - Toward Virtuous Reinforcement Learning
- Parker2024 - Responsibility in a Multi-Value Strategic Setting
- Varshney2026 - An Algebraic Exposition of the Theory of Dyadic Morality
- Jamroga2026 - Towards Probabilistic Strategic Timed CTL
- Dolgorukov2024 - Dynamic Epistemic Logic of Resource Bounded Information Mining Agents
- Galimullin2025 - Changing the Rules of the Game
- Varricchione2023 - Synthesising Reward Machines for Cooperative MARL
- Zhang2025 - Trustworthy Reinforcement Learning under Constraints and Perturbations
- Alatur2024 - Provably Learning Nash Policies in Constrained Markov Potential Games
- Papoudakis2019 - Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
- Agrawal2026 - The Multi-Agent Off-Switch Game
- Kondylidis2025 - Successful Misunderstandings: Learning to Coordinate Without Being Understood
- Rother2023 - Disentangling Interaction using Maximum Entropy Reinforcement Learning in Multi-Agent Systems
- Wang2020 - Too Many Cooks: Coordinating Multi-agent Collaboration Through Inverse Planning
- Ahmed2022 - Deep Reinforcement Learning for Multi-Agent Interaction
- Chelarescu2021 - Deception in Social Learning