Safe Reinforcement Learning via Shielding
Summary
This seeded note is based on metadata and the first pages of the paper. The core idea is to synthesize a shield from a temporal-logic safety specification and an abstraction of the environment so the learner’s unsafe actions can be corrected with minimal interference.
Key Claims
- Reinforcement learning can be combined with hard safety guarantees during learning and execution by interposing a reactive shield.
- The safety specification is expressed in temporal logic, with focus on safety properties.
- A good shield should preserve safety while restricting the learner as little as possible.
Methods / Formalism
- Temporal logic specification of safe behavior.
- Environment abstraction used to synthesize the shield.
- Online monitoring of learner-selected actions with correction when necessary.
Evidence / Experiments
- Full empirical details still need full ingest.
- The paper positions shielding as a practical framework tested on multiple RL scenarios.
Connections
- Core source for Shielding.
- Connects Linear Temporal Logic to safe action selection.
- Sits between offline synthesis and online Runtime Verification.
Open Questions
- How well does shielding extend to decentralized or partially observed multi-agent settings?
- When does intervention distort exploration or long-run learning quality?
- Which abstractions are strong enough for safety without becoming too conservative?
Citation
Alshiekh, M., Bloem, R., Ehlers, R., Koenighofer, B., Niekum, S., and Topcu, U. (2018). Safe Reinforcement Learning via Shielding. AAAI-18.