Probabilistic Controlled Invariant Sets

Definition

A probabilistic controlled invariant set is a subset of a safe region from which a controller can keep the system safe with high probability over a specified horizon.

Why It Matters

PCISs bridge control-theoretic safe sets and reinforcement learning. If a learner proposes actions while a PCIS-based filter restricts execution to certified actions, the learner can explore under a runtime safety envelope rather than relying only on reward penalties or post hoc verification.

Formalism / Key Objects

• For an MDP M=(X,U,P,X_S) and horizon N, define

$$p_{{\Pi }_N}^{\Omega }(x_0)=\mathrm{Pr}(x_k\in \Omega ,\forall k=0,\dots ,N).$$

• A set X_PCIS subseteq X_S is an (N, epsilon)-PCIS if for every x in X_PCIS there exists a deterministic Markov policy Pi_N such that

$$p_{{\Pi }_N}^{X_{PCIS}}(x)\ge 1-ϵ.$$

• The exact safety predecessor operator for a reference set Omega is:

$$Q_{ϵ}^N(\Omega )=\{x\in \Omega :\exists {\Pi }_N\text{s.t.}{p}_{{\Pi }_N}^{\Omega }(x)\ge 1-ϵ\}.$$

• Hashimoto2026 - Data-Driven Synthesis of Probabilistic Controlled Invariant Sets for Linear MDPs computes a conservative approximation of this operator from data under a linear-MDP assumption.

Connections

• PCIS Safety Predecessor records the conservative recursion, sample-splitting certification, and induced safe action map.
• Shielding uses PCISs as runtime action filters: a proposed action is executed only if it belongs to the current safe action set.
• Control Theory provides the controlled-invariance viewpoint behind the safe set.
• Probabilistic Model Checking is adjacent because both reason about probability guarantees in MDPs.
• Invariant Synthesis is adjacent because a PCIS is a probabilistic controlled analogue of an invariant region.

Common Confusions

• A PCIS is horizon- and tolerance-dependent; it is not the same as an absolute infinite-horizon invariant unless extra assumptions are added.
• A data-derived conservative fixed point is not automatically certified if the same data selected it; hold-out certification or another uniform argument is needed.
• A PCIS shield can preserve safety while still affecting task performance, especially when aggressive actions near the boundary are needed for reward.

Key Sources

• Hashimoto2026 - Data-Driven Synthesis of Probabilistic Controlled Invariant Sets for Linear MDPs