COPA
A unified framework for certifying robustness of offline RL against poisoning attacks
The goal of COPA is to systematically certify the robustness of different offline RL algorithms based on certification criteria including per-state action stability and the lower bound of cumulative reward. Specifically, we propose new partition and aggregation protocols (PARL, TPARL, DPARL) to obtain robust policies and provide certification methods for them.
In COPA-leaderboard, we present the certification results in three RL environments under two certification criteria. Notably, we offer direct comparisons from multiple aspects to enable better understanding of different aggregation protocols and offline RL algorithms of subpolicies.
The related paper can be found here.
Robustness certification for per-state action stability in Highway environment. We plot the cumulative histogram of the tolerable poisoning size K for all time steps. We provide the certification for different aggregation protocols (PARL, TPARL, DPARL) on three RL algorithms and different #sub-policies. The results are averaged over 20 runs with the vertical bar on top denoting the standard deviation.
Robustness certification for per-state action stability in Highway environment. We plot the cumulative histogram of the tolerable poisoning size K for all time steps. We provide the certification for different RL algorithms (DQN, QR-DQN, C51) on three aggregation protocols and different #sub-policies. The results are averaged over 20 runs with the vertical bar on top denoting the standard deviation.
