Can Your Policy Weather the Storm?