In this study, a novel online fault compensation control scheme based on policy iteration (PI) algorithm is developed for a class of affine non-linear systems with actuator failures. The control scheme consists of a PI algorithm and a fault compensator. For fault-free dynamic models, the PI algorithm is developed to solve the Hamilton–Jacobi–Bellman equation by constructing a critic neural network, and then the approximate optimal control policy can be derived directly. Alternatively, the actuator failure is reconstructed adaptively to achieve online fault compensation without the fault detection and isolation mechanism. The closed-loop system is proved to be asymptotically stable via Lyapunov's direct method. Two numerical simulation examples are given to demonstrate the effectiveness of the proposed fault compensation control scheme.