Reliability analysis of highly sensitive structures is crucial to prevent catastrophic failures and ensure safety. Therefore, these safety-critical systems are to be designed for extremely rare failure events. Accurate statistical quantification of these events associated with a very low probability of occurrence requires millions of evaluations of the limit state function (LSF) involving computationally expensive numerical simulations. Variance reduction techniques like importance sampling (IS) reduce such repetitions to a few thousand. The use of a data-centric metamodel can further cut it down to a few hundred. In data-centric metamodeling approaches, the actual complex numerical analysis is performed at a few points to train the metamodel for approximating the structural response. On the other hand, a physics-informed neural network (PINN) can predict the structural response based on the governing differential equation describing the physics of the problem, without a single evaluation of the complex numerical solver, i.e., data-free. However, the existing PINN models for reliability analysis have been effective only in estimating a large range of failure probabilities (10-1∼10-3). To address this issue, the present study develops a PINN-based data-free reliability analysis for low failure probabilities (<10-5). In doing so, a two-stage PINN integrated with IS (PINN-IS) is proposed. The first stage is employed to approximate the most probable failure point (MPP) appropriately while the second stage enhances the accuracy of LSF predictions at the IS population centred on the approximated MPP. The effectiveness of the proposed approach is numerically illustrated by three structural reliability analysis examples.