In the metamodelling-based Monte Carlo simulation (MCS) framework for reliability analysis of structures, the training samples used to construct the response surface should be as close to the failure plane as possible to ensure sufficient accuracy in reliability estimates. For this, an algorithm based on the maximin distance criterion combined with an error norm based on leave-one-out cross-validation (CV) is proposed to construct a moving least-squares based response surface for improved reliability estimate. The algorithm hinges on the fact that the MCS points having predicted responses less than the maximum absolute error obtained by the leave-one-out CV approach are likely to have the maximum effect on the accuracy of the reliability estimate. Relying on this, two points are added at each iteration, ensuring that the new data points are sufficiently close to the actual limit state and adequately far from existing data points. The improved reliability estimation capability of the proposed algorithm considering the direct MCS-based results as the benchmark is elucidated numerically by considering two example problems.