Inverse finite element analysis (iFEA) of the atrioventricular heart valves (AHVs) can provide insights into the in-vivo valvular function, such as in-vivo tissue strains; however, there are several limitations in the current state-of-the-art that iFEA has not been widely employed to predict the in-vivo, patient-specific AHV leaflet mechanical responses. In this exploratory study, we propose the use of Bayesian optimization (BO) to study the AHV functional behaviors in-vivo. We analyzed the efficacy of Bayesian optimization to estimate the isotropic Lee-Sacks material coefficients in three benchmark problems: (i) an inflation test, (ii) a simplified leaflet contact model, and (iii) an idealized AHV model. Then, we applied the developed BO-iFEA framework to predict the leaflet properties for a patient-specific tricuspid valve under a congenital heart defect condition. We found that the BO could accurately construct the objective function surface compared to the one from a [Formula: see text] grid search analysis. Additionally, in all cases the proposed BO-iFEA framework yielded material parameter predictions with average element errors less than 0.02 mm/mm (normalized by the simulation-specific characteristic length). Nonetheless, the solutions were not unique due to the presence of a long-valley minima region in the objective function surfaces. Parameter sets along this valley can yield functionally equivalent outcomes (i.e., closing behavior) and are typically observed in the inverse analysis or parameter estimation for the nonlinear mechanical responses of the AHV. In this study, our key contributions include: (i) a first-of-its-kind demonstration of the BO method used for the AHV iFEA; and (ii) the evaluation of a candidate AHV in-silico modeling approach wherein the chordae could be substituted with equivalent displacement boundary conditions, rendering the better iFEA convergence and a smoother objective surface.