The brain is arguably the most complex human organ and modelling its mechanical behaviour has challenged researchers for decades. There is still a lack of understanding on how this multiphase tissue responds to mechanical loading and how material parameters can be reliably calibrated. While previous viscoelastic models with two relaxation times have successfully captured the response of brain tissue, the Theory of Porous Media provides a continuum mechanical framework to explore the underlying physical mechanisms, including interactions between solid matrix and free-flowing interstitial fluid. Following our previously published experimental testing protocol, here we perform finite element simulations of cyclic compression–tension loading and compression–relaxation experiments on human brain white and gray matter specimens. The solid volumetric stress proves to be a crucial factor for the overall biphasic tissue behaviour as it strongly interferes with porous effects controlled by the permeability. An inverse parameter identification reveals that poroelasticity alone is insufficient to capture the time-dependent material behaviour, but a poro-viscoelastic formulation captures the response of brain tissue well. We provide valuable insights into the individual contributions of viscous and porous effects. However, due to the strong coupling between porous, viscous, and volumetric effects, additional experiments are required to reliably determine all material parameters.