Orthogonality catastrophe in parametric random matrices