Understanding the inverse magnetocaloric effect in antiferro- and ferrimagnetic arrangements