Ok, tested a few as "all games" wasn't specific enough: Amidar, Berzerk, Bosconian, Carnival, Choplifter, Commando, Defender, Galaga, Galaxian, Jackal, Joust, Moon Cresta, Mr Do, Moon Patrol, Pacman, Popeye, Rolling Thunder, Salamandar, Scramble, Tempest, Tron, Up n Down. Basically none of the games I try save the high score.
I hate changing mame versions, because often roms that did work suddenly done, like xavious now doesn't work, and it's worked for the last 10 years or more on the older versions.
A previous version I had was mame32 0.83, dated 2004, so not that recent. But that saved the entire game on exit, high scores, credits, the lot. The new one saves nothing. As all the games I tend to play are pre-1990 I shouldn't really matter.