Well he did tried to burn the city and that wasn’t just Jaime excuse since they did later found the wildfire stashes.

Except that his madness touched rather the nobles around him (note there weren’t any stories of him abusing the servants, though i would assume that did happened a lot, just people telling us Mad King horror stories were aristocrats too and servants were below their attention). Except that, even when he was desdending into madness his choice of people ruling as Hand in his stead was not bad (again except the alchemist at the very end).

Overall his reign ended with civil war too, although it was much faster and thus less bloody than the one after Robert death, and the biggest atrocity then was the sacking of King’s Landing by westerners (sounds kinda familiar, huh)