This reminds me of a book I read once that had a lot of talk about four continents in the four cardinal directions.
My imagination always went like “North? Up. East? Up. West? Up. South? Right.”
Though I didn’t have any problems with keeping in mind that north and south, and east and west were opposite each other, so you had to pass through one of the others first to get there.