Thank you for all the help. I have a better grip on it now, I think making my own footprint (and possibly a clearer schematic symbol) is the way.
My main gripe remains, though, but at least now I know that is the way it is supposed to be.
What I think is ”plain wrong” (in my original post) is that the schematic, and how symbols are treated there, is connected to the PCB in an opaque and non-intuitive way. For me, a schematic is a totally symbolic representation of a circuit. I can rotate or mirror a transistor symbol to better fit into the schematic without having to worry how it will end up on the PCB, because that is a representation on a different abstraction level that deals with other things, like the actual board and components. There I can rotate and move the transistor to fit on the physical board.
If I on the other hand rotate or mirror a pot it gets physically different results on the PCB. If I rotate it, as opposed to mirroring it, the outer pins get swapped, which has an effect on the physical world – how a pot is turned.
I cannot fully understand where the difference lies, but one thing is that a pot has a mechanical connection to the user, it kind of reaches across abstraction layers.
Also, the pot symbol has no visible pin numbers in the schematic, and no pin notation in the PCB editor either.
So, I guess I’ll make my own PCB footprint, and possibly also schematic symbol (with pin numbers).
Thank you all!