You need to modify the footprint and add text to the other side of the pcb. Set value of the text to %R and pcbnew will automatically substitute it with reference when footprint is placed on the board.
Thanks for the suggestion. I’m trying to avoid modifying footprints since then I’m out of sync with possible future (git) updates.
I thought/hoped the Duplicate feature would be able do something about this issue (having component references on both sides) but for now I’ll stick by just flipping the component reference to the bottom side in stead of the top side.