Even with the box unchecked in the Preferences > Common it still opens the project that was last opened.
Did you save the setting when exiting the dialog?
There is no Save button. Just an OK. When KiCAD restarted the check box is clear so it remembers the setting , but it still loads the last project open.
When all else fails, read the docs? 
It seems that this option isn’t about going to the last opened project. I don’t have it checked and it always goes back to the last project.
Explanation says: Remember open files for next project launch: When enabled, KiCad will automatically re-open any files that were previously open when a project is re-opened.
So it seems that this option should reopen the editors if you were using them when you closed the project.
Yes this option is unrelated to opening the last project in KiCad.exe
OK so how do I stop KiCAD from reloading the last open project? Is there a way to do that? Or does it just not have an option for that? If so, that seems a bit silly…
Close the project, close KiCad.
The problem is that you cannot have two projects open in a single instance of KiCAD. So if you need to open another instance of KiCAD, it opens the project you were just on. Add to that KiCAD sometimes becomes unresponsive when trying to close a project, it adds to the frustration. There should be an option to have it start up blank. That would be nice. Altium has that option.
When I want to open a second KiCad instance with another project, I usually use a file browser to go to the project directory and then double click on the KiCad project. It’s also easy put some shortcuts for different projects “somewhere”. This can be directly on your desktop, in the “start menu” (or whatever that is called these days), or collect them in some directory. If you are working for some longer time on a project, this may be worthwhile.