The symbol libraries that are automatically added to every new project you create are determined by what is listed in the standard template file located (in linux) under /usr/share/kicad/template/kicad.pro
If you add the symbols.lib to this file (using a text editor) it will be added to every project you create from now on. (Otherwise you will need to do this for every new project you create.)
Well maybe @SchrodingersGat or @jkriege2 know about that.
I don’t even know where this standard template lives on github. (It is not in the kicad library repo under templates where i would have expected it to be)
Disclaimer for everyone reading this in 2018: KiCad version 5 does no longer need this stuff! Here the symbol libs are handled via a library table similar to how the v4 footprints had been handled (and are still handled)