Interesting to revisit this topic since the OP was nearly two years ago. The next major release (v5) will have a new symbol library table, but not new schematic file format. V6 will probably have that, and possibly Python API. Python API is not seem as a primary user interface yet it seems, so may not be a priority. However, KiCad development is based more on developer interest than user demand.
I had hoped for 2 year release cycle on KiCad, we seem to be more on track for 2.5-3 year cycle. So a Python API for eeschema could be 3+ years away. I would plan to develop something outside KiCad, in the long run you might be able to migrate to native KiCad scripts. There are several codebases for parts of KiCad, mainly for footprint and component libraries. I'm not sure if there is code available for processing schematic files, apart from the code I wrote.