Speeding up netlist generation time

I am using eeschema as a way to capture a netlist, and I use a lot of sheets to create a significant hierarchical layout. Some of the netlists I have created so far have ~500k connections, and creating them takes an hour or more.

Are there any techniques that I can use to reduce the netlist generation time? For example, is it possible for components to have bus ports to reduce the number of connections? Or a way to have a component represent multiple components?

Are you creating an electronic brain? That seems like a lot of connections!

As I understand it, eeschema works by matching up graphical coordinates with a brute force search, so it probably does not scale well. A smarter way would be to make the connections as the schematic is drawn, but that might lead to other problems.

The only way I know to reduce the complexity of a schematic is to split into separate projects.

Is this a PCB or something like an IC or fpga layout?

I am using eeschema for data flow visualization, so it is not board design related. Then I am taking advantage of the netlist plugin to convert the XML file created by eeschema into my own format. I believe it is one of the only schematic capture tools that outputs a netlist in an easily usable format.

Based on my experience with the software so far, I figured brute force was mostly being used and that it is typically used for relatively manageable netlist sizes.

Separate projects is one option I had been considering and will probably end up using. But even some of the more simple netlists I have tried take a few minutes to generate, and if I break it down any further I will have too many projects. Unfortunately I do not see any other workable options, unless there is another schematic capture tool that someone can recommend.

As a mindstorm alternative: Lucidchart?

would it be possible to improve the algorithm by either:

  1. binning/gridding? this can help for simple nearest neighbor search where if we look in the closest bin and a few surrounding bins we don’t need to look further out.
  2. k-d tree or other tree datastructure, where 2D points are inserted into a datastructure which adaptively divides into a tree which is fast to search.

(both of these are used in openvoronoi, a pet project of mine… https://github.com/aewallin/openvoronoi)
I could maybe help with this but would need serious guidance from someone who knows the kicad codebase.

Anders