Contributing to update of VRML librarys to STEP?


#1

A quick question, does anyone know if the standard GITHUB .pretty library of VRML models is being updated to include step, and secondly how I could offer some time to help with this endeavour?


#2

@kammutierspule will probably know.


Trouble on place 3d shapes
#3

There was some vague tentatives some time ago related with that topics but, as a probably lack of interest, they didn’t progress. I dont remember the discussion at time.

You can have a look at:
http://kicad-pcb.org/contribute/librarians/
I recomend you join the kicad channel in IRC and try joining the mailing list as it is described on the link.

I believe most of the help need would be to someone create the STEP models.

@Joan_Sparky and @maui did some work on this before (try check first what they did already), but their libraries was not integrated? I guess?
Maybe also @maui remembers something more about this…


#4

I, too, could help out creating STEP models. I will check out the link and IRC channel.


#5

just search the forum to avoid duplicating what has already done


#6

Initially I did a quick test loading up the wrl files to convert to STEP, a little bit of manipulation and I managed to get the meshes into a merged solid.

Then after reading responses above, and having searched the forum i can see that a huge amount has been done already, with all of the step models apparently already in StepUp for MCAD. Is there any reason why these could not be merged into the gihub master librarys at this point?


#7

The 3d files are in one huge repo together with all symbol libraries.
This repo has a backlog of more than 100 pull requests.
(In other words: there are not enough moderators for this repo. Someone needs to be responsible for quality control)

And i think there is not yet a lot of interest in having the step files included in this repo.
(Step files generated by freecad also have unreasonable large file sizes.)


#8

That would be it. Significant added value through quality control, that’s a tricky one to resolve. I guess the options are divide and conquer, which requires multiple moderators taking on different sub-librarys / classes.

Or if the models are parametricly generated (I think i read that in one of the posts) then they could be checked from the generation tables, which would speed things up.

The file size is a tricky one, and is due to the nature of STEP files being a human readable format (one of their great advantages). If there was a method of using zipped up step files this would bring the file size down by x6 or more times (from a quick test with winrar).


#9

I started asking for a need of a 3D MCAD repo since July 2015


You can easily find me also on developer list and forum asking for that, but I think there were a lack of interest and knowledge through developers about the need for a mechanical library… quite weird considering that some developer too are using/used StepUp and my library models to convert the kicad board to MCAD…

But now something has recently changed:

  • STEP is now directly supported by kicad (I still prefer VRML for nice raytracing, transparency support and convert my board with StepUp)
  • a great interest in MCAD conversion/models has been showed in the forum
  • @SchrodingersGat Oliver has just become one of the library maintainers…
    KiCad Library Status
    Oliver is also a developer of some MCAD models converted to VRML with material properties…

So may be this is the right time to have:

  1. parametric models coming from technical datasheet
  2. FreeCAD 1:1 model (optional)
  3. STEP 1:1 model (union of parts)
  4. VRML 1/2.54 model (derived from MCAD model) with Material Properties
    abandoning old and wrong Wings3D models upgrading most of the library

This approach will help users in having MCAD libraries, ready to go for a direct STEP export and nice VRML models with Material Properties aligned to the guide lines that @kammutierspule has created…

Maurice


#10

As far as I understand, GIT repositories are already compressed.
When you clone a repository you are already downloading a compressed “image” of the repository that then is uncompressed locally.
Moreover, repositories are better used with text files (as you said about STEP files) so if there is a little change in a file it will create deltas (differences between files, also compressed) so the changes will be quickly applied and little storage is need to keep track of the changes.

So it is not good to store compressed files on a repository.


#11

So at this stage it’s a case of making some more noises for the developers, and making suggestions about possible methods to upgrade the models and help minimise the time and resources needed to quality control the process.

Maui - from my structural engineering background I agree that parametric is the way forward in terms of defining models and allowing those models to be validated with minimal effort. Even if that is only in terms of initial generation, and the resulting models are stored.

With the inclusion of the ability to export the whole board to STEP there will be an expectation that the default librarys would support the feature. Especially with people likely moving to KiCad from eagle (such as myself) expecting that if a feature is implemented then the it would work with the basic included elements; such as capacitors / resistors etc.

kammutierspule - I’m not overly familiar with GIT, but to clarify is the concern with file size for the repository or space required on users disk?


#12

My guess is that the concern is mostly about transferring large files over the Internet.


#13

I would never recommend converting VRML to step; aside from the fact that there are no gurantees of a “closed shell” (valid solid object), you also end up with a mesh which simply makes the STEP model even more bloated than a comparable STEP model created with solid modeling software. Searching the forum for FreeCAD/STEP modeling work done by maui, JoanSparky, and others would be the way to start. In general, forget about VRML unless you want eye candy in which case maui’s scripts are the best tools you have for creating pretty VRML files from FreeCAD models.


#14

Just to clarify, the “library team” is a different team from the “development team”. So it is the library team you should make noise :slight_smile:

I assume there is no local disk past storage issues on any case.
Also there are less network problems today due the large bandwidth.
However it is not elegant and anti-natura the use of binary files (special compressed) with repositories.

If you use binary files in a repository and if you have the “file version A” and then you make a change to “file version B”
the repository will store: file@versionA and will store file@versionB.

If you use a text file the repository will store: file@versionA and will store difference between (file@versionB - file@versionA).compressed

This also allows you to inspect the textual differences between the files if you need.


#15

This thread touches on a lot of issues that have come up repeatedly over the last couple of years.

There are a few different issues here, I’ll attempt to address (add my 2c) some:

Library Pollution

Currently the 3D models supplied in the official libs are provided in the kicad-library repository. Note that this is separate from where the footprint files are stored - these are the individual .pretty repositories.

There have been suggestions that the 3D models should be stored with the footprints they are associated with e.g. https://github.com/KiCad/kicad-library/issues/162 - I am in strongly in favour of this.

The current approach means that the schematic symbols library grows large with files that are not associated with the schematics. Adding STEP files which are nominally much larger than the WRL files they would replace (or complement) would only add to this.

Separating the 3D models out would at least allow users to select which footprint/3D repos they download.

By my count, the kicad-library repo (inc. 3D models) is just over 1GB. The sum total of all .pretty repos is under 100MB.

This must be addressed by both the lib team and the dev team.

Model Creation
Converting WRL files to STEP files is not a good idea. Many of the recent 3D file additions have been parametrically generated (FreeCAD) which is great. However a lot of models in the official libraries would need to be completely replaced.

Licencing
Any models added to the libraries would need to be checked in terms of copyright - many companies do not allow redistribution of their 3D model data.

Contributing
In the mean time, we gratefully accept any contributions of good-quality 3D models. Currently the best place to submit them is kicad-library


#16

Are there any online repository systems suitable for large files? Off the top of my head, all the repo systems I know are designed for source code, i.e. text, so often don’t handle large blocks of data well. Delta versioning doesn’t really make sense for these files, maybe version control of 3D models is not even needed.

I started putting 3D models into my KiCad common repo, but the repo got noticeably slower to clone. I think I would want to store data files compressed, and uncompress at point of use (e.g. local cache).


#17

GIT it self can deal with whatever type of file, I think it has some different behavior depending on the detected file type (ie: if it is a binary file or text file).
I know that there is an extension to GIT, I quickly find this reference here: https://git-lfs.github.com
that has in mind deal differently with really large files. (but I don’t know much how it works)

For the KiCad library, I don’t have a strong opinion if that should be versioned or not (just stored somewhere).
On you (ours personal) repositories, if you are not planning changing much the binary files, then I think GIT is a good option (or else you will need a different storage place if you sharing between colleagues or online)


#18

Thanks @SchrodingersGat for come here and help clarifying!
Hope you can get more people working in the libraries and coordinate it well!


#19

I think that the “library team” cannot do without the approval of main developers…
I have already covered both roads without results … but as I already said the panorama has changed…

agree so much :smiley:

Best panorama in my point of view

I would suggest something similar to what we discussed more than a year ago and that covers already some of the 3D MCAD parametric models


and

EDIT
SparkFun has a GitHub repo for 3D models which covers STEP, SolidWorks, IGES, STL, SKP, blend type of files…
(not for all parts there are all types of format)
A nice thing is the the README file covers most of the requirements for a file to be uploaded


About licensing I couldn’t find a clear statement
If they use CC BY-NC-SA 3.0 as per images, that licence without exceptions wouldn’t easy fit commercial use, in a interchanging of 3D data
Here an example of a product with 3D models


#20

Personally I wouldn’t merge footprint folders with 3d model folders.
I don’t like the current original structure of the 3dshapes folder within the footprint folder either.
Do you keep STEP/VRML separate or do you mix them up?
In a not so distant future KiCAD will probably have a collection of spice models… will those be co-located with the symbols then?

Personally I keep the 3D models in the same folder structure as the footprint folders, in a separate folder at the same level as the footprint folder:

I hope the symbols, once in .sweet folders, will behave in a similar way.

If you mix them into the footprint repos it will become a lot more messier.

Once that is all set up one can start to write some part management tool to make the life of professional KiCAD users (I’m not :wink: ) a whole lot easier.