You never answered my question countless posts back about why additional fields aren’t sufficient and a first-class field would be. Nonetheless I think you would actually get somewhere by programming instead of writing reams. You might be able to use the experiences to justify a feature request.
Edit: So you want more support for the fields in the UI. This can be accomodated in a more general UI, and can be a separate request. It still pays to keep the framework generic. You may think that once your requirements are implemented, that’s the end of the matter. Unfortunately any specialisation of function will add baggage against a more general solution in future. Sometimes more becomes less in future.
The embedded Python scripting can also be used to implement some of the things you suggest. That’s why one powerful general feature is worth more than lots of small special features.
Nope. That’s not true. That is not what this feature request would be about. Put simply, this is what it would cover:
Add a part number field to every symbol in the stock libraries
Add a manufacturer field to every symbol in the stock libraries
Both of them blank by default
Add semantic functionality to support these two fields if used (I gave an example of this in another reply)
That’s basically it. I need to learn more about the link between the value field and the symbol name. I haven’t had the time to play with it but, if they are linked, this is a bad idea. Value --again, semantically-- has nothing to do with the symbol name. If they are connected all it is doing is confusing matters. I understand that it can be changed at the schematic level. If it cannot be different from the part name at the library level this needs to change and I would add it to the feature request or create a separate one (might even be bold enough to call it a bug).
And you are wrong. Just the addition of these two blank fields would be massively useful for all kinds of users, even professionals. The existence of these fields would allow for sensible treatment of BOMs and other very useful tools, some we might not even be able to imagine right now.
It’s a simple change with a huge positive outcome in terms of the out of the box experience, usability and standardization.
Anyone who doesn’t like it can do whatever they want. They can do that now. No difference to them. Huge positive difference for all others.
You’ll get even more pushback against this. Volunteer resources are stretched enough as it is. And it’s unnecessary to modify the stock libraries. You will be using your own database (when that’s implemented). So it will be your responsibility to import the stock libraries and populate the extra fields however you wish afterwards.
Edit: Even now, it’s not necessary to modify the library to have extra fields. The extra fields are specified in the per-user configuration, the eeschema.json file I programmed a script to modify a while back. IIRC there’s a sample Field4 in the stock configuration and I have added other fields for the purpose of generating BOMs while still using the stock libraries. So maybe you need to observe how the system works first.
Things like prices also do not have much use to add to the KiCad symbols themselves. The big stores do not like it if automated tools crawl their prices and they actively fight against it by constantly changing names and other small things that are invisible to users but break crawling tools. This has been tried with KiCost.
Prices also fluctuate too much to be very useful to add them to the symbols themselves.
I do see an advantage of a list with semantic names that have a special meaning, but that does not mean they have to be added by default to all KiCad schematic symbols. For example, I really see no reason for the “Datasheet” field to exist at all if it is empty.
For example a drop down list to add those fields if needed is a flexible system and it does not need to add empty fields to all the KiCad libraries. There are currently some 17000 schematic symbols in Kicad’s libraries, and adding a bunch of fields to all of them is nothing but a waste of space and makes KiCad slower.
Having an implementation that can work with a list of such names and semi automatically adds them to schematic symbols when needed is much more flexible then a static list added to those symbols, it avoids the clutter in the schematic symbol libraries themselves, It’s easier to extend and it can even be implemented in a way that is transparent to the users.
I’ve never figured out what the exact link is between symbol name and the Value field.
In older versions it was mandatory that they were the same on a library level, but could be changed in the schematic. I think this has already changed, but I’m not sure about that.
I agree that a hard link between these is a bug that should be fixed.
Nope. They do not have to exist at all.
For example: Schematic Editor / Tools / Edit Symbol Fields
Press the Add Field button, type a name and press [Enter]
You now have an extra column, and if you add data in that column, then the new field you created is added automatically to each schematic symbol for which you entered data.
Improving the tools in this way is a lot better then adding static fields to all the library symbols.
If the part number and manufacturer fields are filled you could:
Right-click on a component and trigger a web search of the manufacturer’s website
You have to go to the manufacturer website for:
Simulation models
3D models
Application notes
Samples
Technical drawings (example: a connector has a mechanical drawing and a datasheet and more)
Development boards
Equivalent components/substitution
Support
In the case of MCU’s, FPGA’s etc. development tools and support
Etc.
Right click on a component and perform a pricing search
You could have this be configurable in settings or through a JSON file
You could have it search your favorite vendor or something like Octopart
You could also have it launch an external script to do interesting things
For example, price the part at Mouser and Digikey and pull in the 1000 piece price into a custom field
Right click on the component and trigger an external script that looks it up in your database
Just three quick thoughts. One could do a lot more. However, these fields need to be semantically integrated into KiCad so that, if you use them --remember, I am saying they are blank by default-- the software and the user (through external applications) can make intelligent use of them.
At the most basic level the fields can be completely ignored and nothing whatsoever would change for any KiCad user today. No difference. Just two fields you never use. I seriously doubt most people will ignore them, BTW.
Next level is, you use them to store the most important information you need in order to actually build a board. And, as a side effect, you get to use a few new features that are enabled when the fields are populated.
The third level is a connection to external scripts that you either register to show-up on right click or can reach into the design via an API.
I am pretty sure all of this requires work at the core level, rather than writing a bunch of external Python code. I have not done enough work writing Python for KiCad to know this conclusively. I suspect I am right on a good portion of that assumption. In fact, I am about to launch into a more serious effort with Python/KiCad and need to post for help setting-up a solid development environment under Windows with Pycharm or VSCode (subject for a different discussion).
EDIT:
I’ll add one more to the list. If you have these two fields and they are accessible via the Python API, you could easily integrate these into a database driven process after populating the part number with your internal part number (or just use manufacturer and their part number) to pull in relevant data, instantly create and populate new fields in the schematic and do just about whatever the API allows in both schematic and PCB.
The key is to have these two fields as standard and fully-integrated fields. They can be blank and completely ignored by most. And yet, if sensibly integrated, they functionality they would enable and the experience they would provide would be a vast improvement over the current state of affairs.
Instant databased-driven functionality. Just two fields.
I have done this. This is not a solution. Read my other posts. And this does not improve the out of the box experience at all. For example, one can’t write standard add-ins to handle BOM creation using these fields. I had to go and manually create my own code to extract these into a BOM. That isn’t what a good out of the box experience should look like. And, of course, there’s a lot more to it than just the BOM.
I read back the first 20 or so posts and you are mixing up a few things.
The current KiCad libraries are not a fully specified database, and they probably will not become so in the near future. Adding some fields to them is not going to help.
The database, and the KiCad libraries are not the same thing.
a short piece from one of the first posts:
The current KiCad libraries are simply not fit to be used as the database you want, and adding some fields to schematic symbols is not going to change that.
It makes no sense to add empty fields to the KiCad schematic symbol libraries, and duplicating a resistor 100.000 times for the different manufacturers, types, values also does not make sense. The KiCad project simply does not have the resources to maintain such a database (yet…).
The only thing that makes sense to me is to have a separate database with metadata such as part numbers and have a mechanism to combine the “database” with the “libraries”.
Maybe you started too early with KiCad and should have waited for V7. Maybe you can help in improving the implementation for what this is going to look like in KiCad V7.
Such a database will also not become a part of KiCad, simply because nobody in their right mind would trust it, and would have to verify it for themselves anyway. Your company has spent countless hours creating, vetting and verifying their libraries.
The Semantics is a different thing. In the basic form it is only a list of agreed upon names with specific meanings. It is quite simple to write a script to match an existing and trusted database and combine that info with the (mostly graphical data) in the KiCad Libraries.
The KiCad libraries are not the database you wish for, and they will not become that database.
The KiCad libraries are only a building block of a greater part.
I’m guessing that the database driven part in KiCad V7 is going to have a reference implementation of combining an already existing (and vetted) database of “golden parts” with the graphics in KiCad’s current schematic symbol and footprint libraries, combined with tools to help with further verification and selecting. This way you only have to verify a very small part of the KiCad libraries, and then combine it with the already verified and trusted database.
I guess I don’t understand your goal. IMHO the schematic is not the place to control a production device BOM. And certainly not provide inventory quantities.
If I understand you correctly, you would need library entries for every part you could use, every resistor value and capacitor value. I don’t see how this is practical ( or possible).
What is your use case? Are you handed a fully detailed design with part numbers etc? Or are you designing “on the fly” (this is what I do).
Sorry, you are mischaracterizing what I’ve been saying and continuing along a line of discussion that is pointless for me to pursue. Unless you have something relevant and useful to say I am going to look past your comments from here forward. Other than that, thanks for participating. It has been truly enlightening.
The excerpt you quoted is an example of how overloading a field, like the value field, is a bad idea.
No, I have no interest in keeping inventory data in the schematic. That would be silly. The place for that is in a suitably designed and linked database or, at a higher level, ERP system.
I don’t think I have ever been handed a detailed design in 40 years. No, I typically design devices from scratch. That process typically starts with developing specifications, listing desired functionality, reviewing trades, suggesting devices or parts of interest, etc. An iterative approach follows this and eventually lands on electrical, mechanical, software and other specifications. From there you break out into the different disciplines and execute as needed, each one following a similar iterative pattern.
Depending on the type of project, a reduced or expanded version of the process you’ll find depicted on page 20 of this manual. For example, commercial projects like the one I am working on now don’t require this level of detail and rigidity. When I was working on a camera system to send to the moon (fingers crossed, in the next few years) it was pretty much at this level.
Sounds like you are not a programmer and don’t understand that Python is embedded in the KiCad application and has access to all the data structures, and is not “external” in the RAM domain sense. More research on your part is called for.
I don’t know. I’ve only been writing software for some 40 years? Machine language, Forth, C, C++, LISP, APL, Objective-C, PHP, Javascript, Python even Visual Basic! Embedded, desktop, web even a series of educational apps for kids on the iOS platform way back when. Oh, yes, and FPGA design in Verilog, which isn’t a programming language but, well, there. Hell, I ever published a paper on a programming system I developed for robotics back in the 80’s.
It’s the same idea as the JavaScript embedded in your web browser so you actually have come across the concept. BTW there is no Python API for the schematic editor yet, that’s another thing that was pushed back to 7.
So I reiterate, you can already do what you ask for in 6 and the rest in 7, you don’t need any special considerations.
You can add a field to symbols in the schematic editor, no need to touch the stock libraries. In fact having a couple of placeholder blank fields is a waste of effort since to be able to populate them you need to have write access to the library, so a writable copy has to be made. Then you have the problem of ensuring atomic updates, so eventually a DB is called for.
Python as a scripting language with the appropriate API hooks into the editor is a more flexible and sustainable way of doing the fancy things you suggest.
Also, Tim B-L isn’t god. The idea behind the semantic web are valid but a lot can be done by other means. (So much so that nowadays the concern is that The Man knows too much about you.) You do not have to have known field names. The field content can encode both more than just an identifier. You could attach a mime type, you could even have a snippet of JSON in it. What does it matter if the content isn’t a simple identifier in the database? It can be opaque and the Python scripts can decipher it.
Then too, having a part and manufacturer field is not good DB practice. You have redundancy and possible inconsistency. The part should be used as a key to look up the rest of the fields; that’s what joins are for. Looked at this way, the datasheet field becomes redundant.
So I say, a mininum of special meaning fields, let the user add more, and let all fields participate in “first-class citizen” actions whatever that means. But I would not go as far as just one UUID field.
@robomartin I know it isn’t always easy, but if you would’ve explained in your original post your vision and the benefit with your suggestion I strongly believe this thread had taken a completely different direction. I’ve read through the entire thread (although not carefully so I’ve probably missed a few things), and it took me until post 88 and post 92 to understand your suggestion.
I think you had us fooled by mentioning “out of the box experience” a number of times. That led me to believe that you suggested adding data to the standard libraries.
I can understand the benefit of adding the two fields if:
Left blank in the standard libraries.
At least some basic functionality relies upon these two fields.
I wouldn’t use them myself since I have fully atomic libraries with House Part Number in the Name field (and Value field since they are the same in the libraries). But I wound’t be opposed to adding this functionality into KiCad. I can see the benefit and I could easily ignore these two fields.
In every single symbol in our atomic libraries I’ve filled in the Datasheet field with a link to the component in our web-based product management system. That way it’s really easy for me to just select a symbol in the schematic and press “D” to get all it’s data directly in our system. Some of the things you’re suggesting is sort of similar to this.
You are right that the BOM export isn’t optimal and there is work planned on this. I’m not sure, but I believe that it’s planned for V7. Personally I love it like it is. I’ve written a Python script that gives me exactly the result our Product Management System wants and I never have to think about what’s being outputted. But I totally agree that the “out of the box experience” with the BOM export isn’t great.
But this is another topic. Improving the BOM export has nothing to do with the suggestion of adding the two fields. Improving this should be done anyway. I might add other fields to the symbols which I would also like the BOM to include, and IMHO the BOM export should have an easy way to include these.
I also disagree with the fact we need standardization here, mostly based on two facts:
everyone wants to name the part number field differently. some use abbreviations, some special naming etc. all of these would have to adjust this new mandatory field to their liking and would have to ignore an error thrown by the KLC checkers when handling their symbols.
not every part needs a part number. there are many parts which are not connected to components at all (testpoints, graphical elements, special layout components e.g. for RF), or components where you intentionally don’t want part numbers, either because they are only placeholders for later versions/other populating options or because they are wildcard components where it does not really matter which kind of 10k@1% resistor is placed there. especially the last point is getting more and more important with the shortages we see in all different components where you simply have to adapt your components on the fly to the market situation. for these cases you would get another empty field additionally to the datasheet one which can lead to different time consuming questions down the line of your production process.
I agree that having some standardized names makes it easier to improve BOM tools and other parts, and I think that is all you are trying to say. (In this ridiculously long thread).
Adding the empty names directly to the KiCad libraries is useless.
You’ve never given any response to the Idea of defining those semantic names as some sort of “automatic” variables and tools to work with them.
The database driven ideology is also never going to work without proper integration. If schematic symbols have part numbers but you can still change the value independent of the part number then stuff gets out of sink and data becomes unreliable.
At the moment I’m not even sure if we’re thinking along completely different wavelengths, or if you are deliberately ignoring most of the responses or even trolling. I do think you are using far to many words to describe simple things and apparently still are not able to communicate clearly. And from the other posts I read I’m not the only one thinking this.
I don’t think you’re wrong about adding these fields as defaults. Where you went wrong is thinking the developers care about what’s written here, or anything really that isn’t a well-written code contribution. All this typing is just spitting into the wind.
I only agree with you here, your previous sentence seems invalid to me. Beware wind changes direction sometimes.
Who here wants to create a library as the OP describes and maintain over time? Feel free to do so and share with Package and Content Manager.
Why do you think it is ok to have empty fields? If someone comes to the conclution that those fields are mandatory, then those can be created to cover the need… Could that be more simple? This is not KiCad’s inability to cope with extra fields, to me this looks kind of, someone else should do the dirty work for me.
OP since you know your way around software dev, help make this great program even better, a thread like this about two empty strings, could be also considered funny…
I appreciate your effort in reading through so many posts to understand. As I said at the very start, my intent was to get feedback as a precondition to proposing this as an official feature request in the right place. That, I got, along with a real sense --that evolved throughout the thread-- that the proposal needed clearer and more detailed communication. We’ve all fallen prey to this, I am sure. When you think the other person will surely understand what you are saying because it is obvious to you…and they don’t. Happens with my wife all the time!
One of the particularly troubling assumptions people seemed to make throughout this thread was that I was proposing that KiCad developers provide libraries populated with, I don’t know, millions of specific manufacturer part numbers and such detail. That would truly be crazy to suggest, yet people read that into my proposal.
The other one was that I was somehow suggesting that the value field had to be eliminated (or something like that). Not so.
Then there are people who clearly have no professional engineering experience who chose to engage with passion. This is great, but, honestly, at some point it is important to understand that lacking context it is difficult to make constructive contributions. When someone with a 40 year carees says “hey, this might be important” the right approach isn’t to attack the person, but rather to engage in a constructive conversation from which, perhaps, both might learn about each other’s perspectives. Sadly, the internet tends to bring out a counterproductive warrior mentality in some people. I’ve been around for a while, it was not very different from this in USENET days, so no big deal, I know the type.
Anyhow, I think this thread has served the stated purpose; to refine the communication of the proposal before making an official request. I’ll probably do that next week. I need to run a few more experiments with KiCad and Python in order to understand things a little better.
Right, and that’s OK. What I am looking at is that the raw out of the box experience with KiCad just isn’t good because important data is missing and there is no guidance as to where to put this data and how to handle it. The BOM is a joke. Without coding you can’t get it to do anything useful.
I think it could even go beyond that. There is no place for a component manufacturer to present their products. Let’s say, for example, that Yageo was interested in providing KiCad libraries for their entire product line. Where do they place their part numbers? Where do they store the values? The value field in the libraries is the same as the symbol name? How can Yageo --or anyone else for that matter-- delivery usable libraries that would enrich the KiCad ecosystem and experience.
If everyone is left to do as they please the thing turns into a tower of Babel situation with no baseline standards whatsoever.
This does not mean that you and I cannot do something else entirely. That’s the other thing people read into this conversation that I never actually said. This isn’t about imposing a rigid system, it is about fixing a broken out of the box experience due to the lack of sensible standards.
I think I will add the issue with the value field to my proposal. Sorry KiCad developers, this is silly. What I have heard so far is that it gets set to the symbol name automatically and the only place you can change it is within the schematic. Well, if I wanted to create a library of every capacitor offered by a company, where do I put the value? I can’t name the symbol with the value, that won’t work. If the value is linked to the symbol name at the library level it is no longer a value field (again, semantics matter). So now I have to ignore the provided value field and add my own “_value” field to make it useful.
I really think KiCad needs a bit of a rethink when it comes to the lack of a sensible starting point out of the box. Advanced users do not need this, they will, as you have and as I will, come up with a system that works. I really think this needs to be fixed.
And, BTW, if schematic does not yet have Python API access none of this can be done by users like me without forking KiCad and bifurcating the ecosystem.
