Kicost variable type error

O_Shovah · April 18, 2016, 3:40pm

@devbisme ,Recently i started using kicost for a larger PCB project as i found its features to be really nice for keeping track of the projects pricing for production. So thank you very much devbisme for starting its development.

Unfortunately i ran into a repeating command line error when executing kicost on a xml file.
It seems its an error with a variable type or type conversion in the python program or python
itsself. But im lacking the knowledge for more details.

I have used the test file you provided in this thread: Kicost command line error

My system currently is:

Ubuntu 14.04.3
Kicost 0.21
Python 2.7.10

And the command line output reads:

MyPc@Room-W4:~/Downloads$ kicost -d 1 -i test.xml
Get schematic XML…
Get parts library…
Get components…
Get groups of identical components…
Create HTML page for parts with custom pricing…
Scrape part data for each component group…
No ‘newark#’ or ‘manf#’ field: cannot lookup part [‘PM2’, ‘PM3’] at newark
No ‘digikey#’ or ‘manf#’ field: cannot lookup part [‘PM2’, ‘PM3’] at digikey

---------> Not finding a lot of parts due to missing “#manf” entry--------->

No ‘mouser#’ or ‘manf#’ field: cannot lookup part [‘JP9’, ‘JP8’, ‘JP5’, ‘JP7’, ‘JP6’, ‘JP10’, ‘JP4’, ‘TP1’] at mouser
Traceback (most recent call last):
File “/usr/local/bin/kicost”, line 9, in
load_entry_point(‘kicost==0.1.21’, ‘console_scripts’, ‘kicost’)()
File “/usr/local/lib/python2.7/dist-packages/kicost-0.1.21-py2.7.egg/kicost/main.py”, line 132, in main
num_processes=num_processes)
File “/usr/local/lib/python2.7/dist-packages/kicost-0.1.21-py2.7.egg/kicost/kicost.py”, line 133, in kicost
for id, url, part_num, price_tiers, qty_avail in results:
File “/usr/lib/python2.7/multiprocessing/pool.py”, line 659, in next
raise value
TypeError: ord() expected string of length 1, but int found

Can you or somebody else please point out to me where i went wrong, or if my local python installation
messed up in any way ?

Best regards

Jens

devbisme · April 18, 2016, 9:50pm

Hi, Jens. I tried running kicost 0.1.21 against the test.xml file on my Windows 7 machine and I didn’t get the error that you do. Let’s do the following:

Update kicost to the newest version (pip install -U kicost).
Run kicost in single-process mode with full debug on: kicost -i test.xml -w -s -d 3

Let me know if you’re still getting the error. If so, I’ll start an Ubuntu VM and try it there.

O_Shovah · April 20, 2016, 8:32am

Hello devbisme,
Thank you for you quick reply.
I reinstalled kicost using pip install -U kicost.
An error got surpassed by installing “sudo apt-get install libxml2-dev libxslt-dev”

I also installed BeautifulSoup-3.2.1 and beautifulsoup4-4.4.1 using pip.

Yet when running “kicost -i test.xml -w -s -d 3” i get another but longer error message as attached below:

`MyPc@R2079-W4:~/Downloads$ sudo kicost -i test.xml -w -s -d 3
Get schematic XML…
Get parts library…
Get components…
Get groups of identical components…
Create HTML page for parts with custom pricing…

RS-282G05A3-SM RT

1:0.10;10:0.03

http://www.xess.com

MMM001

1:0.10;10:0.08;100:0.05;1000:0.03;10000:0.01

#21CE8AE097085032

1:1;10:0.5;100:0.25;1000:0.12

http://www.xess.com

{u'digikey': {u'function': u'digikey', u'label': u'Digi-Key', u'order_cols': [u'purch', u'part_num', u'refs'], u'order_delimiter': u',', u'scrape': u'web'}, u'local': {u'function': u'local', u'label': u'local', u'order_cols': [u'purch', u'part_num', u'refs'], u'order_delimiter': u'', u'scrape': u'local'}, u'mine': {u'function': u'local', u'label': u'mine', u'order_cols': [u'purch', u'part_num', u'refs'], u'order_delimiter': u'', u'scrape': u'local'}, u'mouser': {u'function': u'mouser', u'label': u'Mouser', u'order_cols': [u'part_num', u'purch', u'refs'], u'order_delimiter': u' ', u'scrape': u'web'}, u'newark': {u'function': u'newark', u'label': u'Newark', u'order_cols': [u'part_num', u'purch', u'refs'], u'order_delimiter': u',', u'scrape': u'web'}} Scrape part data for each component group... Progress: 0%| | 0/17 [00:00<?, ?part/s]newark ['PM2', 'PM3'] No 'newark#' or 'manf#' field: cannot lookup part ['PM2', 'PM3'] at newark digikey ['PM2', 'PM3'] No 'digikey#' or 'manf#' field: cannot lookup part ['PM2', 'PM3'] at digikey local ['PM2', 'PM3'] No 'local#' or 'manf#' field: cannot lookup part ['PM2', 'PM3'] at local mine ['PM2', 'PM3'] No 'mine#' or 'manf#' field: cannot lookup part ['PM2', 'PM3'] at mine mouser ['PM2', 'PM3'] No 'mouser#' or 'manf#' field: cannot lookup part ['PM2', 'PM3'] at mouser newark ['R1'] Traceback (most recent call last): File "/usr/local/bin/kicost", line 9, in load_entry_point('kicost==0.1.23', 'console_scripts', 'kicost')() File "/usr/local/lib/python2.7/dist-packages/kicost/__main__.py", line 159, in main variant=args.variant, num_processes=num_processes) File "/usr/local/lib/python2.7/dist-packages/kicost/kicost.py", line 129, in kicost id, url, part_num, price_tiers, qty_avail = scrape_part(args) File "/usr/local/lib/python2.7/dist-packages/kicost/kicost.py", line 1842, in scrape_part html_tree, url[d] = get_part_html_tree(part, d, distributor_dict, local_html, scrape_logger) File "/usr/local/lib/python2.7/dist-packages/kicost/kicost.py", line 1807, in get_part_html_tree return get_dist_part_html_tree(dist, part.fields[key], extra_search_terms) File "/usr/local/lib/python2.7/dist-packages/kicost/kicost.py", line 1720, in get_newark_part_html_tree tree = BeautifulSoup(html, 'lxml') File "build/bdist.linux-x86_64/egg/bs4/__init__.py", line 212, in __init__ File "build/bdist.linux-x86_64/egg/bs4/builder/_lxml.py", line 109, in prepare_markup File "build/bdist.linux-x86_64/egg/bs4/dammit.py", line 263, in encodings File "build/bdist.linux-x86_64/egg/bs4/dammit.py", line 33, in chardet_dammit File "/usr/lib/python2.7/dist-packages/chardet/__init__.py", line 24, in detect u.feed(aBuf) File "/usr/lib/python2.7/dist-packages/chardet/universaldetector.py", line 115, in feed if prober.feed(aBuf) == constants.eFoundIt: File "/usr/lib/python2.7/dist-packages/chardet/charsetgroupprober.py", line 59, in feed st = prober.feed(aBuf) File "/usr/lib/python2.7/dist-packages/chardet/utf8prober.py", line 52, in feed codingState = self._mCodingSM.next_state(c) File "/usr/lib/python2.7/dist-packages/chardet/codingstatemachine.py", line 43, in next_state byteCls = self._mModel['classTable'][ord(c)] TypeError: ord() expected string of length 1, but int found

It seems as some hickup with the beautifulsoap package but the latest versions seem to be installed.
Any other ideas ?

( Btw is there any way to insert code in a small srcollable window format?)

Joan_Sparky · April 20, 2016, 11:52am

code looks like this
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. (yes, scroll down please)
.
.
and you just need to select all that needs to be formatted and hit the code button in the reply window (next to the quote button)

…unfortunately the window isn’t particularly small nor is it possible to use 3 backticks as per this guide to create it
http://commonmark.org/help/
(just tested it and only got single line code instead of a window)
Maybe it’s some setting(s) in the forums code, but that would be @ChrisGammell 's job

devbisme · April 21, 2016, 9:49pm

I replicated your exact error on Ubuntu 14.04. Here’s how I fixed it:

apt-get install libxml2-dev libxslt1-dev python-dev zlib1g-dev
pip install -U kicost

The reason for the call to apt-get is to install the libraries that are needed to compile the lxml package used by kicost. I found using pre-compiled versions of lxml led to the problem with kicost. The problem went away once I rebuilt from source.

O_Shovah · April 27, 2016, 7:01am

Sorry for my late reply, but i’m currently pretty busy.

I tried executing the commands you proposed but got the same
error message as before so far.

By rebuilding lxml you think of
doing a local compilation of the lxml source code ?

I will try that later today. Thank you

Tim_Hoeppner · May 7, 2016, 8:03pm

I’m getting the same error as well even after trying your latest suggestion.

Shack · March 30, 2017, 10:15pm

currently Im getting:
Traceback (most recent call last):
File “/usr/local/bin/kicost”, line 11, in
load_entry_point(‘kicost==0.1.33’, ‘console_scripts’, ‘kicost’)()
File “/usr/local/lib/python2.7/dist-packages/kicost-0.1.33-py2.7.egg/kicost/main.py”, line 172, in main
exclude_dist_list=args.exclude, include_dist_list=args.include)
File “/usr/local/lib/python2.7/dist-packages/kicost-0.1.33-py2.7.egg/kicost/kicost.py”, line 218, in kicost
id, url, part_num, price_tiers, qty_avail = result.get()
File “/usr/lib/python2.7/multiprocessing/pool.py”, line 567, in get
raise self._value
UnboundLocalError: local variable ‘ap_trees_and_urls’ referenced before assignment

only getting it with larger BOM’s so if I remove most of the XML file it works fine.
Im running ubuntu 16.10, Kicad V4.0.5 and Kicost V0.1.33

any help would be very appreciated

devbisme · March 30, 2017, 10:48pm

Shack, please send me (devb@xess.com) an XML file that you’ve verified causes this error and I’ll check it out.

Is the error consistent (i.e., it always happens) or transient?

Have you tried running KiCost in single process mode?

Shack · March 30, 2017, 10:57pm

Yes it’s consistent

I tried “kicost -i test.xml -w -s -d 3” and “kicost -i test.xml” with my one files… Both crashes

I’ll send you the file:)
EDIT: file has been send

Shack · March 30, 2017, 11:10pm

Another thing which I find kinda weird is the position of the boardQTY cell in the xlsx file. The placement is quite far to the right meaning I need to move it to the left for all the values to see it as declared

devbisme · March 31, 2017, 3:17am

I’m not seeing the file.

Shack · March 31, 2017, 4:41am

Have been send tried sending it in a PM

devbisme · March 31, 2017, 1:52pm

The problem is triggered by component R51 when it’s searched for on Digi-Key. The part number in the XML is 1-2176057-7 and Digi returns the appropriate web page for that part. But the part number listed on that page is RLP73M3AR047FTDF. That makes KiCost lose its shit and crash. The near-term solution is to change the part number in your schematic/XML. The final solution is for me to fix KiCost.

Shack · March 31, 2017, 2:03pm

thank you for the fix! ofcourse I just finished my manual made version 20 minutes ago

devbisme · March 31, 2017, 9:36pm

The problem is fixed so it will be ready if you ever need it again. Version 0.1.34.

Shack · March 31, 2017, 9:45pm

Thank you so much