What I usually do is plot to SVG from KiCad and do all the vector stuff in a dedicated vector editor (inkscape, illustrator, etc). Dimensions usually get messed up. In the vector editor, set your documents units to cm/inches in the size of your PCB. And scale the imported SVG layers accordingly (group them so you can scale all at once). If your units are setup correctly, adjusting dimension, origin etc is a breeze and easily done by hand. Hide/delete the layers you're not interested in.
Use the vector tool to convert the holes into shapes/outlines (sometimes svg line-width is messed up - adjust to the real diameter). Then, use the path-edit tools to combine / subtract / merge the hole utlines and the edge-cuts into one shape.
I use this technique to create "artsy" silkscreen layers outside of KiCad and import them in one go as one image file - and get perfectly matching dimensions.