This is because, at it’s core GIMP is a bitmap graphics program. In order for vector graphics to “fit” into the internal data model of GIMP it must be rasterized into a bitmap. This is simply a case of the wrong tool being used.
I think the issue is PDFs aren’t really designed for being easily editable. The PDF format is primarily targeted as being an output format, not an intermediary format. They provide for a lot of shortcuts for the sake of streamlined viewing reproduction that structures to make it easy to edit are (or can be) lost. If you want an intermediary format, SVG (as you have discovered) is a better fit.