I have never machined PCB material in any quantities that approached “production” volume. Any cutting, drilling, shaping, etc, has been a one-off effort for demonstration or prototype units, or modifying a handful of boards (no more than a dozen) so that an order could be shipped before a modified PCB design was received from a PCB fabricator.
Based on those experiences . . . . 5mm x 4mm sounds too small to be done safely! Even if I don’t need my fingers, my wife has many uses for them!
I would approach your problem by first conceding that some material - at least a millimeter; probably two, or even three - will be lost as a “machining allowance”. A scroll saw (like your photo) or a jig saw can cut PCB material. A few incarnations ago, my employer had a commercial table that let you mount a jigsaw under the table, in an inverted position. It worked well for cutting PCB material to size - you could set fences, stops, miter heads, etc to guide the cuts. There are at least dozens, perhaps hundreds, of blade styles available for jigsaws and scrollsaws - different tooth pitches and geometries, etc. There may be a few blade styles specifically intended for cutting fiberglass sheet stock - what styles have you tried?
If dimensional accuracy or quality of the edge finish are important, I’d cut the boards oversize and then trim to size using a piloted flush-trim bit in a woodworking router. Machine a metal template to the finished dimensions, temporarily attach your boards using double-sided “carpet tape” (or similar) and let the router bit’s pilot bearing ride against the template.
I have had good results using a sheet metal shear to cut PCB material, but I don’t recall any pieces as small as 4 or 5 mm. Perhaps a local HVAC shop has the experience to make it work. An old-fashioned office paper cutter - the kind with a heavy metal (or hardwood) bed, and a pivoted, knife-edge, cutting arm - is a poor-man’s substitute for the sheet metal shear. With a little muscle and a good karate yell it’ll work on 0.063" (1.6mm) board stock. But once again, your small dimensions don’t encourage optimism.