What you describe is exactly what I do.
Here is the main point: It requires maintaining your own libraries.
Every component in your schematic libraries MUST have the "Footprint" field populated with a footprint that exists in one of your footprint libraries.
So rather than having a generic resistor symbol in your schematic library, you will likely have something like RES_0805 and RES_0603. And instead of a generic op-amp symbol, you will likely have something like OPA552UA, which is in an SOIC-8 and OPA552PA, which is in a DIP-8. And so forth. (And you can take this a step further, and in the schematic component you can define a custom field called "MFR PN" or something similar, and populate it with the manufacturer's part number, so your BOM generation is simplified.)
If you search here and elsewhere, you'll see the holy war between those who believe that placing complete components (with footprint and 3D model known) onto the schematic is the right way to design, and placing a symbol and then back-annotating a footprint later is the right way. I fall into the former category.
In the workflow I advocate, you would never do that. You'd go back to the schematic, change the component from an R_0603 to an R_0402, regenerate the netlist, reimport into pcbnew, and voila, your layout is updated with the correct footprint AND your schematic matches what's in the layout.
Right, there is a lot more cutting and pasting than creation of new stuff. For example, you create the SOIC-8 footprint exactly once, mate it with a 3D model exactly once, and you can use that footprint for a whole host of different parts.
Similarly, if you have a symbol for a dual op-amp, you can simply clone it and change the name for each op-amp you use.
And the thing everyone should hopefully realize is this: most users don't have a million parts in their libraries. They only have the parts they use or would consider using. Thus the library size remains manageable.