The CvPCB workflow is at cross purposes with atomic parts.
If the device is available from its manufacturer in different colors (such as pushbutton switches with built-in LEDs), the manufacturer gives each variant a different part number.
At the least, your schematic symbol ("parts") library must have different entries for each variant you wish to use. If your part symbol includes a (custom) Part Number field, then that field must be different for the variants.
Alternatively, you can have just one symbol for the "family" of these parts, and then the Part Number field you create includes a "family" part number. Make the VALUE field the variant select. In the case of different-color LEDs in the switches, the value can be RED, GREEN, YELLOW, whatever. Some simple post-processing of the BOM can be used to combine the "family" part number with the VALUE field to give you a correct manufacturer's part number.