The op doesn’t say which IC he’s using but it looks like an I2C port expander so I’ll assume it’s the MCP23017. According to the datasheet the max Vil (low input voltage) is 0.15Vdd, assuming Vdd is 3.3V this is 495mV. Across a 10K resistor this amounts to about 50uA. While the min Vih (high input voltage) is 0.25Vdd + 0.8V, for Vdd = 3.3V this is 1.625V leaving 1.675V of drop across a pull-up resistor. For a 10K resistor this is about 168uA. While both of these currents are well below the input current of the IC, using pull-ups instead of pull-downs offers three times the noise immunity.
Also, with all three jumpers installed there are three 10K resistors in parallel, or 3.3K, between power and ground, not ideal in a power sensitive application. One solution to this is to increase the resistor sizes, perhaps to 100K. This would work as a pull-up but would be approaching the limits of a pull-down, not to mention the increase in noise sensitivity. Another solution would be to install only the pull-up or pull-down but not both. So where you install the jumper to ground remove the pull-up.