In addition to losing capacitance, sometimes you want the damping effect of capacitor ESR. Something like a 470 uF capacitor with 20 milliohms of ESR (making up some reasonable numbers) looks like a resistor at frequencies over 17 KHz. This can be very helpful in stabilizing the feedback loop of a switching voltage regulator. Some older linear regulators required tantalum or aluminum electrolytics and might oscillate with the low ESR of a modern ceramic capacitor.
Often times you just want more capacitance than is practical with ceramic capacitors. Also; I notice that there are few if any new low voltage ceramic capacitors in sizes larger than 1210 for example. I think there are cracking issues with large ceramic capacitors with thin layers for low voltage.
You can add resistor in serie with ceramic capacitor. You have better control of ESR you get than when using electrolytic.
I use electrolytic only at high voltage (12,24,48 are for me high voltage).
I step down from 48V (with overvoltage up to 100) directly to 3V3 using LM5017. That IC needs enough ripple at output to work (so electrolytic). Instead of using electrolytic or adding resistor in serie with ceramic I use circuit (type 3, page 13 in datasheet) to put needed ripple into feedback pin and get no ripple at output (thanks to ceramic capacitor).