'RIFA' capacitors should be connected from Live to Neutral, Live to Earth and Neutral to Earth. Removing them should cause no problems with the functionality of the power supply unit.
Sometimes you find capacitors (C) and inductors (L) in a circuit - sometimes in a little box. See (for example):
View attachment 1277130
View attachment 1277131
If you remove the inductors (or the entire box) - then the incoming mains supply is not connected to the power supply unit itself (unless you make these connections yourself).
However, you are now messing around with the mains supply = risk of black smoke and electrocution. So, unless you know what you are doing, don't do it...
Dave