Most modern PCs have an electronic on/off button rather than a switch that interrupts the primary power. The PS has a 5V supply that's always on as long as the PS is plugged into the wall. This voltage runs the on/off logic and can also power the modem and network card if you have enabled the "Wake on ring" or "Wake on network activity" features. Some PS have a second switcher circuit just for this 5V standby supply. When you turn on the PC, the main switcher is turned on to provide the 250+ watts of power to run everything else. So unless you pull the plug, there's always something running inside.

A month ago I had to replace the batteries in my UPS and one had swelled up and refused to come out, so I had to unplug everything from the UPS and put it on the bench, take the cover off, and pry the battery out. New batteries got that going again but after plugging the computer in, it refused to turn on. The 5V standby supply had died. It probably hadn't ever lost input power in the last 10 years. I did find a couple of slightly bulging electrolytic caps inside the PS but these things are so tightly packed they're nearly impossible to work on and it was easier to buy another supply. The computer is a 15 year old Dell that has been working great and I'm going to hate moving to Win10 but there isn't much alternative.