It's like to be either the CPU, Motherboard or PSU - So i'd try and get hold of either those components to try and whittle it down further.
I had something similar happen a few years back where the CPU fan would rotate a few times before cutting out, it turned out to be the PSU.

I'm guessing you don't have any indicators on the motherboard? As most boards now have lights or a 7-segment display displaying error codes, or beep error codes on the internal speaker.

If you've removed everything and tried to turn it on it'll be board or power supply.

When you say you've shorted the pins I assume you mean the ones on the motherboard?

Try taking the main power connector out and shorting the green with a black, I normally use a paper clip.

If the power supply spins up it's probably the board although that's not 100% conclusive.

Yeah, take the battery out. Reset the Cmos etc...

Take the battery out, and power lead, press power button, all lights should go off on the motherboard.

Plug power lead back in and battery back in, should be ok.

Alternatively there should be a jumper on the mobo to reset the cmos

you could try resetting the cmos battery... just take it out for 10seconds then pop it back in and boot it up... otherwise it sounds like faulty motherboard or psu, in which case to test you would need another psu and or motherboard.

Power supply itself maybe fooked.

I am really stumped.

When I plug the power cable in, press the power button, the PC does not start up, but the CPU fan does spin about 5 times (not enough to make any sound though).

The PC simply does not boot.

I have tried the following:

removing/swapping the ram
different power cable
unplugging everything else
manually shorting the 'power on' pins

and I don not have a spare PSU I can check it with :(

Anyone have any ideas?