Well for me, i save the money for harddisk, cdrom and floppy disk.
I bought a low end motherboard ASUS P4MX533 with PXE bootup enabled. All the devices such as network card, sound, and video are builtin. All casing, monitor, mouse and keyboards are very very cheap type but it works fine. I did all the wiring myself as well.
It's a P4 and running on 128 MB of RAM. No problem, running very smooth for me
It really saves a lot of money comparing to buying expensive machines.
However i do admit that the server requires a high processing power. But, Linux gurus proved that everything works fine on a PIII 1GHZ with a 256 MB of RAM as a diskless server.