Best places to buy parts are:
newegg.com and pricewatch.com
If you want to go with a PC then I would highly recommend going with an AMD processor and an Asus motherboard. Both of those sites offer AMD/Asus combo packages to save you some money. AMD processors are the alternative to Pentium and support nearly 100% of anything an authentic Pentium chipset runs. The advantage of AMD is the cheaper price and the higher reliability, life and overclockability..don't be fooled by the big name company, it's like buying Nike because of the name. I don't think I've ever known a custom PC builder who preffers pentium over AMD, but it's up to you. The price differance might be insignificant so go with what you like. Here's what you're going to need to build your custom pc:
1.) A case (you might be better off buying one with a power supply already installed to save some headache with wiring) - size is up to you, get one with an easily removable side panel though so you can get at the internals easily, saves tons of time and effort doing installs.
2.) Power supply (unless your case comes with one)
3.) Case fans - get at least 2 decent sized fans, some cases come with them installed also.
4.) Motherboard and Processor combo - you should buy these as a combo from a reputable dealer (like newegg.com). You'll want to buy a nice sized heatsink as well to keep the processor cool. Some places will sell the combo already installed and others will send them seperately. If they don't come installed have someone else install the processor. You need to press the processor onto the motherboard firmly and if done improperly you can ruin the processor or board. Sometimes people will not have the pins properly aligned and bend them by forcing the parts together or you can even ruin it by pressing too hard while installing and cracking the board. So that's why you should either get them pre-installed, have a friend whose competent of the task do it or take it to a computer store and have them do it. Have that same person mount the heatsink on the processor with thermal paste. If you've never done this before or watched it being done many times DON'T DO IT.
5.) System Memory aka RAM - you're going to have to look at the manufacturers website of the motherboard you buy (Asus for example if you go with an Asus board, which I'd highly recommend) to see what specific type of ram sticks your motherboard accepts. Buy a gig of ram if you can afford it, you'll be glad you did.
6.) Drives - I'd recommend buying 2 hard drives, one for the operating system and one for storage. All you need is a 20-30gig drive for windows xp if you can find one that small, it might be cheaper to get a 40, 50 or 60 gig drive though depending on the sales going on. Anywho, you install your operating system and any other programs and such on this drive ONLY, all saved data and music and such go on your large storage drive. This saves time and headache when you want to reformat your system since you don't need to back anything up. You have all your music and critical files on the storage drive so you only have to clean the windows drive off. Also be sure to get a 3.5" floppy disk drive and then your dvd burner, dvd reader, cdr or whatever else you want to install on your system. You NEED the 3.5" usually to get your system running so don't forget it, they're cheap anyway.
7.) Video, Sound, USB, Firewire, LAN, Networking...cards. Most newer motherboards have decent onboard (built in) sound and video processors but depending on what you're using the machine for you might want to install a better sound or video card. The motherboard probably already has USB and Firewire slots built in as well, but if you want more inputs you can buy a USB2.0/Firewire card and install it. You're going to either need a wireless networking or LAN card to connect to the internet, modem if you're on dialup. Be sure to again check the specs on your motherboard to see what kind of slots are available on the board. Newer motherboards should have the latest versions of PCI and AGP slots, so be sure to get cards made for the specific PCI/AGP speeds your board supports. As you can tell, most of what you buy is dependant on the motherboard you choose.
8.) Cables - The cables that come with drives are usually those flat grey pieces of crap that take up tons of space. You'll be very glad if you spend the extra $20-30 for ROUND cables to hook up all of your drives. Not only are the round cables much easier to install, they also keep your system much cooler by taking up less space which alows better air flow through the case. You're going to need 1 cable with 3 plugs that connects BOTH hard drives to your motherboard. Then depending on how many other drives you install you'll need cables to hook the rest of your gear up. If you install both a burner and a cd/dvd reader get 1 cable to hook both of them together as well, I'll explain why during the install instructions. Don't forget a nice cable for the floppy drive as well.
That's your shopping list, parts you buy are totally up to you. Be sure to look at your motherboards specs before you buy any ram or expansion cards. I'd recommend Serial ATA (SATA) hard drives over SCSI or EIDE drives since SATA is faster. EIDE used to be the standard so if you're not going for top speed in your machine you can probably save a few dollars going EIDE, but go SATA if you can. Just make sure that anything you buy is compatable with your machine and both hard drives must use the same connection (SATA or EIDE). If it says Pentium compatable it will work in an AMD system as well. You should buy the fastest processor you can afford so you don't get an itch to upgrade anytime too soon. But if this is just an internet machine and not going to be doing intensive audio/video work get something cheaper. Now I'll BRIEFLY explain how to put your new system together, I'd recommend looking up more detailed information subject by subject if you need too. If you know much about computers you should be able to do this all yourself.
How to assemble your new PC
(Read all of the instructions before you start removing anything from packages!)
1.) Install power supply and case fans in your new case. Make sure there are enough power cords for all of the hard drives and media drives you purchased. If there aren't enough you'll need to purchase Y-Adapters to split the power to the devices. If your case has a removable motherboard tray, remove it now. You can also remove the caps where your dvd/cd drives are going to be installed now to save some hassle later.
2.) Open up your motherboards manual and read the section on setting jumpers. There should be detailed information about jumper configuration for the initial bootup and for everyday use. This is a CRUCIAL step, if you do it wrong your PC won't work. You're not going to break anything if the jumpers aren't set properly but you will be kicking yourself in the head trying to figure out why your machine doesn't boot. You read the instructions and installed the jumpers? Now is a good time to run the power supply cables to the areas of the case where power will be needed. You can read a few steps ahead if you're not sure what I mean. You should run the power cables behind the motherboard so they don't get in the way of anything that moves. Sweet, go to step 3.
3.) Connect your motherboard (with processor already installed of course, since you did that before you started assembly anyway) to the motherboard tray in the case. Like I said in step 1, some cases have removable/slide out motherboard trays which makes it easier to install but if yours doesn't the tray will be built into the cases rack. There should be only 1 proper way to mount your motherboard, the way that positions the keyboard/mouse/monitor connectors at the rear of the case. The side of the motherboard with the processor and all the slots on it should be facing outward (towards you) if your confused still. Screw that motherboard in there tight enough so it doesn't jiggle at all and then connect the power supply to the motherboard as shown in your instruction manual.
4.) Install the System Memory (RAM) sticks into your motherboard. Try to handle these as little as possible and only touch the top and side edges. Avoid touching the connector if you can, you could kill your ram stick. If you don't know which way the stick faces when installed look in your motherboard manual again and follow instructions. Just be sure to have the stick perpendicular to the board when installing and firmly press it into place. Once it's in the ram slot it should be tight and won't fall out, so be sure you push it in all the way.
5.) Install the cards that you bought (video/sound/LAN/wireless networking/ect...) in the proper slots. Again, look in the motherboard manual to identify the proper slots for each device. They usually slide in snugly so just line the card up with the slot and push it firmly in place. You'll need to remove the slot cover where the input/output of the cards fits in the rear of your case which is usually done by breaking the metal piece loose or just unscrewing them if your case has them screwed on. Now screw the cards into place in the case rack and you're ready to move on.
6.) Remove your hard drives from the packages and open the manuals to Jumper Settings. Set the jumpers on your Windows drive (the smaller one) to Master. The storage drive jumpers should be set to Slave. Now mount the hard drives into your cases hard drive rack. I'd recommend putting the Windows drive (master) in the top space and the storage (slave) drive directly below it. Locate 2 power connector and connect them to the hard drives. Now find that 3 connector cable you bought for your hard drives and insert the end connector to your Master drive, the middle connector to your Slave drive and the end of the cable connects directly to your motherboards SATA or EIDE slot depending what kind of drives you bought.
7.) Install the 3.5" floppy drive now since the slot is usually kind of hard to get at. Screw it in place and connect a power supply cable to it. Now connect it to your motherboard with that nice round cable your bought.
8.) Now it's time to install your CD/DVD drives. If you're only installing one drive be sure to set the jumper on the back of the drive to MASTER (usually labelled with an M). If you have multiple drives to install the 2nd drive must have the jumper set to SLAVE (usually labelled with an S). I'd recommend using your CD/DVD reader as the MASTER device and the burner as the SLAVE device. It doesn't really matter which goes where, but having the reader in the top position in your case will make you less prone to using your burner to play cds and what not. I only use my burners for burning to maximize the life of the drive. Now that you have the jumpers set you just need to slide the drives into their positions in the case (master on top, slave on bottom again) and screw them into place. Locate 2 more power supply connectors and insert them into the drives. Now find that final nifty round cable you bought for these drives and again insert the end connector to the MASTER, middle to the SLAVE and insert the other end in your motherboards slot. If you bought a nice sound card there will probably be a small cable which you connect from the sound card to the drive you will be using for playing cds/dvds. There will only be 1 place which the connector will fit, so just insert it there.
9.) TAKE A BREAK. Unless you bought anything extra like a case temperature display or all kinds of neons and shit your computer should be assembled. Again, refer to the motherboard manual to do the next step.
10.) You now need to load the BIOS onto your motherboard. Some boards come pre-loaded with BIOS and others are not. Procedures for loading BIOS on a motherboard are specific to each style of board so you're going to need to read the manual. The reason you bought that 3.5" drive is to do BIOS installs and updates (and to run the windows recovery program if you ever need to) so you'll need to find the BIOS disk that came with your motherboard. Follow the instructions in the manual for installing BIOS if your board is not preloaded and be sure to complete every step. If the board was pre-loaded with the BIOS the jumpers were probably set to safety when you originally did the install. If you had to manually install the BIOS you'll most likely need to switch the motherboard jumper from the open setting to a safety setting. This is by far the most important part of the install! If you do this wrong your PC will just beep at you when you turn the power on, so follow the instructions and do it right the first time. Once you get the BIOS loaded on your motherboard, power your computer up again and enter the BIOS mode during bootup. You're going to want to set your BOOT PRIORITIES in this order:
Hard Drive (Master)
CD/DVD Drive (Master)
3.5" Drive
CD/DVD Drive (Slave)
Hard Drive (Slave)
Once you do this you can exit the BIOS menu and you're ready to get your new machine up and running! Insert the Windows XP disc into your Master cd/dvd drive and reset the machine. The computer should first try to boot from the empty hard drive and then it should move to the XP disc and begin the install. Complete the install and you've built your own machine!
I wrote this up myself just now so I might have left something out, but I tried to explain everything in plain english. If you have any further questions feel free to ask. You CAN do this yourself, so don't get intimidated. If you have problems during the install I'm sure there's plenty of folks who can help you out. Have fun!