The WRT54g is a pretty popular wireless router although if it is at all possible, use cables for the desktops. the cables are much cheaper than buying wireless NICs for the 2 machines and also much more stable. The WRT54g works like a regular router in that it has 4 ports to plug into on the back as well as being able to serve wirelessly.
I would get the linksys. They are proven to work well and are owned by Cisco, a company that does professional large scale networking. Belkin, D-Link, etc may be more hit and miss in terms of quality.
Depending on your connection to the internet, you will not feel the 35% increase in speed as the "g" protocol offers 54Mb bandwidth, which is significantly higher than most ISPs offer.