I will second the Sony PCM-M10. It is a great recorder with long battery life, good onboard omni microphones and a very nice preamp for use with external microphones.
It does not work as a computer microphone.
That said, for your main stated purpose it will do you well. One thing to consider is that the PCM-M10 has internal omni-directional microphones, all noise in the area will be picked up, not just what is in front of you. For many foley applications, you may be better served with microphones with a tighter pattern. (less side and rear pickup) In this situation cardioid or hyper/super cardioid mics will be useful. Some handheld recorders have a stereo pair of cardioid mics that will capture more sound in front of the recorder than to the back.
For most of us on this board, we use separate mics, choosing the pattern that fits the needs of the situation, in this case an M10 will do well. However the M10 does not have the power supply to power many mics (phantom power) so you will either need a separate power supply, such as the denecke PS-2 (used by many field videographers) or choose mics that will work with PIP (plug in power) a lower voltage power supply that the M10 does provide.