Sony PCM-M10
Here's why...
Runs for 24 hours or longer on a pair of AA batteries
Simple to use
Built in 4GB of memory, supplemented by microSD cards
Great level monitors (on screen scale, but also includes green light for levels above -12db, red light about 0db...perfect for stealth monitoring during a show)
Low cost at about $230US new price
Easily fits in your shirt pocket
Internal mics better than most handhelds (still not as good as externals though)
Nice on/off scheme...almost impossible to accidentally switch off during a show
Easy to build a larger rig around.
Negatives I can think of
MicroSD cards are tiny...easy to lose if you fill one up at the gig and need to change it out.
The Zoom H1 really wouldn't be the best decision, IMHO. My opinion is that it's designed more for the musician looking to capture idea's while they're rehearsing, or to capture rehearsals for quick playback.
The Sony M10 is really ideally suited for what most of us here do...recording live gigs. It's easily the most popular hand-held on this site for the last...oh...18 months or so (before then, the Edirol R-09HR was probably the most popular handheld).