Small recorders with the mics pointing "up" from the top of the recorder include
Olympus LS-10
Marantz PMD620
Sony PCM-D1 and PCM-D50
Nagra ARES-M
Zoom H4
Recorders with the mics pointing "sideways" include
Edirol R-09 and R-09HR
Tascam DR-1
M-Audio Microtrack 2496 and II (using the T-mic)
These recorders have omni or cardioid mic capsules, so they do
record sounds from all around. In my experience, the recorders
work best when you point the top of the recorder at the subject
you're recording, sort of like the way you use a TV remote
control.
Recorders with mics pointing "forward"
Zoom H2 (forward and backward)
Yamaha Pocketrak 2G (mics pivot up or forward)
Only a couple of these recorders are tall enough to have the
mics sticking out above the top of a typical shirt pocket. If
carried in a pocket, the recorder will pick up a lot of noise
from the movement of the shirt fabric.
I bought this flexible mini-tripod for $8 on eBay
http://cgi.ebay.com/275g-Camera-Flexible-Gripping-165x35x35mm-Mini-Tripod_W0QQitemZ130238916909QQihZ003QQFlintstone