I was in the same position as the OP but I was on a limited budget and I got myself a Microtrack II on eBay for $99. The screen is a little dark and I thought it was flaky but ultimately it turns out it's just very picky about the CF card you use. Symptoms included jumping out of record, missing segments in the recording, and random noise bursts or clicks. I ended up getting the 16 gig Transcend 133X card that someone recommended on these forums (in the memory card roundup, which I can never @#$@%!$!! find, BTW!) and since then it's worked fine. I increased the contrast on the screen to max, making it usable. So YMMV, but it currently is the cheapest bit bucket that does 24-bit 96kHz. I'd have bought from someone here, had there been one available at the time.
For future reference, it looks likely that we'll be able to eventually use Apple iOS 4G+ devices like the iPhone or the iPod as bit buckets, but they may be limited to 48kHz 16bit. We'll know more over the next few months.