I'm new to taping as well. I will agree that a good basic rig to start is a decent set of mics, a battery box/pre-amp, and a recorder. I use a Zoom H4N with Church Audio mics (CA-14 and CA-11) and a Church Audio 9200 pre-amp. But, I only do open taping, so I don't have to worry about a stealth rig. The Zoom H4N seems like it might be bulkier than some of the Sony models mentioned, and probably not as compact for stealth purposes. It also kills batteries, so I use an external USB battery for it. With that basic set up I've pulled some good tapes with some experimentation. I also have a hard drive full of shows that will never be heard by anyone. It's like learning to do anything though. First you suck and make mistakes, but you learn from them and get better.
There's lots of great information on this forum, and I learn something new every time I come here.