I have an RH1 and it works just fine. Yes, there is the advantage of removable media. You can plug stereo mics into "mic-in" or use a battery box and go in through "line-in" for those really loud shows. The user manual will cover that clearly. Line-in has no pre-amp which you will not need for the ears bleeding shows.
The folks who reflexively bad-mouth MD's have usually not had experience with the Hi-MD units. As for the mics, I was warned about SONY mics not having a good bottom end. Well, I have an ECM-MS957 which does pretty well. I do prefer the AT and Sennheiser based mics I am using but there are times that a stereo mic is a good thing. Microphone Madness and Sound Professionals sell good, reasonably priced mics. Chris Church also makes mics but I found him rude and hard to deal with. Fortunately lots of other sources exist for mics. Some folks have had the same experience with Chris I had; other think he is swell. I have not yet heard a complaint or had a problem dealing with MM or SP.
Summing up I would say that you can use the RH1 as a platform: it works well. Generally, SONY's mics are not the most musical.
Welcome aboard, recording is a lot of fun. Learning where to place the mics for acoustic music is the art. Standing in front of the speakers at a rock show is easy. ;o)
Cheers
PS - the RH1 is a great MP3 player, too.