Maybe, but I wouldn't do it, at least not the R4. A UA-5, yes. Maybe the MMP, although I honestly don't know enough about them to comment.
It's not so much the 8.4volts versus 9.0 that concerns me. I think the 8.4 is close enough. The other part of the equation that's not mentioned is the intended current draw. When an R4 boots, and the hard drive starts, they pull quite a bit of current, and if you happen to have mics attached with the phantom power switch on when you start the rig, more so. The factory power supply is rated for something like 2 amps. I suspect those little things are not designed to carry that load... either the voltage will dip, or you will pop a little internal breaker.
Another thing is this... batteries hold change in watts... These milli-amp hour ratings are relevant for a fixed voltage battery pack, but in this case with the battery switchable for various voltages, I don't think you can assume it is 4000mah at 8.4v. I suspect this packaging marked 4000 mah is only correct at 3V. If you increase voltage by a factor of 2.8 (to 8.4 v), then you run time will go down by a factor of 2.8, so it will run an R4 for 1.3 hours, not 4 hours.
My 2 cents worth. Probably great for running an iRiver, possibly a Korg MR-1, but not an R4 with an upgraded hard drive and 4 channels of phantom power.