Because you're running through both the UA5's ADC and the iRiver's ADC, you need to set your levels twice: once for the UA5, and again for the iRiver. If you clip at either ADC stage, it will have a negative impact. If you're too low at either stage, it will have a negative impact.
As others have said, set your UA5 levels using the "Peak" light on the UA5. If memory serves, it lights up at -3 dBFS. If it flashes only occasionally, you're in good shape for setting levels for the UA5. (Make sure phantom is turned ON, of course.)
The next step, then, is to set levels for the iRiver. First, confirm you're running Line In. You'll control levels through two mechanisms: <1> the Output knob (above the Phones jack) on the UA5, and <2> the gain settings on the iRiver. Ideally, you want the iRiver as close to unity gain as possible, controlling the levels on the iRiver using the UA5's Output knob. But if that doesn't provide good enough levels on the iRiver, you'll want to add gain using the iRiver, too.
If your levels are still too low, as others have said: check to confirm the pads on your mics are OFF, and you may even have to run Mic In (though that seems unlikely). But for loud music, that really ought not be necessary.