Adding a comment on the Nyquist thing - there's another reason for higher sample rates. When doing the a>d, you must filter out all frequencies above Nyquist to avoid aliasing. So when we record at 44.1, nothing above 22kHz is allowed to hit the a>d. At the same time, we want frequencies up to 20 kHz. This abrupt filtering (no effect up to 20kHz, full blocking at 22kHz) is supposed to be quite challenging from a design standpoint.
A major advantage of going to something like 96k is that Nyquist is now 48k. Creating a filter that cuts everything above 48k while leaving everything up to 20k unaltered is supposed to be much easier to accomplish.
It's been written that much of what we don't like about the digital sound is a by-product of the required filtering vs. just being digital.
All theory...I haven't done any 96k recordings myself (or even played back someone elses).
I always record in 24 bit these days but haven't necessarily concluded the benefits are there. I like the idea, have the gear and figure it couldn't hurt, especially with respect to nailing the levels. One day I'll get a proper a/b test going but that's not so easy to do and would still only cover off one particalur a>d.