I'm with page. I can't hear a difference in critical listening tests between my 96 kHz vs 48kHz recordings of unamplified acoustic material where higher sample rates may actually catch additional angel harp harmonics and use less steep highpass filters on them than those required with lower sampling rates (which is the one aspect for which I can entertain the idea of a plausible agrument for an audible difference) . Yet I also don't make much of an effort to persuade others to listen closely and make their own conclusions on it because it causes no harm other than consuming twice as much storage space, so err'ing on the side of caution is harmless and perhaps wise as long as storage space isn't an issue.
However in the practical world, storage space becomes a very real problem.
Just to nip one persistant misunderstanding about higher sample rates, understand that by recording at a higher rate you do not record the frequency range that would be covered by a lower sampling rate at a 'higher resolution', you only extend the upper range of frequencies that can potentially be recorded. Actually recording those ultrasonic frequencies requires that they are present to begin with and that your equipment is sensitive to them and can transmit them to the recorder. Maybe they are and maybe it does, maybe not. At least you can look for them when analyzing the files in DAW software to visually confirm if there is anything up there at all.. though even if you see something up there it may only be noise.
My personal assement and sugestion goes something like this: I always record 24 bits. The improvement in dynamic headroom over recording 16 bits is real at the recording end and allows freedom in setting levels and not clipping. That very real recording end improvement over recording 16 bits increases my file sizes by 50%. I can live with that. My equipment will record at 96 or even 192 kHz but even for pristine on-stage acoustic matierial I don't use those rates. Once you've amassed a personal recorded library of several Terabytes, keeping the library managable and redundantly backed up becomes a real practical challenge. As my recording techniques have evolved to multi-channel, I REALLY don't even consider using higher rates for the close acoustic stuff- the files for a single show would double in size if I switched from 48 to 96 kHz, and for some admittedly overblown efforts that have used like 11 or 13 channels that would mean going from something like 15-20 GB of raw data to 30-40GB for a single show for the raw audio files alone, much less any photos or video.. and for more typical stuff it means going from something like 30-40 GB of raw file storage for a music festival of 4-channel recordings to twice that. Stick with the hobby for long and you'll amass large amounts of data more much rapidly than you think.
When it comes down to brass tacks, consider that nearly absolutely everything else you do will have a far, far larger sonic concequences (your recording location, the recording venue, the stage / FOH setup, mics, connnections, your processing choices) and so it's really a moot point that doesn't merit much argument eitherway.
For all the technical talk, good recordings mostly boil down to getting the practical stuff right and making sure things work and do so smoothly. Workflow practicalities become far more important than the last 99th percentile of perfection. Most improvement comes from concentrating on mics, choice of mic array and recording position.. and leaning your own ways of working smoothly to record without stress or hassle.
I'm glad I listened and convinced myself with good equipment that it didn't make any audible difference to my ear because the size of my library really does matter. A lot!