No, if you're downsampling then you MUST filter first (or simultaneously), and there's no special, more straightforward approach that applies to simple integer ratios. You might just as well go from 96 kHz to 44.1 kHz (or any other, arbitrary rate, even if it's not an integer!) as to 48 kHz. There is no gain or loss in efficiency, accuracy, dynamic range, distortion, ethical purity, sexual desirability, followers on social media, or anything else either way.
If you're upsampling, then part of the code could be made simpler for the special case of integer ratios of sampling rates. However, no actual sampling rate conversion hardware OR software that I've ever seen, smelled, tasted, or heard of uses such an approach. I'm trying and failing to find a suitable metaphor to explain why this is, since metaphors seem to convince people where actual descriptions of reality do not.
Yes, those special cases would be easier to program than the general case. But the writers of the software or firmware would STILL need to support the general case as well, test it, AND THEN have the "easier" way standing by for whatever percentage of cases in which it could be used. They'd also have to build in additional logic to detect the special cases that could use the simpler code and switch over to it. But that's a worse situation for the software or firmware developer, not a better one. Yes? The inconvenience and risk that they would have to take on in order to code, test and integrate their implementation of the "easier" approach, would come in addition to the inconvenience and risk of handling the more general case that they have to handle anyway.
So no one that I've ever heard of, or can realistically imagine, does that when writing sample-rate conversion software.