It's not only the general quiet in a well-isolated room that gets to you--though most of us are accustomed to having noise all around us, and it can surely be a shock to realize the extent of that, when one is temporarily "deprived" of it. But dead quiet is still within the range of normal experience. Most of us have "heard" it in everyday acoustical settings many times before.
The lack of any significant reverberation, on the other hand, is a much less common experience. If you've spent time in a desert or in deep snow when there is no wind, you get some part of it but if you make a sound in an anechoic chamber, the room tells you essentially nothing at all about itself. That's disorienting--we're used to a constant stream of auditory "feedback" about our surroundings.
If you bring a musical instrument into an anechoic room and play it, the effect can be devastating if you weren't expecting it--suddenly it feels as if there's just no point to anything, it all sounds so awful. The useful lesson in that is to realize how greatly the subjective experience of music depends on reverberant space of some kind.
The same is true of recording, and the experience of being in an anechoic chamber can help a person realize how broken the concept is of "aiming" a microphone at a sound source in a normal room. Even a directional microphone is something that you immerse in a complex field of sound energies coming from all directions at once. That's also why the off-axis response of a microphone is every bit as important as its on-axis response; only in an anechoic room is that not the case.
--best regards