That is bizarre and fascinating. Which decks have you noticed this with?
The speed of sound changes with temperature and humidity. So the speed of sound will be different in a club at the beginning of a show when people just walked in, vs later in the evening when people start to sweat and raise the temperature and humidity level.
If you're mixing 2 pairs of audience mics in the same spot, the differences will mainly be due to clock drift between the two recorders. But if you're mixing a video (speed of light) with audience mics, or a soundboard feed (close enough to the speed of light) with audience mics, you will have the issue of clock drift between the recorders and also the drifting speed of sound (which changes subtly) relative to the constant speed of light or electricity. Short answer, the varying speed of sound will make the drift between sources change over time, beyond that from clock drift between recorders alone.