A/D converters have a small but definite transit time, depending largely on how their filters are implemented. Two different models or brands of converter might have different transit times, so even if you are using a common word clock to synch them, they might be a sample or two apart from each other by the time your recorder receives their output.
On the other hand, as I was writing this, it occurred to me that my standard setup for 3- and 4-mike recording is vulnerable to the same effect. I've never noticed any problem--most likely because the third mike (or the second pair of mikes) isn't coincident with the main pair, and the different arrival times of sound at the microphones is far greater than any difference in latency that is likely to exist among different converters. Unless you're working with an ambisonic-type or "SoundField"-type microphone, this is probably not worth worrying about.
--best regards