Impedance is half of it, but the voltage for AES is 5v. The voltage for SPDIF is .5v, so even with the proper impedance, the SPDIF input will be getting 10 times the voltage the connector is designed for.
That said, the DR680 has settings which purport to adjust for this, but whether they adjust for the file format, impedance and voltage remains to be seen.
Thoughts?