People: AES/EBU and S/P-DIF differ as to voltage, characteristic impedance, connector types, and balanced vs. unbalanced cabling. But they also differ slightly in the actual content of their signals. A few "flag bits" (important ones) are defined differently between the two signal formats.
So even if you can overcome, or get away with ignoring, the issues of voltage, characteristic impedance, connector type and balanced vs. unbalanced cabling in a given instance, the two signals as signals are still slightly different from one another.
Thus you should realize that you are basically lucky if you can connect a device with one type of output to a device of the other type of input and have it work at all, let alone work correctly and reliably. And that luck isn't necessarily transferable to other pairs of devices. Nor can you reason, or argue, or whine your way into making it work if it's not going to work.
Basically, the way to make it work reliably is either (a) to stick with one standard--connect AES/EBU sources to AES/EBU inputs, and S/P-DIF sources to S/P-DIF inputs, or else (b) use an adapter that takes apart the digital bitstream from the one format and generates a correctly formatted signal in the other format. Obviously such a device (such as the old Digital Domain FCN-1) would need to be electrically active and would need its own power supply; a passive device such as an adapter cable or a transformer can't do that.
Always remember: There are professional audio standards, and there are consumer audio standards, but there are no standards for connecting professional audio equipment to consumer audio equipment.