I've not listened, but..
The non cassette source is almost certainly the steady one, use that as the constant speed baseline. If it's that much out of sync before 5 min in, you'd likely need to use a number of sync points just within the first song, shrinking/stretching the cassette source to fit between each. You might check to see if after the first song the drift becomes less egregious and somewhat more manageable. Perhaps reasonable to do if after that point you only need to sync at the start/end of each song. Still a lot of work. How difficult depends on the tool you use, and your patience.
Like many others here I used to do this regularly when I was using two separate digital recorders, but that usually meant sync points at something like 30 min intervals or more. If both are constant speed, just pick one or the other to use as the baseline. I used to select the shorter/faster one as baseline just so that I was shrinking the other to fit instead of stretching it, reasoning that slightly raises the hf limit rather than lowering it if the shrink were
significant, although given the incrementally small speed/time changes involved it probably didn't matter. Same process either way.
Musing..
Always wondered about checking pitch of some held synth or piano note to determine which had a better absolute time basis to use as the baseline file.. assuming the band to be in tune. Never did the math to determine what the time basis of a one cent pitch change equates to time-wise. May not be significant.
The professional way of correcting tape speed variation is the Plangent Process which monitors the bias tone of the tape and adjusts speed on the fly to keep it constant.
https://www.plangentprocesses.com/