Topic: How to archive legacy MD's losslessly on OS X in 20 (not so) easy steps

[taper420]

As of version 0.9.13-2-g768bf75 (build date 06/01/2014), the python scripts below have been converted to C and integrated into QHIMDTransfer, so most if not all of the steps listed below are now obsolete, however it may still be necessary to install libusb (step 9, but omit git py27-crypto)


First off… this is not easy…but if you're reading this, chances are you aren't quite happy with the easy methods out there, so here's the hard way. It's hard, but the payoff is that this is, from what I can gather, the ONLY way you will get the original untouched digital data that you recorded so long ago. There are many steps, and command lines, and hopefully in the not too distant future there will be a GUI for this (maybe we could even get Scott to incorporate it into xACT). Now I'm not sure about the pre-reqs here…. bottom line is you need to have an RH1, which is the last Hi-MD unit ever made, and the only unit that ever allowed faster than real time uploads of legacy md's (only on PC's, in conjunction with SonicStage, and converted to 16 bit fixed int wav). I did this on a mac (running Snow Leopard)…. I think it may be possible on a PC, and the tools are linux so that should work.. but I'm only positive about how to get it to work on a mac. Though I will include links so you can do your own research.

I've seen a couple threads hear discussing this. It's not new news. But no one really explains it. The netmd python page that everyone links to is what got me through it, but I could have saved a lot of time with a more thorough (noob) walk-through. So while it's fresh in my head, I'm going to put this all out there right now to help some folks searching in the future.

I'm not gonna get into a debate as far as the ATRAC1 codecs audio fidelity…. the point is when we recorded an analog signal into our old MD decks, it was converted to a digital signal (using a pretty dam good ADC) and stored in a file format that has been notoriously difficult to actually get our "hands" on. I'm discussing how to get that original file without conversions that will introduce any lossyness from that original file that was stored at recording, and of which there actually is no higher definition data that was ever stored… it was encoded directly from the recorders ADC. So technically if we can get that original ATRAC file, nothing has been lost, other than the opportunity to have recorded it with better equipment at the time it was made.

I spent all night last night figuring this out, and I'm finally staring at ATRAC files that are archived on my computer, and I've been able to play them as well (with what I would deem a lossless conversion). I've found the best secondary storage method for the smallest file and playability (other than somehow playing the ATRAC file directly) is wavepack, as it allows a compression ratio similar to FLAC, but also allows the ATRAC file to retain its native floating point format… which can not be said about any other transfer method… be it a real time digital, even with a 24 bit output… or using the RH1 over USB > sonicstage > conversion to 16 bit fixed integer wave.

So this link pretty much explains it all:
https://wiki.physik.fu-berlin.de/linux-minidisc/doku.php?id=netmdpython#macos
but I'll try to walk you through it…. there's some hiccups I encountered

1. Get your Mac
2. Get your RH1
3. Get your Mac install discs > install xcode (additional installs, or something like that)
4. Turn off peer guardian or any other firewall that may cause trouble (IMPORTANT)
5. Install macports, http://www.macports.org
6. Open terminal (applications>utilities)
7. Type echo "PATH=$PATH:/opt/local/bin" >> ~/.bashrc
8. Type sudo port -v selfupdate

If you get an error about index(es) missing, I found the only fix was to install them manually. If you get no error, goto step 9.

8a. Goto http://www.macports.org/files/ports.tar.gz and unzip it to /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/

9. Type sudo port install libusb git py27-crypto

wait for it to download and install a bunch of stuff. It may error out… it happened to me a few times due to connection issues. If it does, type the same thing again (press the up key)… it will check what's been done, and pick up where it left off. Keep at it until you get no errors.

10. Type git clone git://minidisc.dyndns.org/linux-minidisc
11. Type "cd .." a few times. That's "cd .." (without the quotes). Then type ls and see if you see a folder that says opt. If not do cd .. until you do. When you do see opt, type this: cd opt/local/etc/macports/linux-minidisc/netmd
12. Ok hook your Rh1 up to your mac with a usb and put a legacy md in there.
13. Type sudo ./lsmd.py
This is going to confirm you're on the right track, the computer can see the md deck, and it will show you a listing of the tracks on the disc along with their length.

14. Are you ready? OKay… here we go…
type: sudo ./upload.py

Now wait and watch as all your amazing, old, ancient ATRAC1 recordings are copied to your computer in their pristinely untouched original lossy glory. But no more lossy than they've ever been!
15. Type ls and check them out… they have an .aea extension, and they're probably labeled: 01 - .aea, 02 - .aea, etc. Note the the names for the next step (or copy and paste).

16. Okay now lets get them converted to a wavepack, and maintain their original floating point format… we'll use 32 bits… even though the highest ADC was 24 bits (and most we're 20), wavepack only lets us store 32bit floats. Also, due to the encoding method, the ADC used isn't going to quite correlate to the bit depth you'll get out of the files… but much like with mp3's, if you encoded from a 24 bit file, it's going to use the extra resolution. It follows that it will use the extra resolution in reverse as well, hence the 20 and 24 bit PCM spidf outputs on many home decks (but they're not floating point).

The only way to decode the ATRAC files is with ffmpeg (I tried a GUI, it didn't work), so we'll go command line. Unfortunately, we can't encode directly to wavpack with ffmpeg, and I tried doing a wav container with a 32bit float and it didn't like it, so I used a Core Audio Container… type:

sudo ffmpeg -i "01 - .aea" -f caf -acodec pcm_f32le ~/Music/Track1-32FL.caf
ffmpeg -i "01 - .aea" -f wav -acodec pcm_f32le ~/Music/Track1-32FL.wav

this assumes the filename is 01 - .aea, and it puts the file in your music folder as Track1-32FL.wav
you'll have to change the file name for each track… this is the template:

sudo ffmpeg -i "FILENAME.aea" -f caf -acodec pcm_f32le ~/LOCATION_IN_HOME_FOLDER/FILENAME.caf
sudo ffmpeg -i "FILENAME.aea" -f wav -acodec pcm_f32le ~/LOCATION_IN_HOME_FOLDER/FILENAME.wav

Bam!
17. What I did from there is convert the .caf to a .wav using Audition (quicktime doesn't like converting to floating point waves, and converting to wavpack from aiff somehow converts it to fixed int)
18. Encode to wavpack using xACT
19. Decode to wav just to check and make sure you're still floating point… if so congrats… you've got some work to do!
20. Archive the ATRAC files (.aea)… copy them from opt/local/etc/macports/linux-minidisc/netmd to your choice of archival method. Then delete them from there and make way for the next batch. (if someone can figure out how to specify an alternate output location, please let us know)

There you go… 20 steps and you're home free. Now get to some lossless archiving of lossy-ONLY-due-to-lost-opportunity material.

[beatkilla]

Nice.But i thought that was what sonic stage was for? You could import the atrac only or import to library and convert to wav.

[taper420]

I don't believe you can gain access to the original ATRAC file with SonicStage... I believe you must convert it to a wav. It may be possible to play the atrac from disc in sonicstage... it may even be able to import it into sonicstage (actually i don't think this is possible without conversion), but the only way to get it out of sonicstage (without a secondary lossy conversion) is to export to a fixed int 16 bit PCM wave. I believe that encoding directly from atrac to a 32 bit float point lossless format would be a better conversion option, and more natural for a floating point codec such as atrac.

Also, I don't have ready access to a PC, and I hate sonicstage and installing crappy software on computers that aren't mine. I honestly never thought we'd have access to the legacy MD's on a mac, so for one this is great for that alone, but also because I believe it is the only (known?) method for extracting the raw, untouched ATRAC data from the disc, without even converting it to (what is thought as lossless) 16 bit fixed int waves.

You can also play the raw atrac files in VLC without any conversion whatsover.