Topic: massive storage options

[Simp-Dawg]

you guys just blew my mind 

is this something i should start looking into??  having essentially an HTPC with mucho storage attached hooked directly up to the stereo, instead of getting a network attached player to hook up to my stereo?

the only downside  is eventually i'm going to want access to the storage from multiple systems so a network music player is inevitable.

Hey man, with hardware nowadays I wouldn't worry about disk access speed or any performance metric, they're all more than enough to stream audio/video to multiple destinations.

FWIW, I have a simple system that uses a "silent" case that holds 8 drives, and some 4 port PCI SATA adaptors; Promise Technology, Inc. PDC20718 (SATA 300 TX4)

I happen to be comfortable with linux, but the same can be done with any OS these days.

A cron job mounts the redundant disk, rsync's one disk to another, then unmounts the backup disk in the middle of the night. I generally run this script manually immediately after copying a buncha shows onto the server just for paranoia.

When the pair is full I remove one disk for offline storage and insert another pair. I size the disks according to the best dollar to gig ratio of Seagates at the time.

The box has an audiophile 2496 card for stereo playing (ALSA driver) and uses windows network file sharing (via samba) to stream tunes elsewhere.

The organization part is 100% done in player software (mpd, amarok) via tags on the flac files and there's some minor directory stucture for file level organization.
Tagging flacs/converting shns is done with a modified etree script, flacify.

Its a simple system that's easily recoverable from most failure conditions, with no RAID horror possible.
The evolution of listening habits with instant, searchable, randomizable, categorized, sliced and diced access to all music is fantastic. There is no going back 

wow!  almost a deja vu!

ALSA, check... audiophile 2496, check... "silent case", check... linux, double check 

lirc for handheld remote control... lcdproc for LCD display... moc for remote control from any of the other systems

but back on topic... removable RAID 1 via sata 3ware controller with WD RE2 drives (but i really like your rsync idea!)... sticking with "high end" WD because I've not (yet) had one problem in these many years... and this system runs 24/7 for months while the other boxes are up and down a lot.

hooking this up directly in the stereo chain prevented any (of my) network worries, as the box's only connection is wireless (no keyboard, mouse, or display) and I'd just entered the world of 24/96... well that, and I'm jam'n to a just finished torrent on the good sound system while still uploading

Right on, +T on the *nix chops 

I notice people digging the WD drives on this thread. I had some particularly painful WD's fail on me a few years back, so even though I consider HD failures inevitable I decided to blow them off for life 

I knew Seagate from their highend SCSI drives at the time (cheetah's?) and they were branching into the consumer market with ATA controllers. So even though I consider choosing a HD basically a crapshoot because they all fail, you gotta use some criteria, so that became the next choice. I'm not really involved in that stuff any more but we used to use diverse brands of drives in our arrays so when a batch went bad there were enough other batches still running..

Fortunately (coincidentally) I have not experienced a single failure since.  <knocks on wood.> Whatever works for ya.. if yer backups are solid a drive failure is a time suck, nothing else.   

[bobbygeeWOW]

you guys just blew my mind 

is this something i should start looking into??  having essentially an HTPC with mucho storage attached hooked directly up to the stereo, instead of getting a network attached player to hook up to my stereo?

the only downside  is eventually i'm going to want access to the storage from multiple systems so a network music player is inevitable.

hehe, I used to love blowing people's minds...    well I don't know about "should" look into but I think its worth it. I mean fundamentally its just a good sound card (or reliable spdif out) with winamp and way ya go.. you can use remote desktop to control winamp from somewhere else and windows file sharing to access the tunes from computers in other rooms..

[Colin Liston]

In this months Popular Science they have an article about the new "holodraphic drives" which can store 300 gigs on a single disc, and they expect each disc to eventually hold 1.6 terabytes.

[bobbygeeWOW]

I also run linux on all my systems but really don't have much interest in dealing with remote control integration, etc. I've done it before to run mythtv (in 2002). Too much hassle and time - for me - when I can just run a squeezebox (though the 24/96 support is limited since it is downsampled to 24/48).

I haven't found the need to get a remote control going for my use though I have speculated... Those squeezeboxes are pretty slick alright!

[phanophish]

FWIW I'm running Windows XP Media Center for a playback, storage and torrent PC.  I currently have a 160GB boot drive, 2 500gig SATA internal and about 3 external drives ranging from 160 to 300 Gigs.  I love the macine from a playback standpoint.  it is optically connected to my reciever so 24/96 playback is a breeze.  Everything I have recorded is stored both as a FLAC and in a second directory as a MP3 for synch to my IPod.  I'm also looking at consolidating to some form of RAID sotrage but haven't quite made up my mind.  One of the biggest problems you can have in machines with lots of internal HDs is power.  Drives pull a fair amount so you need a big PS that is rated accurately and reliable.  Finally heat is the enemy of HDs.  Others have mentioned that many of the external housings don't cool drives sufficently and I agree.  I'll probably end up looking at some sort of NAS box that I can house in the basement so the noise is not a bother with honkin big cooling fans to keep it nice and chilly.  Then a "playback" box ideally in a stereo component form factor for playack of media and as a interface to the whole thing.  Once I have the time it will be a project I dig in to so we'll see what happens.

[it-goes-to-eleven]

In this months Popular Science they have an article about the new "holodraphic drives" which can store 300 gigs on a single disc, and they expect each disc to eventually hold 1.6 terabytes.

We've been hearing about holographic breakthrus for so many years it isn't funny... It seems like we haven't had any really big breakthrus, just incremental improvement..

Raw storage in 2002-2003 was over .51/GB for 250GB WD ATA drives. That was the sweet spot.  More recently, it is around .27-.28 with SATA WD 500's at $139.  I think it might get down closer to .20 with sale prices.  I always look at the raw storage cost and then factor in what it takes to use it.

So if I can build a linux tower that can take 10 500 GB drives for $400.. That's  $400+$1390 (10x$139) for about 4850 formatted GB...  Or about 36.9 cents per networked gig = $375 per networked TB. And that would be a pretty powerful machine that could do other stuff.

I've seen a lot of 4 drive enclosures for $600 or so.. Even at 500GB per drive that's 62 cents per gig. So those have never worked for me.  My main music machine is maxed and I'd like to expand the drive space without building *and powering* another whole machine.. Maybe a new PSU in a new case with shielded cables to the orig box.. Biggest obstacle seems to be getting more cheap SATA ports.


Anyone using power mgmt on their drives?  Most of mine sit idle and could be spun down.. Though I wonder what impact that might have on long term reliability?

[pigiron]

you guys just blew my mind 

is this something i should start looking into??  having essentially an HTPC with mucho storage attached hooked directly up to the stereo, instead of getting a network attached player to hook up to my stereo?

the only downside  is eventually i'm going to want access to the storage from multiple systems so a network music player is inevitable.

My advice ain't worth nothing... but when I was in your shoes, I simply sat down with a beer and came up with a list of my criteria... and since it ended up being long and pretty specific, I spent a TON of time (that I didn't have) on the internet researching on how to make it happen... tweaked the criteria when forced to... then a TON more time making it happen... and of course, it came in at about twice the price I originally thought it would 

but in the end I got exactly what I want... so what's not to like 

[tapeworm48]

anyone have any experience with the Cavalry hard drives?  i'm curious about their reliability in comparison to the maxtor, WD, etc.  there is an external 500gb drive for sale through buy.com which is $128 after rebates....

[bobbygeeWOW]

Anyone using power mgmt on their drives?  Most of mine sit idle and could be spun down.. Though I wonder what impact that might have on long term reliability?

I'd spin the drives down, gotta be good. I also have a syslog tailer let me know if smartctl is logging any heating/error conditions.

[Simp-Dawg]

update: so i found a good deal on a WD MyBook Pro Premium II 1TB external drive for $400 at CompUSA and picked it up over the weekend...I figure this is as good a start as any to getting all the neglected, untracked recordings off of my PC and various other drives so I can at least get a good look at what all I have out there!  This also gives me some breathing room on my PC to start defragging and optimizing things so the processing work that I am doing at this time can be done more efficiently.

At a later time I am definitely going to look into more robust solutions as well as redundancy, but for now I am content to have a brand new external drive to dump stuff on to.

[JoeKiller]

I'm sooooo happy I put the effort into migrating all my optical media to HDD for storage (but not backup, of course) and playback.

What are you using for backup Brian?

One downside to using usb or firewire is that the normal SMART based drive diagnostics and monitoring won't work. That's a pretty big disadvantage.  Apparently the industry doesn't have a standard yet for SMART on those interface types.

I've never had SMART actually catch an error before the drive decided it was going to die.


As to hard drive brands.  I think it really goes down for what you personally like.  I have Maxtor, Seagate, and WD all represented in my machines.  None of really more reliable than the others.  All I know is Seagate used to have the best warrenty, 4 years, I dunno if others have caught up.

The sweet spot for drives right now is about 24 cents per gig.  If you go over 28 or 29 you are getting ripped.

[Brian Skalinder]

I'm sooooo happy I put the effort into migrating all my optical media to HDD for storage (but not backup, of course) and playback.

What are you using for backup Brian?

Overall, my storage/backup looks like this:

• all recordings, and masters when appropriate, on redundant HDDs
• all recordings, and all masters, on optical media in-house
• all recordings shared off-site with fellow fans*

It's not perfect, but I feel pretty good about it so far.  I have a backup gap right now with some of my multi-channel masters.  The final mix is backed up off-site with fellow fans, but the masters are strictly in-house.  One of these days, I'll back up all my audio data regularly at my sister's place for safekeeping, but haven't made it happen yet.

* I've shared most of my recordings with no edits, so at the moment I'm comfortable reconstructing the master from the final versions I share with fellow fans.  But I look forward to getting all my recordings and masters off-site at my sister's house one of these days, mainly for peace of mind.

[tapeworm48]

just came across this internal 500GB on sale today at Frys.com for $110:

http://www.frys.com/product/5080555

[cleantone]

just came across this internal 500GB on sale today at Frys.com for $110:

SOLD! +t. Anyone have experience ordering from here? I just ordered this drive and they upgraded me to 2 day for free. The free shipping advertised was 3-5 day. I wonder if I can fit all of my The Slip recordings on here...

[imgoinmad]

just came across this internal 500GB on sale today at Frys.com for $110:

SOLD! +t. Anyone have experience ordering from here?

Fry's is reliable, i've ordered from them a bunch of times and also have a local store I've shopped at too.

Some of you might be interested in my inexpensive storage/archive strategy discussed here:
http://taperssection.com/index.php/topic,81962.0.html