Topic: massive storage options

[Simp-Dawg]

hey all,

so with my nice refund check i'm going to get from uncle sam, i am going to invest in a bunch of storage and start cataloging my collection.

i'm thinking i want between 1.5-2 TB at the minimum, and i want RAID 5 for speedy access and redundancy.

there are several NAS boxes out there that will do this for decent prices (~$1500 give or take a few hundred), but i would also consider building my own server if it was cost effective.

anyone have experience/suggestions to share? 

any good software suggestions for keeping track of all of this stuff?

basically...the server will hold ALL my tracked and compressed recordings (traded or taped myself), all yet-un-processed recordings (and will help me keep track of what i have to get working on), and eventually my ripped music as well.  currently all the stuff i have backed up is on hundreds of random cds and dvds...along with backing them all up to the server, i will organize these into a library of sorts, so i want to use a disc database system to catalog them and label them all in jewel cases on a rack or other physical storage system as well.

thanks in advance for the input, and if this should go into another forum feel free to move it, mods.

[Brian Skalinder]

I think RAID is overkill for home applications when the data is not changing frequently, is not mission-critical, and super-speedy access is not truly required.  You can achieve reasonable redundancy for significantly less $$$ with an external HDD enclosure (or two) and a bunch of HDDs.  Unless you're using high quality RAID controllers (doubtful on most consumer-ish NAS solutions), recovering from RAID failure may prove dicey.  I don't think it's worth the hassle and potential risk.  Granted, you'll need more HDDs in a reasonable redundancy setup to achieve your total storage goals, but IMO reasonable redundancy:

• offers easier set up / configuration
• provides for easier and more reliable recovery
• enables easier storage expansion to address your growing capacity needs

[pfife]

I did what Brian suggests above, and I think its pretty sweet, using 2 internal 320gb drives on a 1ghz machine running XP.

[dnsacks]

office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.

[dcaley]

ive been running 4x200gb sata seagate drives in raid5 for a few years now in my own pc and im pretty much out of space. ive been looking to upgrade to around the same amount as you, 1.5-2tb. but the majority of the new raid cards use the pci-x and pci-e slots which my mainboard doesnt have, so i might be looking at a rather large upgrade. ive been looking at the 320gb seagate sata2 drives as they are ~100$ and an 8 port raid card. not sure which brand, but either promise, highpoint, or 3ware. depends on the cash. im not familiar with nas boxes or how throughput and access times would differ between one and having the hdd's in your pc, if they differ at all.

for 1500$ you could grab 8 320gb sata2 drives, or maybe even 400gb drives, and an 8 port raid card and might have a bit of cash left over to upgrade the ecc memory on the raid card. that'd give you around 2tb or 2.5tb respectively after all is said and done.

[kfrinkle]

office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.

Where did you find this out?  I was planning on buying one of those drives next month, but if they are on sale this month....  I do not see it on their website.

[Simp-Dawg]

I think RAID is overkill for home applications when the data is not changing frequently, is not mission-critical, and super-speedy access is not truly required.  You can achieve reasonable redundancy for significantly less $$$ with an external HDD enclosure (or two) and a bunch of HDDs.  Unless you're using high quality RAID controllers (doubtful on most consumer-ish NAS solutions), recovering from RAID failure may prove dicey.  I don't think it's worth the hassle and potential risk.  Granted, you'll need more HDDs in a reasonable redundancy setup to achieve your total storage goals, but IMO reasonable redundancy:

• offers easier set up / configuration
• provides for easier and more reliable recovery
• enables easier storage expansion to address your growing capacity needs

to expand on my needs...

first of all, initially, the data will be changing frequently...here's my steps to get this set up:
1. transfer all data on backup cds and dvds to the HDD storage system
2. organize tracked/completed recordings in one area, and unfinished recordings in another
3. import all finished and unfinished recordings from various hard drives, organize accordingly
4. begin processing on all uncompleted files, or at least correctly label them and sort accordingly
5. continue adding new recordings to library

after that (as well as cataloging the backup discs in a practical way), i intend to set this server up with a slingbox or something similar so i can access all this content through my stereo.

with all i've read it just seems like RAID 5 is the best solution for speed and redundancy.  plus having a NAS box or building a server of my own keeps all this stuff in one box rather than a number of hd enclosures that may not be able to be accessed wirelessly if my computer is off or whatever.

granted, if 500 gb's is $150, i could get 2TB (4 TB before reasonable redundancy) for $1200, but i have no way other than my windows PC to manage them and make them accessible to the network...unless i set up one of my older machines to do this

[dnsacks]

office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.

Where did you find this out?  I was planning on buying one of those drives next month, but if they are on sale this month....  I do not see it on their website.

It was in their weekly ad (for colorado, at least) -- check out www.salescircular.com for particulars.  I recently picked up the same drive from dell for under $160 shipped --

[silentmark]

i will organize these into a library of sorts, so i want to use a disc database system to catalog them and label them all in jewel cases on a rack or other physical storage system as well.

thanks in advance for the input, and if this should go into another forum feel free to move it, mods.

I am going through this process right now. I finally couldn't take the mess and having no clue of what music I actually have, or don't have.

For disc storage I am using these: http://www.meritline.com/cd-dvd-510-holder-wallet-holders-wallets-ehj510.html , each sleeve is numbered. With that number I setup an excel spreadsheet which relates to what particular disc is in what sleeve. Filtering is simple, by artist, date, venue, source, etc. I also setup a hotlink to each particular show's text file, so with a click click I can call up all the info about the show.

It is abit time consuming but I am seeing the light at the end of the tunnel. After this, next up is the DAT's ...

For HD storage, I thought about a RAID setup, but opted for simple redundant HD's, works well so far ...

[it-goes-to-eleven]

External drives are very useful and definitely have their place..  but I'm not keen on most external enclosures because they don't adequately cool drives.. Especially BIG drives.  There is also the potential for data corruption due to loose external cables, etc..

I've been buying WD 3200JB's and 3200JD's for a few years.. I have around 16-18 of them. They've been Very good to me.  But it appears that it is finally time to move up.  I wish the storage price-performance was moving faster.. The capacity just isn't growing fast enough.

These look good to me in terms of price/performance.  3 year warranty.  This is a 3 platter, so it should run cooler than the previous 4 platter designs.  Review feedback seems good.. $140 with free ship is 28 cents/gig raw.

http://www.westerndigital.com/en/products/products.asp?driveid=214&language=en
http://www.newegg.com/Product/Product.asp?Item=N82E16822136073&name=Item-%23%3A-N82E16822136073

So far, I have been protecting my data with my workflow rather than raid.  Recordings come off the 722 and go to a linux server.. then are copied temporarily to a usb drive and then copied from there to the windows box.. and then backed up to dvd on the linux box.  So the copies exist in multiple places but not in raid.  Given that consumer raid is pretty flakey (compared to real enterprise), I consider this safer.. in most cases it is also far cheaper.  I'm sticking about 10 drives in each PC so the total cost per gig is very low.

I think my next step is to stuff a bunch of drives in a case and then run the sata cables back to the main pc.  Not quite sure about shielding the sata cables yet or what case I'm going to use.

I've been using these 4 port SATA controllers.. $25... Can't beat the per-port cost. I have used two per system but don't know if you can use more than that.

http://www.newegg.com/Product/Product.asp?Item=N82E16815124020

[Brian Skalinder]

first of all, initially, the data will be changing frequently...

I don't think any of your goals / requirements are inconsistent with a reasonably redundant setup.  By constantly changing data, I mean real-time transactional data.  Not an hour or two's (or even a bunch of hours) worth of copying data from CD/DVD to HDD every evening.

with all i've read it just seems like RAID 5 is the best solution for speed and redundancy.  plus having a NAS box or building a server of my own keeps all this stuff in one box rather than a number of hd enclosures that may not be able to be accessed wirelessly if my computer is off or whatever.

RAID -can- provide good redundancy.  But IME only if you get a quality (read $$$) controller.  I've had a couple different arrays fail using consumer-ish controllers that rendered the array unrecoverable.  Not fun.

As for enclosures, you can buy multi-slot enclosures, so you don't need to have a slew of them cluttering up space.  As FL mentions, just make sure it has adequate ventilation (just as you'd need on your NAS box).

The only real issue I see with what you've mentioned so far is the need to have a computer on in order to access the HDDs.  My PC's on all the time anyway, so I it's not a problem for me to simply tack the external enclosure onto my primary desktop.

At any rate, whatever you decide - have fun!  I'm sooooo happy I put the effort into migrating all my optical media to HDD for storage (but not backup, of course) and playback.  It'll take some time, but I suspect you'll really enjoy the results.

[Wiggle]

office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.

Where did you find this out?  I was planning on buying one of those drives next month, but if they are on sale this month....  I do not see it on their website.

FWIW, I just picked up a WD My Book 250gig drive for $90 off of eBay this morning. There appears to be a glut of them on there and if you pick the right auction they seem to be going pretty cheap. Of course, it cost me an additional $30 to ship to Canada - YMMV.

[cleantone]

Office Depot's WD products right now.

I wish that box was Firewire. I don't have USB2 on my older TiBook. That plus the fact that the closest store is 25 miles away I might not jump on that as I thought I would.

The 500GB USB WD for $150

I just found 200GB Maxtor internal for $40 at Staples too! Can't turn that down right now. Maybe two even!

[heikki]

For disc storage I am using these: http://www.meritline.com/cd-dvd-510-holder-wallet-holders-wallets-ehj510.html

ooooohhhhhh, i REALLY like that. 
While I do store all my new masters and downloads on HDDs I still back up all my new masters on CD/DVD and have a lot of old DAT > SHN/FLAC conversions that have not yet been copied to HDD.

Secure HDD back-up is so the way to go.  All of your music is organized, redundantly stored, and most importantly available for listening!

RAID5 is great efficiency, but as others mentioned if you don't have a good card it can be slow and unreliable.  I use the RAID5 NAS strictly for long-term storage and as a kind of "jukebox".  I use the RAID10 as my normal HDD.  I find it a bit quicker than RAID5 and still very reliable, though not as efficient.  This is where my OS, apps, docs, and in-process music resides.

-Heikki
4x500GB RAID5  NAS Box Archive/Jukebox
4x250GB RAID10 PC Working Storage

[kfrinkle]

Office Depot's WD products right now.

I wish that box was Firewire. I don't have USB2 on my older TiBook. That plus the fact that the closest store is 25 miles away I might not jump on that as I thought I would.

The 500GB USB WD for $150

I just found 200GB Maxtor internal for $40 at Staples too! Can't turn that down right now. Maybe two even!

Awesome, thanks!

[silentmark]

For disc storage I am using these: http://www.meritline.com/cd-dvd-510-holder-wallet-holders-wallets-ehj510.html

ooooohhhhhh, i REALLY like that. 
While I do store all my new masters and downloads on HDDs I still back up all my new masters on CD/DVD and have a lot of old DAT > SHN/FLAC conversions that have not yet been copied to HDD.

Yeah I find they work real well for my archive discs. With the numbered sleeves you don't have to worry about what order things are in (band name, dates etc ...), just as long as your reference file is documented accordingly. Nice solid construction, although I have broken a few of the plastic little hangers that the sleeves come with, no biggie, easily replaced or fixed. They make even larger ones then the 500 size ...

[imgoinmad]

Fry's has a 500gb SATA drive for $139 right now with free shipping (ends today)
http://tinyurl.com/2adbgk

I'm not to into the raid setup for all my music, although if you're going to keep them on all the time you'll need the redundancy. I've currently got about 47 hard drives...what I do is attach and external drive until it's full and then document the contents, unplug it an put it on the shelf. If I need to retreive a show, I look it up in the index to see which drive it's on and pull it off the shelf. I buy the same enclosures so I can use the same power supply for each. If the drive is on the shelf the chance of failure is relatively small, since the drive doesn't ever got hot enough to cause damage to the bearings...makes for a great archival system. Of course not everything is online all at once, but I'd rather do it this way than risk drive failure and data loss...not to mention it saves power and reduces heat in the room.

[cleantone]

Touvaly is a great FREE Mac software that makes a pseudo copy of an drive and it's contents. Basically you can do this with your external drive and have faux copy that shows ALL of the contents but doesn't actually have the contents to the file/image is really small, maybe a MB or less I forget. Anyway it is pretty handy for people on a Mac with multiple external drives and such.

EDIT:

looks like it is not free anymore. probably not worth $25.

[Simp-Dawg]

bump...

ok so i decided to follow skalinder's advice and go for reasonable redundancy.

my question now is, what's the optimal storage configuration for high-speed transfer, network accessibility, and media streaming?

i'm looking at a couple of these right now, but i'm not sure if i want them attached to my main pc, or to a nas device, or to an older, dedicated pc that i can set up only to be a file server essentially.  if i went the latter route, what's the best way to set that up?

i've got a 550mhz amd machine sitting around, and a 333mhz amd aptiva too.  they're both in working condition, but will they be fast enough to serve up data quickly? 

[pigiron]

bump...
[snip]
my question now is, what's the optimal storage configuration for high-speed transfer, network accessibility, and media streaming?
[snip]

I can't answer the question, but you previously mentioned wireless and I've found that 24/96 saturates my encrypted 802.11g causing dropouts while playing... (but that was wireless box to wireless box)... but perhaps you have different criteria in the streaming department???

[Simp-Dawg]

bump...
[snip]
my question now is, what's the optimal storage configuration for high-speed transfer, network accessibility, and media streaming?
[snip]

I can't answer the question, but you previously mentioned wireless and I've found that 24/96 saturates my encrypted 802.11g causing dropouts while playing... (but that was wireless box to wireless box)... but perhaps you have different criteria in the streaming department???

i do want to stream up to 24/96 actually

[bobbygeeWOW]

Hey man, with hardware nowadays I wouldn't worry about disk access speed or any performance metric, they're all more than enough to stream audio/video to multiple destinations.

FWIW, I have a simple system that uses a "silent" case that holds 8 drives, and some 4 port PCI SATA adaptors; Promise Technology, Inc. PDC20718 (SATA 300 TX4)

I happen to be comfortable with linux, but the same can be done with any OS these days.

A cron job mounts the redundant disk, rsync's one disk to another, then unmounts the backup disk in the middle of the night. I generally run this script manually immediately after copying a buncha shows onto the server just for paranoia.

When the pair is full I remove one disk for offline storage and insert another pair. I size the disks according to the best dollar to gig ratio of Seagates at the time.

The box has an audiophile 2496 card for stereo playing (ALSA driver) and uses windows network file sharing (via samba) to stream tunes elsewhere.

The organization part is 100% done in player software (mpd, amarok) via tags on the flac files and there's some minor directory stucture for file level organization.
Tagging flacs/converting shns is done with a modified etree script, flacify.

Its a simple system that's easily recoverable from most failure conditions, with no RAID horror possible.
The evolution of listening habits with instant, searchable, randomizable, categorized, sliced and diced access to all music is fantastic. There is no going back 

[bobbygeeWOW]

Oh, and adding replay-gain to the flacs is key for random playback 

[thegreatgumbino]

Interesting discussions.  I'm jumping in just to track the thread.

[Simp-Dawg]

ok...what about HDD brands?  i've heard mixed reviews on the WD MyBook's, but there are sme good deals on them.  are they a safe bet?  if not, what do you recommend?

[Brian Skalinder]

I've been happy with WDs, too.  I don't buy bottom-of-the-barrel cheapest drives from any brand, though - IME they typically tank fairly quickly.  I typically look for drives targeted more towards a business environment.  A rough way to identify these HDs is that they offer a 5yr warranty instead of 3yr or 1yr.

[tfs8271]

Just went 24/96 with my newly purchased slightly used R4. Four channels running sure takes a lot of memory.

Just joining in to see what others are considering. I still have about 300GB available but it won't take long to fill it.

[OFOTD]

I wanted to share a little of what I am running right now.

Currently I have a dedicated home server set up.  I download alot of classic tv shows and video.  My music collection is greater than I could ever keep up with as well.  What I have done is built a server that is just stuffed with hard drives (11 x 500GB internal) and (8 x 300GB external)  I keep all of my videos, mp3's, photos, important documents as well as as much music as I can fit. 

For an OS I am running a beta of Windows Home Server.  Its a new product MS is developing for just this type of application.  It allows me to add or remove drives as I see fit and also allows me to choose the type of redundancy that I want on a file by file or folder by folder basis.  For example I have all my photos and documents backed up.

If you have a little bit of time look into WHS.     http://blogs.technet.com/homeserver/default.aspx

As for hard drives I stick to only Maxtor and WD nowadays.  Never go below their mid-level line of drives. 

24/96 won't work wirelessly.  To much info and you get lots of dropped data.  802.11 N  maybe but that hear is still a little way off for affordable public consumption.

[pigiron]

Hey man, with hardware nowadays I wouldn't worry about disk access speed or any performance metric, they're all more than enough to stream audio/video to multiple destinations.

FWIW, I have a simple system that uses a "silent" case that holds 8 drives, and some 4 port PCI SATA adaptors; Promise Technology, Inc. PDC20718 (SATA 300 TX4)

I happen to be comfortable with linux, but the same can be done with any OS these days.

A cron job mounts the redundant disk, rsync's one disk to another, then unmounts the backup disk in the middle of the night. I generally run this script manually immediately after copying a buncha shows onto the server just for paranoia.

When the pair is full I remove one disk for offline storage and insert another pair. I size the disks according to the best dollar to gig ratio of Seagates at the time.

The box has an audiophile 2496 card for stereo playing (ALSA driver) and uses windows network file sharing (via samba) to stream tunes elsewhere.

The organization part is 100% done in player software (mpd, amarok) via tags on the flac files and there's some minor directory stucture for file level organization.
Tagging flacs/converting shns is done with a modified etree script, flacify.

Its a simple system that's easily recoverable from most failure conditions, with no RAID horror possible.
The evolution of listening habits with instant, searchable, randomizable, categorized, sliced and diced access to all music is fantastic. There is no going back 

wow!  almost a deja vu!

ALSA, check... audiophile 2496, check... "silent case", check... linux, double check 

lirc for handheld remote control... lcdproc for LCD display... moc for remote control from any of the other systems

but back on topic... removable RAID 1 via sata 3ware controller with WD RE2 drives (but i really like your rsync idea!)... sticking with "high end" WD because I've not (yet) had one problem in these many years... and this system runs 24/7 for months while the other boxes are up and down a lot.

hooking this up directly in the stereo chain prevented any (of my) network worries, as the box's only connection is wireless (no keyboard, mouse, or display) and I'd just entered the world of 24/96... well that, and I'm jam'n to a just finished torrent on the good sound system while still uploading

[bobbygeeWOW]

Right on, +T on the *nix chops 

I notice people digging the WD drives on this thread. I had some particularly painful WD's fail on me a few years back, so even though I consider HD failures inevitable I decided to blow them off for life 

I knew Seagate from their highend SCSI drives at the time (cheetah's?) and they were branching into the consumer market with ATA controllers. So even though I consider choosing a HD basically a crapshoot because they all fail, you gotta use some criteria, so that became the next choice. I'm not really involved in that stuff any more but we used to use diverse brands of drives in our arrays so when a batch went bad there were enough other batches still running..

Fortunately (coincidentally) I have not experienced a single failure since.  <knocks on wood.> Whatever works for ya.. if yer backups are solid a drive failure is a time suck, nothing else.   

[Simp-Dawg]

you guys just blew my mind 

is this something i should start looking into??  having essentially an HTPC with mucho storage attached hooked directly up to the stereo, instead of getting a network attached player to hook up to my stereo?

the only downside  is eventually i'm going to want access to the storage from multiple systems so a network music player is inevitable.

Hey man, with hardware nowadays I wouldn't worry about disk access speed or any performance metric, they're all more than enough to stream audio/video to multiple destinations.

FWIW, I have a simple system that uses a "silent" case that holds 8 drives, and some 4 port PCI SATA adaptors; Promise Technology, Inc. PDC20718 (SATA 300 TX4)

I happen to be comfortable with linux, but the same can be done with any OS these days.

A cron job mounts the redundant disk, rsync's one disk to another, then unmounts the backup disk in the middle of the night. I generally run this script manually immediately after copying a buncha shows onto the server just for paranoia.

When the pair is full I remove one disk for offline storage and insert another pair. I size the disks according to the best dollar to gig ratio of Seagates at the time.

The box has an audiophile 2496 card for stereo playing (ALSA driver) and uses windows network file sharing (via samba) to stream tunes elsewhere.

The organization part is 100% done in player software (mpd, amarok) via tags on the flac files and there's some minor directory stucture for file level organization.
Tagging flacs/converting shns is done with a modified etree script, flacify.

Its a simple system that's easily recoverable from most failure conditions, with no RAID horror possible.
The evolution of listening habits with instant, searchable, randomizable, categorized, sliced and diced access to all music is fantastic. There is no going back 

wow!  almost a deja vu!

ALSA, check... audiophile 2496, check... "silent case", check... linux, double check 

lirc for handheld remote control... lcdproc for LCD display... moc for remote control from any of the other systems

but back on topic... removable RAID 1 via sata 3ware controller with WD RE2 drives (but i really like your rsync idea!)... sticking with "high end" WD because I've not (yet) had one problem in these many years... and this system runs 24/7 for months while the other boxes are up and down a lot.

hooking this up directly in the stereo chain prevented any (of my) network worries, as the box's only connection is wireless (no keyboard, mouse, or display) and I'd just entered the world of 24/96... well that, and I'm jam'n to a just finished torrent on the good sound system while still uploading

Right on, +T on the *nix chops 

I notice people digging the WD drives on this thread. I had some particularly painful WD's fail on me a few years back, so even though I consider HD failures inevitable I decided to blow them off for life 

I knew Seagate from their highend SCSI drives at the time (cheetah's?) and they were branching into the consumer market with ATA controllers. So even though I consider choosing a HD basically a crapshoot because they all fail, you gotta use some criteria, so that became the next choice. I'm not really involved in that stuff any more but we used to use diverse brands of drives in our arrays so when a batch went bad there were enough other batches still running..

Fortunately (coincidentally) I have not experienced a single failure since.  <knocks on wood.> Whatever works for ya.. if yer backups are solid a drive failure is a time suck, nothing else.   

[bobbygeeWOW]

you guys just blew my mind 

is this something i should start looking into??  having essentially an HTPC with mucho storage attached hooked directly up to the stereo, instead of getting a network attached player to hook up to my stereo?

the only downside  is eventually i'm going to want access to the storage from multiple systems so a network music player is inevitable.

hehe, I used to love blowing people's minds...    well I don't know about "should" look into but I think its worth it. I mean fundamentally its just a good sound card (or reliable spdif out) with winamp and way ya go.. you can use remote desktop to control winamp from somewhere else and windows file sharing to access the tunes from computers in other rooms..

[Colin Liston]

In this months Popular Science they have an article about the new "holodraphic drives" which can store 300 gigs on a single disc, and they expect each disc to eventually hold 1.6 terabytes.

[bobbygeeWOW]

I also run linux on all my systems but really don't have much interest in dealing with remote control integration, etc. I've done it before to run mythtv (in 2002). Too much hassle and time - for me - when I can just run a squeezebox (though the 24/96 support is limited since it is downsampled to 24/48).

I haven't found the need to get a remote control going for my use though I have speculated... Those squeezeboxes are pretty slick alright!

[phanophish]

FWIW I'm running Windows XP Media Center for a playback, storage and torrent PC.  I currently have a 160GB boot drive, 2 500gig SATA internal and about 3 external drives ranging from 160 to 300 Gigs.  I love the macine from a playback standpoint.  it is optically connected to my reciever so 24/96 playback is a breeze.  Everything I have recorded is stored both as a FLAC and in a second directory as a MP3 for synch to my IPod.  I'm also looking at consolidating to some form of RAID sotrage but haven't quite made up my mind.  One of the biggest problems you can have in machines with lots of internal HDs is power.  Drives pull a fair amount so you need a big PS that is rated accurately and reliable.  Finally heat is the enemy of HDs.  Others have mentioned that many of the external housings don't cool drives sufficently and I agree.  I'll probably end up looking at some sort of NAS box that I can house in the basement so the noise is not a bother with honkin big cooling fans to keep it nice and chilly.  Then a "playback" box ideally in a stereo component form factor for playack of media and as a interface to the whole thing.  Once I have the time it will be a project I dig in to so we'll see what happens.

[it-goes-to-eleven]

In this months Popular Science they have an article about the new "holodraphic drives" which can store 300 gigs on a single disc, and they expect each disc to eventually hold 1.6 terabytes.

We've been hearing about holographic breakthrus for so many years it isn't funny... It seems like we haven't had any really big breakthrus, just incremental improvement..

Raw storage in 2002-2003 was over .51/GB for 250GB WD ATA drives. That was the sweet spot.  More recently, it is around .27-.28 with SATA WD 500's at $139.  I think it might get down closer to .20 with sale prices.  I always look at the raw storage cost and then factor in what it takes to use it.

So if I can build a linux tower that can take 10 500 GB drives for $400.. That's  $400+$1390 (10x$139) for about 4850 formatted GB...  Or about 36.9 cents per networked gig = $375 per networked TB. And that would be a pretty powerful machine that could do other stuff.

I've seen a lot of 4 drive enclosures for $600 or so.. Even at 500GB per drive that's 62 cents per gig. So those have never worked for me.  My main music machine is maxed and I'd like to expand the drive space without building *and powering* another whole machine.. Maybe a new PSU in a new case with shielded cables to the orig box.. Biggest obstacle seems to be getting more cheap SATA ports.


Anyone using power mgmt on their drives?  Most of mine sit idle and could be spun down.. Though I wonder what impact that might have on long term reliability?

[pigiron]

you guys just blew my mind 

is this something i should start looking into??  having essentially an HTPC with mucho storage attached hooked directly up to the stereo, instead of getting a network attached player to hook up to my stereo?

the only downside  is eventually i'm going to want access to the storage from multiple systems so a network music player is inevitable.

My advice ain't worth nothing... but when I was in your shoes, I simply sat down with a beer and came up with a list of my criteria... and since it ended up being long and pretty specific, I spent a TON of time (that I didn't have) on the internet researching on how to make it happen... tweaked the criteria when forced to... then a TON more time making it happen... and of course, it came in at about twice the price I originally thought it would 

but in the end I got exactly what I want... so what's not to like 

[tapeworm48]

anyone have any experience with the Cavalry hard drives?  i'm curious about their reliability in comparison to the maxtor, WD, etc.  there is an external 500gb drive for sale through buy.com which is $128 after rebates....

[bobbygeeWOW]

Anyone using power mgmt on their drives?  Most of mine sit idle and could be spun down.. Though I wonder what impact that might have on long term reliability?

I'd spin the drives down, gotta be good. I also have a syslog tailer let me know if smartctl is logging any heating/error conditions.

[Simp-Dawg]

update: so i found a good deal on a WD MyBook Pro Premium II 1TB external drive for $400 at CompUSA and picked it up over the weekend...I figure this is as good a start as any to getting all the neglected, untracked recordings off of my PC and various other drives so I can at least get a good look at what all I have out there!  This also gives me some breathing room on my PC to start defragging and optimizing things so the processing work that I am doing at this time can be done more efficiently.

At a later time I am definitely going to look into more robust solutions as well as redundancy, but for now I am content to have a brand new external drive to dump stuff on to.

[JoeKiller]

I'm sooooo happy I put the effort into migrating all my optical media to HDD for storage (but not backup, of course) and playback.

What are you using for backup Brian?

One downside to using usb or firewire is that the normal SMART based drive diagnostics and monitoring won't work. That's a pretty big disadvantage.  Apparently the industry doesn't have a standard yet for SMART on those interface types.

I've never had SMART actually catch an error before the drive decided it was going to die.


As to hard drive brands.  I think it really goes down for what you personally like.  I have Maxtor, Seagate, and WD all represented in my machines.  None of really more reliable than the others.  All I know is Seagate used to have the best warrenty, 4 years, I dunno if others have caught up.

The sweet spot for drives right now is about 24 cents per gig.  If you go over 28 or 29 you are getting ripped.

[Brian Skalinder]

I'm sooooo happy I put the effort into migrating all my optical media to HDD for storage (but not backup, of course) and playback.

What are you using for backup Brian?

Overall, my storage/backup looks like this:

• all recordings, and masters when appropriate, on redundant HDDs
• all recordings, and all masters, on optical media in-house
• all recordings shared off-site with fellow fans*

It's not perfect, but I feel pretty good about it so far.  I have a backup gap right now with some of my multi-channel masters.  The final mix is backed up off-site with fellow fans, but the masters are strictly in-house.  One of these days, I'll back up all my audio data regularly at my sister's place for safekeeping, but haven't made it happen yet.

* I've shared most of my recordings with no edits, so at the moment I'm comfortable reconstructing the master from the final versions I share with fellow fans.  But I look forward to getting all my recordings and masters off-site at my sister's house one of these days, mainly for peace of mind.

[tapeworm48]

just came across this internal 500GB on sale today at Frys.com for $110:

http://www.frys.com/product/5080555

[cleantone]

just came across this internal 500GB on sale today at Frys.com for $110:

SOLD! +t. Anyone have experience ordering from here? I just ordered this drive and they upgraded me to 2 day for free. The free shipping advertised was 3-5 day. I wonder if I can fit all of my The Slip recordings on here...

[imgoinmad]

just came across this internal 500GB on sale today at Frys.com for $110:

SOLD! +t. Anyone have experience ordering from here?

Fry's is reliable, i've ordered from them a bunch of times and also have a local store I've shopped at too.

Some of you might be interested in my inexpensive storage/archive strategy discussed here:
http://taperssection.com/index.php/topic,81962.0.html

[tfs8271]

just came across this internal 500GB on sale today at Frys.com for $110:

SOLD! +t. Anyone have experience ordering from here?

Fry's is reliable, i've ordered from them a bunch of times and also have a local store I've shopped at too.

Some of you might be interested in my inexpensive storage/archive strategy discussed here:
http://taperssection.com/index.php/topic,81962.0.html

Ditto but you have it by now.

[OFOTD]

I bought a 500GB external USB Western Digital drive a while back...  It was $150 on sale.

It has been working well but has one major PIA issue.  It automatically goes to sleep after a couple minutes of idle time. It looks like that can't be disabled..  I think this feature is what keeps the drive from overheating. The enclosure has lots of vents but is plastic and has no fan.

I mostly use the drive for shuffling data but it is a real pain if you're doing work on it.

Have you checked your computers power settings?

[kskreider]

A buddy of mine said he wanted a 1TB NAS.  I shopped around and found this:

Infrant ReadyNAS ($615 at Newegg)
4 x 750GB SATAII (4 x $199 at Fry's)
= ~$1500 for 3TB
add the white, on sale, Squeezebox for $249 (less $20 code) and you have a new playback server.

Apparently the Infrant ReadyNAS will run the Slim Server for the Squeezebox but only runs at SATAI speeds and not SATAII.  Just and FYI for folks looking for the MASSIVE option.

[Nick's Picks]

I have a netgear NAS box that I bought for $50 (no drives)
it takes two IDEs, of any size.  Sets up some basic RAID/MIRROR action...if  you want.
add two 500gb drives, and its a wonderful and cheap storage device.  Add Squeezebox .....and you're in heaven.

[tfs8271]

So I have my Sony Vaio connected via opti cable to my receiver. Therefore, I wouldn't need a squeezebox. Correct?

It looks like I could use the NAS/4-750GB combo.

[Carrera2]

I've been using this one with some inexpensive 400 GB drives and been happy with it:

http://snipurl.com/1mcei

[ford prefect]

Anyone using power mgmt on their drives?  Most of mine sit idle and could be spun down.. Though I wonder what impact that might have on long term reliability?

I'd spin the drives down, gotta be good. I also have a syslog tailer let me know if smartctl is logging any heating/error conditions.

My understanding (and limited experience) is that if the computer is on 24/7 it's best to have them spin all the time.  The Spin Up/Spin Down process is much more wear and tear on the drives than maintaining a constant spin and will eventually lead to an earlier failure.  IME - I have 4x 120GB drives that have been on as long as the machine has power for the past ~5 years.  (Knock on wood), no failures yet...

[fischer]

I have been using and deploying un-RAID servers.  www.lime-technology.com  There is nothing else like it on the market... where you can use any size HD's, parity just has to be the largest disk. I have had great success.

Few drawbacks,
1. Security... there really is none, but in the works
2. Slimserver.. again this is in the works.

[bobbygeeWOW]

Anyone using power mgmt on their drives?  Most of mine sit idle and could be spun down.. Though I wonder what impact that might have on long term reliability?

I'd spin the drives down, gotta be good. I also have a syslog tailer let me know if smartctl is logging any heating/error conditions.

My understanding (and limited experience) is that if the computer is on 24/7 it's best to have them spin all the time.  The Spin Up/Spin Down process is much more wear and tear on the drives than maintaining a constant spin and will eventually lead to an earlier failure.  IME - I have 4x 120GB drives that have been on as long as the machine has power for the past ~5 years.  (Knock on wood), no failures yet...

<Not that I know anything>
Well what hurt drives is hot/cold, start/stop. The theory is that for servers being used 24x7, you keep the drives spinning, much like driving on the highway, to avoid the stop/start wear and tear of city driving.

My general usage pattern of the music server is about a couple hours a day, say when I'm not sleeping or out of the house. With that pattern I decided that spinning down would be the way to go - since once I start listening, the drives will remain spinning the entire time as they stream flacs off the drives. Mind you I don't think that's any sort of absolute - you'd have to decide what makes sense for you.   

[OFOTD]

For me it all comes down to buying a quality drive from a quality manufacturer.  Sure there are bad drives in every batch but for the most part it is very much worth buying a good quality drive.  I can't tell you how many people I know who bought the $75 on sale drive from <insert store here> and had them fail.  They call and ask me how to fix it and I always go back to you get what you pay for.  Now mind you that the drives are usually Hitachi's or some other crap ass brand but you get my point.  Do a little research, look at the different product lines from a given manufacturer.  Spend a few extra bucks get the better drive.

[ford prefect]

Now mind you that the drives are usually Hitachi's or some other crap ass brand but you get my point. 

Ha, Hitachi is the brand that archive.org uses for its Petaboxes.   They find them to be low power, low cost, high reliability. 

[OFOTD]

Now mind you that the drives are usually Hitachi's or some other crap ass brand but you get my point. 

Ha, Hitachi is the brand that archive.org uses for its Petaboxes.   They find them to be low power, low cost, high reliability. 

Exactly the complete opposite of my experiences with many of the Hitachi drives.  Possibly they have some Europe-only reliable drives in that rack because they aren't reliable here in the States.

Cool webpage though for the server rack.

[ford prefect]

Exactly the complete opposite of my experiences with many of the Hitachi drives.  Possibly they have some Europe-only reliable drives in that rack because they aren't reliable here in the States.

Nah, their datacenter is in SF now.  But I'm pretty sure they work directly with Hitachi due to the volume of drives they use, so you're right that it's probably not consumer grade stuff.