Become a Site Supporter and Never see Ads again!

Author Topic: massive storage options  (Read 14100 times)

0 Members and 1 Guest are viewing this topic.

Offline Simp-Dawg

  • Bad Little Dawggie
  • (2)
  • Needs to get out more...
  • *****
  • Posts: 15077
  • Gender: Male
  • Daddy needs a drink!
    • Colorado Tapers
massive storage options
« on: February 20, 2007, 11:43:09 AM »
hey all,

so with my nice refund check i'm going to get from uncle sam, i am going to invest in a bunch of storage and start cataloging my collection.

i'm thinking i want between 1.5-2 TB at the minimum, and i want RAID 5 for speedy access and redundancy.

there are several NAS boxes out there that will do this for decent prices (~$1500 give or take a few hundred), but i would also consider building my own server if it was cost effective.

anyone have experience/suggestions to share? 

any good software suggestions for keeping track of all of this stuff?

basically...the server will hold ALL my tracked and compressed recordings (traded or taped myself), all yet-un-processed recordings (and will help me keep track of what i have to get working on), and eventually my ripped music as well.  currently all the stuff i have backed up is on hundreds of random cds and dvds...along with backing them all up to the server, i will organize these into a library of sorts, so i want to use a disc database system to catalog them and label them all in jewel cases on a rack or other physical storage system as well.

thanks in advance for the input, and if this should go into another forum feel free to move it, mods.
CO Crüe Benchwarmer

Playback: Denon DVD-2910 > Denon AVR-3806 > Segue Doghouse Speaker Cable > B&W DM-610i / Klipsch RW-10 Subwoofer

Offline Brian Skalinder

  • Complaint Dept.
  • (28)
  • Needs to get out more...
  • *****
  • Posts: 18868
  • Gender: Male
Re: massive storage options
« Reply #1 on: February 20, 2007, 12:06:15 PM »
I think RAID is overkill for home applications when the data is not changing frequently, is not mission-critical, and super-speedy access is not truly required.  You can achieve reasonable redundancy for significantly less $$$ with an external HDD enclosure (or two) and a bunch of HDDs.  Unless you're using high quality RAID controllers (doubtful on most consumer-ish NAS solutions), recovering from RAID failure may prove dicey.  I don't think it's worth the hassle and potential risk.  Granted, you'll need more HDDs in a reasonable redundancy setup to achieve your total storage goals, but IMO reasonable redundancy:

  • offers easier set up / configuration
  • provides for easier and more reliable recovery
  • enables easier storage expansion to address your growing capacity needs
Milab VM-44 Links > Fostex FR-2LE or
Naiant IPA (tinybox format) >
Roland R-05

Offline pfife

  • Emperor of Ticketucky
  • (0)
  • Needs to get out more...
  • *****
  • Posts: 12354
  • I love/hate tickets.
Re: massive storage options
« Reply #2 on: February 20, 2007, 12:07:04 PM »
I did what Brian suggests above, and I think its pretty sweet, using 2 internal 320gb drives on a 1ghz machine running XP.

Tickets are dead to me.  Except the ones I have, don't have, and lost.  Not to mention the ones you have, don't have, and lost.   And the ones that other dude has, doesn't have, and lost.  Let me know if you need some tickets, I'm happy to oblige. 

Tickets >>>>>>>> Oxygen

Offline dnsacks

  • (9)
  • Taperssection All-Star
  • ****
  • Posts: 1640
Re: massive storage options
« Reply #3 on: February 20, 2007, 12:16:07 PM »
office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.


Offline dcaley

  • (2)
  • Taperssection Member
  • ***
  • Posts: 862
  • Gender: Male
  • Team Toronto
    • http://www.davecaley.com
Re: massive storage options
« Reply #4 on: February 20, 2007, 12:19:55 PM »
ive been running 4x200gb sata seagate drives in raid5 for a few years now in my own pc and im pretty much out of space. ive been looking to upgrade to around the same amount as you, 1.5-2tb. but the majority of the new raid cards use the pci-x and pci-e slots which my mainboard doesnt have, so i might be looking at a rather large upgrade. ive been looking at the 320gb seagate sata2 drives as they are ~100$ and an 8 port raid card. not sure which brand, but either promise, highpoint, or 3ware. depends on the cash. im not familiar with nas boxes or how throughput and access times would differ between one and having the hdd's in your pc, if they differ at all.

for 1500$ you could grab 8 320gb sata2 drives, or maybe even 400gb drives, and an 8 port raid card and might have a bit of cash left over to upgrade the ecc memory on the raid card. that'd give you around 2tb or 2.5tb respectively after all is said and done.

Offline kfrinkle

  • (0)
  • Needs to get out more...
  • *****
  • Posts: 2082
  • Gender: Male
    • Karl Frinkle Dot Net
Re: massive storage options
« Reply #5 on: February 20, 2007, 12:23:05 PM »
office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.


Where did you find this out?  I was planning on buying one of those drives next month, but if they are on sale this month....  I do not see it on their website.
-Karl
MBHO KA200n's > Naiant Tinybox (v2.1) > Sony PCM-M10
http://tooldriveproject.net

Offline Simp-Dawg

  • Bad Little Dawggie
  • (2)
  • Needs to get out more...
  • *****
  • Posts: 15077
  • Gender: Male
  • Daddy needs a drink!
    • Colorado Tapers
Re: massive storage options
« Reply #6 on: February 20, 2007, 12:42:38 PM »
I think RAID is overkill for home applications when the data is not changing frequently, is not mission-critical, and super-speedy access is not truly required.  You can achieve reasonable redundancy for significantly less $$$ with an external HDD enclosure (or two) and a bunch of HDDs.  Unless you're using high quality RAID controllers (doubtful on most consumer-ish NAS solutions), recovering from RAID failure may prove dicey.  I don't think it's worth the hassle and potential risk.  Granted, you'll need more HDDs in a reasonable redundancy setup to achieve your total storage goals, but IMO reasonable redundancy:

  • offers easier set up / configuration
  • provides for easier and more reliable recovery
  • enables easier storage expansion to address your growing capacity needs


to expand on my needs...

first of all, initially, the data will be changing frequently...here's my steps to get this set up:
1. transfer all data on backup cds and dvds to the HDD storage system
2. organize tracked/completed recordings in one area, and unfinished recordings in another
3. import all finished and unfinished recordings from various hard drives, organize accordingly
4. begin processing on all uncompleted files, or at least correctly label them and sort accordingly
5. continue adding new recordings to library

after that (as well as cataloging the backup discs in a practical way), i intend to set this server up with a slingbox or something similar so i can access all this content through my stereo.

with all i've read it just seems like RAID 5 is the best solution for speed and redundancy.  plus having a NAS box or building a server of my own keeps all this stuff in one box rather than a number of hd enclosures that may not be able to be accessed wirelessly if my computer is off or whatever.

granted, if 500 gb's is $150, i could get 2TB (4 TB before reasonable redundancy) for $1200, but i have no way other than my windows PC to manage them and make them accessible to the network...unless i set up one of my older machines to do this
CO Crüe Benchwarmer

Playback: Denon DVD-2910 > Denon AVR-3806 > Segue Doghouse Speaker Cable > B&W DM-610i / Klipsch RW-10 Subwoofer

Offline dnsacks

  • (9)
  • Taperssection All-Star
  • ****
  • Posts: 1640
Re: massive storage options
« Reply #7 on: February 20, 2007, 12:46:02 PM »
office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.


Where did you find this out?  I was planning on buying one of those drives next month, but if they are on sale this month....  I do not see it on their website.

It was in their weekly ad (for colorado, at least) -- check out www.salescircular.com for particulars.  I recently picked up the same drive from dell for under $160 shipped --

Offline silentmark

  • Shine with or without cherries ?
  • (5)
  • Needs to get out more...
  • *****
  • Posts: 2682
  • Gender: Male
  • Boat, cucumber, wire ...
Re: massive storage options
« Reply #8 on: February 20, 2007, 12:46:53 PM »
i will organize these into a library of sorts, so i want to use a disc database system to catalog them and label them all in jewel cases on a rack or other physical storage system as well.

thanks in advance for the input, and if this should go into another forum feel free to move it, mods.

I am going through this process right now. I finally couldn't take the mess and having no clue of what music I actually have, or don't have.

For disc storage I am using these: http://www.meritline.com/cd-dvd-510-holder-wallet-holders-wallets-ehj510.html , each sleeve is numbered. With that number I setup an excel spreadsheet which relates to what particular disc is in what sleeve. Filtering is simple, by artist, date, venue, source, etc. I also setup a hotlink to each particular show's text file, so with a click click I can call up all the info about the show.

It is abit time consuming but I am seeing the light at the end of the tunnel. After this, next up is the DAT's ...

For HD storage, I thought about a RAID setup, but opted for simple redundant HD's, works well so far ...
« Last Edit: February 20, 2007, 12:49:35 PM by silentmark »
Dissent is the highest form of patriotism. - Howard Zinn, not Thomas Jefferson ...

Mics: Neumann AK50/AK40/AK30/AK20(1 for M/S), AKG568eb's (gathering dust)
Decks: R-44 (OCM), Fostex FR2LE (OWM), Microtacker (semi-retired), D8 (retired), D7 (retired)
Pre-amps: Apogee Minime (semi-retired), Sonosax SX-M2 (semi-retired), Oade mod SBM-1 (retired)
Cables: LC3 actives (older lemo style x2), Audio Magic Hyper Conductor interconnects

http://www.youtube.com/watch?v=J3qrWOOposQ

Offline it-goes-to-eleven

  • (58)
  • Needs to get out more...
  • *****
  • Posts: 6696
Re: massive storage options
« Reply #9 on: February 20, 2007, 12:50:35 PM »
External drives are very useful and definitely have their place..  but I'm not keen on most external enclosures because they don't adequately cool drives.. Especially BIG drives.  There is also the potential for data corruption due to loose external cables, etc..

I've been buying WD 3200JB's and 3200JD's for a few years.. I have around 16-18 of them. They've been Very good to me.  But it appears that it is finally time to move up.  I wish the storage price-performance was moving faster.. The capacity just isn't growing fast enough.

These look good to me in terms of price/performance.  3 year warranty.  This is a 3 platter, so it should run cooler than the previous 4 platter designs.  Review feedback seems good.. $140 with free ship is 28 cents/gig raw.

http://www.westerndigital.com/en/products/products.asp?driveid=214&language=en
http://www.newegg.com/Product/Product.asp?Item=N82E16822136073&name=Item-%23%3A-N82E16822136073

So far, I have been protecting my data with my workflow rather than raid.  Recordings come off the 722 and go to a linux server.. then are copied temporarily to a usb drive and then copied from there to the windows box.. and then backed up to dvd on the linux box.  So the copies exist in multiple places but not in raid.  Given that consumer raid is pretty flakey (compared to real enterprise), I consider this safer.. in most cases it is also far cheaper.  I'm sticking about 10 drives in each PC so the total cost per gig is very low.

I think my next step is to stuff a bunch of drives in a case and then run the sata cables back to the main pc.  Not quite sure about shielding the sata cables yet or what case I'm going to use.

I've been using these 4 port SATA controllers.. $25... Can't beat the per-port cost. I have used two per system but don't know if you can use more than that.

http://www.newegg.com/Product/Product.asp?Item=N82E16815124020


New product - Get your NEUMANN active mounting bars
Active microphone mounting bars for Schoeps and AKG, and Naiant AKG, ready to ship

Offline Brian Skalinder

  • Complaint Dept.
  • (28)
  • Needs to get out more...
  • *****
  • Posts: 18868
  • Gender: Male
Re: massive storage options
« Reply #10 on: February 20, 2007, 01:22:39 PM »
first of all, initially, the data will be changing frequently...

I don't think any of your goals / requirements are inconsistent with a reasonably redundant setup.  By constantly changing data, I mean real-time transactional data.  Not an hour or two's (or even a bunch of hours) worth of copying data from CD/DVD to HDD every evening.

with all i've read it just seems like RAID 5 is the best solution for speed and redundancy.  plus having a NAS box or building a server of my own keeps all this stuff in one box rather than a number of hd enclosures that may not be able to be accessed wirelessly if my computer is off or whatever.

RAID -can- provide good redundancy.  But IME only if you get a quality (read $$$) controller.  I've had a couple different arrays fail using consumer-ish controllers that rendered the array unrecoverable.  Not fun.

As for enclosures, you can buy multi-slot enclosures, so you don't need to have a slew of them cluttering up space.  As FL mentions, just make sure it has adequate ventilation (just as you'd need on your NAS box).

The only real issue I see with what you've mentioned so far is the need to have a computer on in order to access the HDDs.  My PC's on all the time anyway, so I it's not a problem for me to simply tack the external enclosure onto my primary desktop.

At any rate, whatever you decide - have fun!  I'm sooooo happy I put the effort into migrating all my optical media to HDD for storage (but not backup, of course) and playback.  It'll take some time, but I suspect you'll really enjoy the results.
Milab VM-44 Links > Fostex FR-2LE or
Naiant IPA (tinybox format) >
Roland R-05

Offline Wiggle

  • (0)
  • Taperssection Member
  • ***
  • Posts: 385
  • Gender: Female
  • So soft. Clearly meant to be loved by humans.
Re: massive storage options
« Reply #11 on: February 20, 2007, 01:23:45 PM »
office depot has 500gb external western digital mybook drives on sale this week for $149 apiece -- these are nice external drives and have the added advantage of automatically shutting down when not in use.


Where did you find this out?  I was planning on buying one of those drives next month, but if they are on sale this month....  I do not see it on their website.

FWIW, I just picked up a WD My Book 250gig drive for $90 off of eBay this morning. There appears to be a glut of them on there and if you pick the right auction they seem to be going pretty cheap. Of course, it cost me an additional $30 to ship to Canada - YMMV.
Open: SP C4/MSH-1O > Nick's Picks hydra cables > BM2P+ UA-5 > iRiver H120
Optional: Giant Squid cards > iRiver H120

Offline cleantone

  • (2)
  • Taperssection All-Star
  • ****
  • Posts: 1676
  • Gender: Male
Re: massive storage options
« Reply #12 on: February 20, 2007, 01:26:48 PM »
Office Depot's WD products right now.

I wish that box was Firewire. I don't have USB2 on my older TiBook. That plus the fact that the closest store is 25 miles away I might not jump on that as I thought I would.

The 500GB USB WD for $150

I just found 200GB Maxtor internal for $40 at Staples too! Can't turn that down right now. Maybe two even!
« Last Edit: February 20, 2007, 01:29:49 PM by cleantone »
ISO: your recordings of The Slip, Surprise Me Mr. Davis and The Barr Brothers. pm me please.

Offline heikki

  • (1)
  • Taperssection Member
  • ***
  • Posts: 880
  • Gender: Male
  • ...and i'm as honest as a jersey man can be
    • OutDated List
Re: massive storage options
« Reply #13 on: February 20, 2007, 01:27:08 PM »
For disc storage I am using these: http://www.meritline.com/cd-dvd-510-holder-wallet-holders-wallets-ehj510.html

ooooohhhhhh, i REALLY like that. 
While I do store all my new masters and downloads on HDDs I still back up all my new masters on CD/DVD and have a lot of old DAT > SHN/FLAC conversions that have not yet been copied to HDD.

Secure HDD back-up is so the way to go.  All of your music is organized, redundantly stored, and most importantly available for listening!

RAID5 is great efficiency, but as others mentioned if you don't have a good card it can be slow and unreliable.  I use the RAID5 NAS strictly for long-term storage and as a kind of "jukebox".  I use the RAID10 as my normal HDD.  I find it a bit quicker than RAID5 and still very reliable, though not as efficient.  This is where my OS, apps, docs, and in-process music resides.

-Heikki
4x500GB RAID5  NAS Box Archive/Jukebox
4x250GB RAID10 PC Working Storage
C 480 B + CK61/63
          Edirol R-4 (Oade T-Mod x 4)

Masters on LMA: http://www.archive.org/bookmarks/heikki

Offline kfrinkle

  • (0)
  • Needs to get out more...
  • *****
  • Posts: 2082
  • Gender: Male
    • Karl Frinkle Dot Net
Re: massive storage options
« Reply #14 on: February 20, 2007, 01:41:24 PM »
Office Depot's WD products right now.

I wish that box was Firewire. I don't have USB2 on my older TiBook. That plus the fact that the closest store is 25 miles away I might not jump on that as I thought I would.

The 500GB USB WD for $150

I just found 200GB Maxtor internal for $40 at Staples too! Can't turn that down right now. Maybe two even!

Awesome, thanks!
-Karl
MBHO KA200n's > Naiant Tinybox (v2.1) > Sony PCM-M10
http://tooldriveproject.net

 

RSS | Mobile
Page created in 0.065 seconds with 43 queries.
© 2002-2021 Taperssection.com
Powered by SMF