Taperssection.com
Gear / Technical Help => Post-Processing, Computer / Streaming / Internet Devices & Related Activity => Topic started by: morst on July 28, 2008, 01:14:14 PM
-
Last fall I got the Zoom H2 solid state recorder, and ever since then, I have been a little concerned about keeping "masters" of all my files. When I recorded on cassette and DAT, it was easy to keep and keep track of masters since they existed in 3-d space. Now I wonder how many copies I need to make before I believe that the file will exist in the future.
So I just got a pair of matching 750GB Seagate PATA drives to help max out my storage capacity on my powermac G4, and I was thinking I would install them as a RAID1 Mirror for reliablility. But now I am thinking that I could keep them separate, and just use one for the original WAV files, and one for FLAC backups. If a file is deleted or modified on a RAID setup, it changes it on both drives, and you don't really have a backup. Because the RAID mirror system makes the contents of both (or more) drives identical, it's possible to really screw things up.
Does anyone have any experience with RAID, good or bad, that they could share to help illustrate which way they would suggest I go with my new storage space?
Thanks!
-
I like to keep a copy of my recordings offsite just in case something horrible happens (house burns down, flood, computer stolen, etc).
I keep the other HD that I would use in the RAID in my safe deposit box at the bank. I update it every six months or so. Which reminds me...last time I did that was in Jan. Time to go to the bank.
RAID + off site would be a good option IMO.
-
To RAID or not for me depends on <1> my budget, and <2> whether it's critical to have real-time redundant data. IME, inexpensive / consumer-level RAID controllers aren't terribly reliable, which means I'd have to budget more than I'd like for a HQ RAID card. And personally, my critical data doesn't change frequently enough to warrant (real-time) RAID, so implementing RAID would be overkill. So instead, I employ reasonable redundancy without RAID (http://taperssection.com/index.php/topic,65307.0.html), in my case mirrored on a nightly basis. I'll never lose enough data on a daily basis that it will cause undue harm or inconvenience. Others may define a different period between mirroring, like 12 hrs, 8 hrs, 4 hrs, etc., depending on individual needs.
Regarding your concern about deleting a file, for example - that's what optical / tape / off-site backups are for. Redundancy simply allows swift recovery from a single-point failure. Don't count on a redundant HDD system as your backup - it'll cause you great pain, eventually.
Edit to add: Free FileSync (referenced in the Reasonable Redundancy link above) allows one to configure it to maintain files on the target / "mirrored" HDD even if they're deleted on the source HDD. This means that if you accidently delete a file, it'll still reside on the target / "mirrored" HDD. The downside to using this option: it could potentially eat up a lot of space, eventually.
Edit to add again: Recommend having at least 3 separate instances of your data. I have my redundant HDDs, optical backups, HDD offsite backup, and for most recordings also an offsite backup in the form of music fans who've downloaded my recordings.
-
I like the old post by Brain very, very much. Get two matching hard drives and some kind of software to back it up automatically every night.
hard drives are cheap these days so you could get two 500gig drives and write to one and back up to the every day. Since I have--too much--faith in hard drives, I would do very little back up work except when I am working on a major project. Hell, get thumb drives as well and you can move it from machine to machine without anything but plug-n-play.
Don't forget, you can also upload to servers online all over the world if you want to do that.
Thanks for that post.
-
i don't use any software, just do it all manually - its really not too much effort if you keep on top of it and its much more flexible and failsafe IMO
i keep a txt file with all the folders/files that have had the checksum performed and a spreadsheet with the contents of each drive. it allows me to be much more flexible by using different sized drives - i also keep an external (which needs to be updated) at my parents house for offsite safety
at any given time aside from new recordings I have at least 3 raw/master backups of my recordings - may be overkill but the price of HDDs is cheap enough for the peace of mind
i also keep backups of all my Samplitude Virtual projects as well as CD Wave cue sheets and have a single external drive with backups of all tracked/tagged shows that i've recorded
-
Working:
RAID 1 mirror, online
USB drive backup (robocopy)
Archive:
DVD+r and USB HD of all masters and tracked shows
-
Great responses. +t's all around. I am thinking that I will use try to figure out how to schedule rsync to automatically keep the backup drive updated, and then get one more large hard drive for offsite backup. I am already making at least two copies of everything on two different brands of optical media, and uploading what I can to archive.org, or various torrent sites.
If anyone else has any comments, please continue to post 'em! 8)
-
One other nice option for data storage is the Drobo, it's not the highest performance unit, but for simple easy redundant data storage it is a great option. It does NOT give you an offsite copy, but I like them a lot. The original units can be found in the $350 range on Amazon and eBay right now brand new w/o any drives.
-
A consideration may also be Mozy online, offsite backup. I'm in the process of backing up every bite of data on my two harddrives-- you can get unlimited storage space for just $4.95 / month. Definitely worth checking into imo
https://mozy.com/home
-
I went w/ RAID 0 (striping) for a PC I built (w/ some help from friends). I wanted it to be super fast for this program at work that has a lot of Disk I/O. It worked great, then my BIOS got corrupted somehow, now I can't even get to Windows. All the PC gurus I talk to say tha because I didn't have a RAID controller (used the RAID functionality on the mobo) that it is next to impossible to get the data off the discs. My lesson, don't rely on RAID 0 w/out having a physical RAID controller. Luckily I made backups of the important files. I realize this isn't exactly what you are asking but figured I would share the experience.
-
Great responses. +t's all around. I am thinking that I will use try to figure out how to schedule rsync to automatically keep the backup drive updated, and then get one more large hard drive for offsite backup.
Was going to suggest using rsync to an external drive.
I run mostly internal drives, and use software to backup overnight to a 500GB external once a month. I don't practice off-site redundancy, so keeping the drive in its original box and in my dark closet is as good as it gets right now.
I have run a RAID0 for 3 years now and not had a drive fail *knock on wood* but that's probably because I leave my machine running almost 24/7, except in the summer time when it's too hot. The start-up/shut-down of the disks puts the most wear on them an d makes them more prone to fail. If you run an external RAID, I always suggest keeping those disks turned on and spinning. Think of the extra electricity as long-term sustainability for your data.
-
A consideration may also be Mozy online, offsite backup. I'm in the process of backing up every bite of data on my two harddrives-- you can get unlimited storage space for just $4.95 / month. Definitely worth checking into imo
https://mozy.com/home
How has this worked out for you? It looks like you have to install software; what protocol does data transfer through (http or ftp) and how was the upstream speed? Also, can you download files via a website link or do you need to access it using the software?
-
I went w/ RAID 0 (striping) for a PC I built (w/ some help from friends). I wanted it to be super fast for this program at work that has a lot of Disk I/O. It worked great, then my BIOS got corrupted somehow, now I can't even get to Windows. All the PC gurus I talk to say tha because I didn't have a RAID controller (used the RAID functionality on the mobo) that it is next to impossible to get the data off the discs. My lesson, don't rely on RAID 0 w/out having a physical RAID controller. Luckily I made backups of the important files. I realize this isn't exactly what you are asking but figured I would share the experience.
Never rely on RAID 0 for primary data storage it has no redundancy, just improved performance. It should be used a scratch disks to work with data that is backed up and stored elsewhere.
-
A consideration may also be Mozy online, offsite backup. I'm in the process of backing up every bite of data on my two harddrives-- you can get unlimited storage space for just $4.95 / month. Definitely worth checking into imo
https://mozy.com/home
How has this worked out for you? It looks like you have to install software; what protocol does data transfer through (http or ftp) and how was the upstream speed? Also, can you download files via a website link or do you need to access it using the software?
I have used Mozy for backup of my Photography files for about 8 months. It works great but the initial upload (~400 GB) took months. Since then it's set and forget. As I add data (typically 6-8 GB chunks) to the folder that I store my images in it picks them up and uploads overnight and usually completes within a day or two. Recovery can be time consuming as it;s either a massive download or you pay extra for them to burn to DVD or other media and mail it to you.
You also need to be aware, it is not archival storage. If you delete data and dont restore it within 30 days Mozy assumes this was on purpose and delete it from their servers as well.
-
well you cant 'rely' on RAID0 at all, as its designed for performance, not redundancy, and in fact has a higher failure rate than a single drive.
im sure your data is recoverable, it has nothing to do with it being an onboard chip, either you know the chip mfr and stripe size, or you dont.
RAID0 is a lot less complex than RAID5 or 6, it should be recoverable.
I went w/ RAID 0 (striping) for a PC I built (w/ some help from friends). I wanted it to be super fast for this program at work that has a lot of Disk I/O. It worked great, then my BIOS got corrupted somehow, now I can't even get to Windows. All the PC gurus I talk to say tha because I didn't have a RAID controller (used the RAID functionality on the mobo) that it is next to impossible to get the data off the discs. My lesson, don't rely on RAID 0 w/out having a physical RAID controller. Luckily I made backups of the important files. I realize this isn't exactly what you are asking but figured I would share the experience.
-
You also need to be aware, it is not archival storage. If you delete data and don't restore it within 30 days Mozy assumes this was on purpose and delete it from their servers as well.
This part is VERY important to understand. Unless you have room on your main computer's hard drive to permanently store your masters, this is not a good option.
-
best option, store on two separate hard drives and have a good friend also copy all your shows on one (or more) hard drives...worked for me so far. it gets expensive having multiple copies on HD's, but the gear I own to record the shows is more expensive, so it's all relative.
last but not least....share your recordings, do not keep them to yourself. if you have hard drive failure, you'll be happy you did that bittorrent.
-
Working:
RAID 1 mirror, online
USB drive backup (robocopy)
Archive:
DVD+r and USB HD of all masters and tracked shows
This is essentially my set-up, and it works like a charm on my Mac. I use Super Duper (http://www.shirt-pocket.com/SuperDuper/SuperDuperDescription.html) to do my HD back-ups automagically. I'm also running Time Machine to a Time Capsule for good measure... too many HDs. ???
-
Most redundant inexpensive option is RAID 1 for online - To allow for a hard disk failure and not lose data. You can get the DLink DNS-321 (I believe) for about $100 shipped these days -> this is a empty NAS RAID 1 box that allows you to add two of your own drives (use blank drives). Attaches the drives to your Ethernet network through a GigE port.
With RAID you are not garanteed to be able to recover your data if the controller fails - some controllers you can and some you can't.
As others have noted, I would keep 2-3 copies on differing media and definitely keep at least one copy offline (i.e. DVD-R or Large Cap Tape backup - LTO/SDLT, etc)
-
can anybody recommend a (free?) program that will allow me to perform an initial complete and subsequent periodic incremental backups of my 1.5tb raid 5 NAS array onto a series of smaller disks (i.e. IDE hard drives connected to my vista pc via usb2). I've yet to find a program that will allow for backup of one BIG disk to a series of smaller ones.
FWIW, I'm using a buffalo terastation pro and, while I have not had any problems with it, none of its drives (4x 500gb sata drives) have yet failed and I've therefore not yet been faced with rebuilding the array.
-
How has this worked out for you? It looks like you have to install software; what protocol does data transfer through (http or ftp) and how was the upstream speed? Also, can you download files via a website link or do you need to access it using the software?
Mozy is a superb backup solution. I use it on my PC and have also played with it on my Mac (although I now just copy all important data from my Mac to my PC, from where Mozy backs it up without my needing a second Mozy account). You do have to install some software, but it's pretty lightweight on both platforms and very easy to use. I don't know what protocol it uses as it just handles everything automatically. Mozy reports sustained upstream rates of over 700kpbs for me. I'm connected via ethernet to a cable modem with a stated upstream limitation of 800kbps, so Mozy seems to take advantage of pretty much my full upstream capacity (note, this sometimes causes issues with people hearing us clearly when we call them using our Vonage phone while Mozy is running a backup). It sometimes seems to "pause" in the middle of a big backup (it'll say a backup is in process, but will sit idle at 0kbps for a period of time before resuming uploading), which can be frustrating, but if you just leave your computer on and connected all the time, it'll eventually catch up as long as you aren't throwing gigs of new info at it every day. With my 800kbps upstream cable modem, it can do a few gigs in a day if I'm not heavily using the computer.
All that said, Mozy is a backup solution and NOT an archival solution. It is designed to make copies of things that are on your computer. If you delete something from your computer, Mozy will mark it deleted on their servers as well. They'll keep it for 30 days in case you change your mind (they also keep previously-backed-up versions of files you change for 30 days), but after that, they'll delete it. So you can't use your unlimited space with Mozy to shift data off of your computer, you can only use it to make backups of the things that you keep on your computer. It will also work with external hard drives that are attached directly to your computer, but the "home" version won't work with anything your computer sees as a "removable" disk (e.g., your SDHC card reader or your optical drive) or as a network share (i.e., it will NOT backup a NAS).
I also have not yet had to do a restore. You have a choice of restoring files through the Mozy software client, or directly through their website, or by paying them to mail you DVDs of your data (which I would think would probably be the solution I'd use if I ever need it, as it will take a long time to download the couple hundred gigs of data I've backed up).
Compared with RAID 1, Mozy is a vastly superior backup solution. RAID 1 only protects against a physical drive failure in one of the two drives. If the array is stolen, destroyed by water or fire, or if you accidentally delete something you didn't mean to delete, RAID 1 will do you no good, while Mozy protects against all of these things. The only thing RAID 1 does better than Mozy is to add real-time redundancy, so that if a drive fails before Mozy has had a chance to back up some new data, you can probably recover your data from the other drive. Ideally, you could combine RAID 1 with Mozy to get the advantages of both. But if your budget only allows for an either/or approach, I'd much rather spend $5 a month for real backup with Mozy than a couple hundred bucks up front for the limited protection of RAID 1.
Mozy also has a free version that allows you backup up to 2GB, so you can try that and if you like it, you can upgrade to the $5/mo unlimited data version.
And now for the selfish part: If you sign up through this link (http://www.mozy.com/?ref=3f9a896b&kbid=38325&m=4&i=76), I'll get a small referral commission. I don't work for them, I'm just a member of their affiliate program and really like their product, so I recommend it to everyone. But if you want to sign up directly, I won't be offended (this board has been enormously helpful to me, and y'all don't owe me squat).