Become a Site Supporter and Never see Ads again!

Author Topic: Reasonable redundancy without RAID: protecting the music (data) on your HD  (Read 7205 times)

0 Members and 1 Guest are viewing this topic.

Offline spott

  • Trade Count: (0)
  • Taperssection All-Star
  • ****
  • Posts: 1100
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #15 on: October 01, 2007, 07:51:56 PM »
Here's a free virtual Network Attached Storage (NAS) solution that runs on a FreeBSD platform http://www.freenas.org/ - a lot of people are using this to support the SlimServer (Squeezebox) software. I've been trying to find a large volume *secure* storage solution as well. 

There's also a variety of appliance based solutions to choose from; Drobo, ReadyNAS, Windows Home Server (due this month), Buffalo Terastation, etc. - most of these are SATA drive enclosures that have a RAID similar protection (either OS or hardware RAID). 



kskreider

  • Guest
  • Trade Count: (0)
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #16 on: October 02, 2007, 12:20:27 PM »
Good advise, but you can save yourself $25 with a little simple scripting.
On Windoes make a batch file (a text file with a .bat extension) with something like this in it:

xcopy d:\*.* e: /s /d

Then setup the command scheduler to run that at whatever interval you like.

Great tip, Matt!  I should've thought of it.  And now, due to a fatal flaw in Argentum Backup (see original post), I'm using a similar batch file:

Code: [Select]
set mydate=%date:~10%-%date:~4,2%-%date:~7,2%
set mytime=%time:~0,2%-%time:~3,2%
xcopy i: j: /d /s /c /f /y > f:\z_backup\log\I-J_%mydate%_%mytime%.log

set mydate=%date:~10%-%date:~4,2%-%date:~7,2%  |  captures the date for use in my log filename
set mytime=%time:~0,2%-%time:~3,2%  |  captures the time for use in my log filename
i:  |  my source drive
j:  |  my destination drive
/d  |  copies any new or updated files from source to destination
/s  |  copies all subdirectories and files
/c  |  continues copying through errors (not positive I want to do this, yet)
/f  |  displays full source and destination paths/filenames I capture in the log
/y  |  suppress prompting to confirm an overwrite of an existing destination file
>   |  sends the results of the operations to a log file
f:\z_backup\log\i-j_%mydate%_%mytime%.log  |  path and filename of the log file

This option will only copy new or updates files from source to destination, and does not deletions at all.

Can't use SyncToy or Robocopy (referenced in another thread) since I'm on Win2K and have no desire to change.

I have been using xcopy for about 4 years now.  It is SOoo easy (as you can see by Brian's script)  I did not know about the log file though so I just added that feature.  Now if someone could help me with a command line FTP script that would be worth a few T's to me.

Offline it-goes-to-eleven

  • Trade Count: (58)
  • Needs to get out more...
  • *****
  • Posts: 6702
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #17 on: October 02, 2007, 12:27:39 PM »
I'd strongly recommend an rsync based approach over xcopy. Here's why:

rsync uses checksums to verify that data has been copied correctly.  You can easily have rsync verify that a copy is correct. It will recalc the checksum on every source file and every destination file. I don't think xcopy uses checksums?

New product - Get your NEUMANN active mounting bars
Active microphone mounting bars for Schoeps and AKG, and Naiant AKG, ready to ship

kskreider

  • Guest
  • Trade Count: (0)
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #18 on: October 02, 2007, 02:37:38 PM »
I'd strongly recommend an rsync based approach over xcopy. Here's why:

rsync uses checksums to verify that data has been copied correctly.  You can easily have rsync verify that a copy is correct. It will recalc the checksum on every source file and every destination file. I don't think xcopy uses checksums?


I will give DeltaCopy a whirl then.  Thanks for the info.  I love MD5's and verification tools.  They are our friends. 

Offline Brian Skalinder

  • Complaint Dept.
  • Trade Count: (28)
  • Needs to get out more...
  • *****
  • Posts: 18886
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #19 on: October 02, 2007, 02:44:26 PM »
Ah, excellent...I'll give DeltaCopy a try.  Thanks!

Working great, after I enabled Protected Storage service to fix a scheduling problem.  Thanks again.  :)
Milab VM-44 Links > Fostex FR-2LE or
Naiant IPA (tinybox format) >
Roland R-05

Offline davepeck

  • Trade Count: (17)
  • Taperssection All-Star
  • ****
  • Posts: 1107
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #20 on: October 02, 2007, 05:47:17 PM »
Ah, excellent...I'll give DeltaCopy a try.  Thanks!

Working great, after I enabled Protected Storage service to fix a scheduling problem.  Thanks again.  :)

excellent. glad to hear it. i've been using it for a while now, and it definitely does the trick. i backup 1 drive a night on my PC.
Telefunken M60 + TK60 / TK61 / TK62 > Tascam DR-70D

Audio | Video | Photo

Offline jerryfreak

  • Trade Count: (20)
  • Needs to get out more...
  • *****
  • Posts: 2300
  • I'm a llama! (taboot)
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #21 on: October 06, 2007, 02:02:52 AM »
with all respect, unless someone is hardware illiterate, a RAID0 card (as cheap as`$20, top quality under $60) is a better alternative, because it eliminates the requirement to manually back up.

the biggest failure point of a backup system is human intervention. the possibility of making a mistake and copying the wrong drive over, or simply forgetting to back up is very real.

if you can install a hard drive, you can install a RAID card

Argentum costs money and your computer has to be on all the time to take advantage of scheduled backups


Originally posted this in another thread, but thought it deserved it's own thread:

Just a thought for some folks not wanting to mess around with RAID, or wanting to save some money:  pick up multiple discs and do your own automated (non-real time) redundancy handling.  For example:

  • Get two 250 GB USB/Firewire/internal drives; designate one drive X and the other drive Y
  • Store all your data on drive X (use drive Y as your backup)
  • Install an app like Argentum's Backup ($25) to automate duplicating the data from drive X to drive Y on some fixed schedule (weekly, daily, hourly, whatever suits your purposes)
  • After an initial copy of all of drive A's data to drive B, the app will perform only incremental backups, i.e. the app will only copy new or changed data;  I bet there're free apps available, too, or you could even write your own batch file easily enough, but FWIW I like Argentum's Backup because it's cheap and has the option to store data in non-proprietary formats (unlike many s/w backup apps)

START EDIT

I cannot recommend Argentum Backup any longer and strongly recommend anyone using it STOP, and find another solution (I'm now using an xcopy script, see my latest reply to the thread).  I'm kicking myself for not finding this fatal flaw earlier (a result of no real testing of my backup routines until just now...dumb dumb dumb).  Argentum Backup's fatal flaw:  if the source is not available for any reason - network goes down, drive goes offline / crashes for any reason, etc. - Argentum assumes the user intentionally deleted / removed the source and will update the destination (backup) accordingly, i.e. it will delete the destination (backup)!  WTF?!?  Example:

  • I have all my data stored on my F: drive.
  • I want to backup my data from F: onto my G: drive.
  • I set up Argentum Backup (AB) to backup my data from F: to G: nightly.
  • AB runs the first time, and copies all data from F: to G:.
  • The next day, my F: drive suffers a catastrophic failure.
  • AB runs again.
  • AB deletes all the files on G:.

I emailed AB support about this problem and they basically said:  it's not a bug, it's by design, and changing it would be really difficult.  What a lame response.

END EDIT

Not real-time, but it will provide redundancy protection against (daily / hourly / whatever) losses, depending on how frequently you perform the automated backup.  I figure I can recover from any one day / hour's losses easily enough.  It's the loss of many days (weeks, months) worth of data that scares me - I have no desire to re-transfer 300 GB of data from CD-R/DVD-R to HD.

Note 1:  I say "redundancy" and not "backup" because storing data on multiple HDs is not really an effective backup strategy
Note 2:  RAID or the above scenario is NOT a sufficient backup program.  A comprehensive backup plan should also include physical backups on optical (CD-R/DVD-R) or magnetic (tape) media and off-site backups.
in:
small: MK4/4V > CMR > V3 or J.Williams Mod MicMan Jr or no pre at all >AD2K/R07/PCM-A10/Oade Warm 661/DR100-MKIII

smaller: 4061/4099 CORE or 4011/4018V> d:vice MMA



out:
home:Benchmark DAC1 HGC > Dynaudio BM15A active monitors
shop: Musical Fidelity VLink 192 Asynchronyous> coax> DAC1 > Rokit RP8 active monitors

Offline Brian Skalinder

  • Complaint Dept.
  • Trade Count: (28)
  • Needs to get out more...
  • *****
  • Posts: 18886
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #22 on: October 06, 2007, 01:22:12 PM »
with all respect, unless someone is hardware illiterate, a RAID0 card

I assume you meant RAID1.

When people who do storage for a living and are considered experts in the field say they won't trust consumer raid solutiatons, I listen to'em..

The above, combined with two unfortunate personal experiences with cheap RAID cards, convinced me to bail on cheapo RAID solutions.  Not going back to the (IMO) false sense of security from cheapo RAID cards, thanks, and think others should be aware of the pitfalls and alternative options.

the biggest failure point of a backup system is human intervention. the possibility of making a mistake and copying the wrong drive over, or simply forgetting to back up is very real.

Once scheduled, there's no human intervention required for DeltaCopy SyncToy Free FileSync.

Argentum costs money and your computer has to be on all the time to take advantage of scheduled backups

If you read the thread, I've edited to recommend -against- using Argentum (now recommend DeltaCopy SyncToy Free FileSync).  You're right that using these tools requires the PC to be on.  :shrugs:

As for it costing money, so do the cheapo RAID cards.  I'm willing to pay a few bucks for good software that does what I want it to do (unfortunately, Argentum doesn't, and fortunately, DeltaCopy SyncToy Free FileSync does and is free):  CDWave, for example, which it seems very few around here actually purchase, is a great, inexpensive option for tracking -- well worth purchasing a license, even though free alternatives exist.
« Last Edit: May 17, 2012, 11:40:05 AM by Brian Skalinder »
Milab VM-44 Links > Fostex FR-2LE or
Naiant IPA (tinybox format) >
Roland R-05

Offline jerryfreak

  • Trade Count: (20)
  • Needs to get out more...
  • *****
  • Posts: 2300
  • I'm a llama! (taboot)
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #23 on: October 06, 2007, 03:27:39 PM »
yeah, it was like 4am, i obviously meant raid1.

one other tidbit about consumer-level cards and raid recovery.

raid1 data is almost always written in a standard format, where in event of controller failure, one of the volumes could be easily read by a standard ide/sata controller. not so for raid0, which has a variety of block and stripe sizes generally specific to the controller.
in:
small: MK4/4V > CMR > V3 or J.Williams Mod MicMan Jr or no pre at all >AD2K/R07/PCM-A10/Oade Warm 661/DR100-MKIII

smaller: 4061/4099 CORE or 4011/4018V> d:vice MMA



out:
home:Benchmark DAC1 HGC > Dynaudio BM15A active monitors
shop: Musical Fidelity VLink 192 Asynchronyous> coax> DAC1 > Rokit RP8 active monitors

 

RSS | Mobile
Page created in 0.185 seconds with 36 queries.
© 2002-2019 Taperssection.com
Powered by SMF