Become a Site Supporter and Never see Ads again!

Author Topic: Reasonable redundancy without RAID: protecting the music (data) on your HD  (Read 12187 times)

0 Members and 1 Guest are viewing this topic.

Offline Brian Skalinder

  • Complaint Dept.
  • Trade Count: (28)
  • Needs to get out more...
  • *****
  • Posts: 18868
  • Gender: Male
Please note I now recommend FreeFileSync.

Originally posted this in another thread, but thought it deserved it's own thread:

Just a thought for some folks not wanting to mess around with RAID, or wanting to save some money:  pick up multiple HDDs and do your own automated (non-real time) redundancy handling.  For example:
  • Get two drives (internal or external, or one of each); designate one drive X and the other drive Y (or whatever letters you wish)
  • Store all your data on drive X and use drive Y as your redundant copy
  • Install and configure Free FileSync to mirror the entire drive X to drive Y.  Use a Task Scheduler for scheduling (I do mine nightly using Windows 7 Task Scheduler).
  • Once set up and configured, the initial run of Free FileSync will copy all of drive X's contents to drive Y.
  • Future runs (which users may schedule) will then only copy new files, update those that have changed, and delete the backups if the original is deleted.  (Alternatively, you may customize the sync rules for Free FileSync to follow, depending on your own requirements.)
While not real-time, the Free FileSync. solution will provide redundancy protection against (daily / hourly / whatever) losses, depending on how frequently you perform the scheduled sync.  I figure I can recover from any one day / half day / 4 hour (whatever) losses easily enough.  It's the loss of many days (or weeks, months) worth of data that scares me.

Note:  I say "redundancy" and not "backup" because storing data on multiple HDs is not really an effective backup strategy.  For proper backup, it's best to have at least one regulary-updated off-site copy.



I've tried and rejected the following, based on my experiences:

  • Argentum.  If the source drive (or directory) disappears, Argentum deletes the target drive, assuming the user intentionally deleted it and wants the synced target drive / directory deleted, too.  So if your source HDD fails, Argentum deletes your backup. Dumb!
  • DeltaCopy.  Generally works okay, but I've had consistent permissions issues with the synced directories & files.
  • SyncToy.  I know some people for whom it works very well.  But for me:  sometimes copies all the files it should, and sometimes doesn't.  Inconsistent = b-a-d.

« Last Edit: February 12, 2013, 12:15:48 PM by Brian Skalinder »
Milab VM-44 Links > Fostex FR-2LE or
Naiant IPA (tinybox format) >
Roland R-05

Offline live2496

  • Trade Count: (6)
  • Taperssection Member
  • ***
  • Posts: 692
  • Gender: Male
    • Gidluck Mastering
MD5 checksums can also be used to tell if a file is still good or the copy of it is good.

Gordon
AEA R88MKII > SPL Crimson 3 > Tascam DA-3000

Offline mattb

  • Trade Count: (3)
  • Taperssection All-Star
  • ****
  • Posts: 1965
  • Gender: Male
  • Yo.
    • Matt's Page
Good advise, but you can save yourself $25 with a little simple scripting.
On Windoes make a batch file (a text file with a .bat extension) with something like this in it:

xcopy d:\*.* e: /s /d

Then setup the command scheduler to run that at whatever interval you like.

Also, the new MS utility SyncToy for WinXP will do the same thing with a nice gui interface and a little smarter logic that will amke it faster and more efficent.
SyncToy can be downloaded from here:
http://www.microsoft.com/downloads/details.aspx?FamilyId=E0FC1154-C975-4814-9649-CCE41AF06EB7&displaylang=en

The help file shows you how to schedule it. I use it to back up my wife's laptop to my server (on a RAID5 volume) every night.
AT933s (C/H/O) > AT8531s > UA-5 > H120 (Rockboxed)

Offline Brian Skalinder

  • Complaint Dept.
  • Trade Count: (28)
  • Needs to get out more...
  • *****
  • Posts: 18868
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #3 on: September 27, 2007, 04:04:22 PM »
Good advise, but you can save yourself $25 with a little simple scripting.
On Windoes make a batch file (a text file with a .bat extension) with something like this in it:

xcopy d:\*.* e: /s /d

Then setup the command scheduler to run that at whatever interval you like.

Great tip, Matt!  I should've thought of it.  And now, due to a fatal flaw in Argentum Backup (see original post), I'm using a similar batch file:

Code: [Select]
set mydate=%date:~10%-%date:~4,2%-%date:~7,2%
set mytime=%time:~0,2%-%time:~3,2%
xcopy i: j: /d /s /c /f /y > f:\z_backup\log\I-J_%mydate%_%mytime%.log

set mydate=%date:~10%-%date:~4,2%-%date:~7,2%  |  captures the date for use in my log filename
set mytime=%time:~0,2%-%time:~3,2%  |  captures the time for use in my log filename
i:  |  my source drive
j:  |  my destination drive
/d  |  copies any new or updated files from source to destination
/s  |  copies all subdirectories and files
/c  |  continues copying through errors (not positive I want to do this, yet)
/f  |  displays full source and destination paths/filenames I capture in the log
/y  |  suppress prompting to confirm an overwrite of an existing destination file
>   |  sends the results of the operations to a log file
f:\z_backup\log\i-j_%mydate%_%mytime%.log  |  path and filename of the log file

This option will only copy new or updates files from source to destination, and does not deletions at all.

Can't use SyncToy or Robocopy (referenced in another thread) since I'm on Win2K and have no desire to change.
« Last Edit: September 27, 2007, 04:10:04 PM by Brian Skalinder »
Milab VM-44 Links > Fostex FR-2LE or
Naiant IPA (tinybox format) >
Roland R-05

Offline it-goes-to-eleven

  • Trade Count: (58)
  • Needs to get out more...
  • *****
  • Posts: 6696
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #4 on: September 27, 2007, 05:12:34 PM »
That's a bummer..   it all comes back to xcopy.. gotta love the 80's! ;)

I like rsync but it isn't windows native.  And of course rsync handles this case and can either delete upon change or not.

It is so hard to keep up with backups and off-siting..

Offline JackoRoses

  • Trade Count: (0)
  • Needs to get out more...
  • *****
  • Posts: 2836
  • Gender: Male
  • lost cause
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #5 on: September 27, 2007, 09:41:51 PM »
I used abakt before but not for media. I just looked and unfortunately the author stopped development but it seemed to work ok for me when I did use it. Which wasn't very extensively but I did see it has command line function as well so a little tweaking and I'm sure it could be ran on a schedule.
it is free also fwiw...
http://www.archive.org/bookmarks/jackoroses
AKG ck61's/ck62's/ck63's/480b's > zaolla's/Dogstar silver cables > optimod V3  > zaolla spdif> HD-P2
"None are more hopelessly enslaved than those who falsely believe they are free. "
Johann Wolfgang von Goethe

Big Brother is here and he is watching you.

Offline davepeck

  • Trade Count: (17)
  • Taperssection All-Star
  • ****
  • Posts: 1106
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #6 on: September 28, 2007, 09:13:50 AM »
That's a bummer..   it all comes back to xcopy.. gotta love the 80's! ;)

I like rsync but it isn't windows native.  And of course rsync handles this case and can either delete upon change or not.

It is so hard to keep up with backups and off-siting..


i'm using a windows build of rsync called DeltaCopy.
Telefunken M60 + TK60 / TK61 / TK62 > Tascam DR-70D

Audio | Video | Photo

Offline Brian Skalinder

  • Complaint Dept.
  • Trade Count: (28)
  • Needs to get out more...
  • *****
  • Posts: 18868
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #7 on: September 28, 2007, 10:24:59 AM »
i'm using a windows build of rsync called DeltaCopy.

Ah, excellent...I'll give DeltaCopy a try.  Thanks!
Milab VM-44 Links > Fostex FR-2LE or
Naiant IPA (tinybox format) >
Roland R-05

Offline Shawn

  • is old and tired
  • Trade Count: (11)
  • Needs to get out more...
  • *****
  • Posts: 3250
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #8 on: September 28, 2007, 10:46:16 AM »
I emailed AB support about this problem and they basically said:  it's not a bug, it's by design, and changing it would be really difficult.  What a lame response.
that is just laughable. sorry about your troubles brian. I'm reconfiguring my media server this weekend. I upgraded to 2 500gb hard drives and I don't want to mess with raid so I'm going to do basically the same thing you've done.

Offline Shawn

  • is old and tired
  • Trade Count: (11)
  • Needs to get out more...
  • *****
  • Posts: 3250
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #9 on: September 28, 2007, 12:53:17 PM »
I'd do the same... but 2 500gig drives wouldn't be enough :P

+T for making me think about this again.
yeah I'm afraid that 500gb is only going to last me about 6 more months. 24 bit recording eats up hard drive space quick especially If I save off unedited 4 channel stuff.

Offline davepeck

  • Trade Count: (17)
  • Taperssection All-Star
  • ****
  • Posts: 1106
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #10 on: September 28, 2007, 12:54:49 PM »
i'm up over 5TB and i'm running out of room!
Telefunken M60 + TK60 / TK61 / TK62 > Tascam DR-70D

Audio | Video | Photo

Offline boojum

  • Trade Count: (1)
  • Needs to get out more...
  • *****
  • Posts: 3629
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #11 on: September 28, 2007, 12:55:01 PM »
Teddy, whom I have not seen on the board for some time now, recommended that if the data is not in three separate places it is not safe.  This is a good, prudent approach, especially if someone is paying you.  I plan on getting a USB 500GB for music, having it on the PC on an internal 500GB and burning a copy to DVD of every stage as it develops.  "I'm sorry, I lost it," does not work with customers.
Nov schmoz kapop.

Offline boojum

  • Trade Count: (1)
  • Needs to get out more...
  • *****
  • Posts: 3629
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #12 on: September 29, 2007, 03:19:36 PM »
^^^ Don't leave us hanging.  What happened?  Are you in a taper protection program??    ;o)
Nov schmoz kapop.

Offline dmonkey

  • Trade Count: (7)
  • Taperssection All-Star
  • ****
  • Posts: 1087
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #13 on: October 01, 2007, 12:16:50 AM »
Another nice program to help automate these processes is called SyncBack. Can be simple, can be complex...very configurable. I use it to backup photos to a RAID server that I run, but it can be used just as easily to mirror a drive to another.
MK4's, KM140's or MC930's >  Tinybox or Aerco MP-2 > R-09, M-10, R-44 (Oade CM) or MixPre-6

Offline spott

  • Trade Count: (0)
  • Taperssection All-Star
  • ****
  • Posts: 1099
  • Gender: Male
Re: Reasonable redundancy without RAID: protecting the music (data) on your HD
« Reply #14 on: October 01, 2007, 07:51:56 PM »
Here's a free virtual Network Attached Storage (NAS) solution that runs on a FreeBSD platform http://www.freenas.org/ - a lot of people are using this to support the SlimServer (Squeezebox) software. I've been trying to find a large volume *secure* storage solution as well. 

There's also a variety of appliance based solutions to choose from; Drobo, ReadyNAS, Windows Home Server (due this month), Buffalo Terastation, etc. - most of these are SATA drive enclosures that have a RAID similar protection (either OS or hardware RAID). 



 

RSS | Mobile
Page created in 0.265 seconds with 44 queries.
© 2002-2024 Taperssection.com
Powered by SMF