XYMer's Home away from Home

When http://bbs.xlr8yourmac.com is down (i.e. always)
Privacy Policy
It is currently Sun Oct 21, 2018 8:16 pm

All times are UTC - 8 hours




Post new topic Reply to topic  [ 14 posts ] 
Author Message
PostPosted: Sun May 06, 2018 9:49 pm 
Offline

Joined: Thu May 15, 2008 8:05 pm
Posts: 399
My personal Mac mini server had been running on 10.11.6 for several years without a hitch but I began to have instability that appeared to be caused by the walking wounded failure of the Crucial BX100 SSD I had in there. I have been doing Time Machine backups to one of these attached with Thunderbolt and configured in a RAID10 with 6TB (3TB x 4) of storage. http://www.datoptic.com/ec/1u-rackount- ... rbolt.html

I also do Time Machine backups on all my machines to a Time Capsule.

I'm also running PaperCut NG, MySQL, a million and one NetBoot images, a TFTP server, Slink Agent and a few other things.

So I bought a new SSD (a Samsung Evo 850 for all the difference it makes) and figured I might as well install High Sierra (10.13.4) since I was starting from scratch.

However I am finding that backing up to the Time Machine destinations as configured in High Sierra that the backups are becoming corrupt within a week to ten days and at that point any subsequent backup fails. The backup of my desktop Mac mini is approximately 700GB and it seems to be the most susceptible to corruption, but my iMac at 150GB has also exhibited the behavior.

Has anyone else experienced this?


Top
 Profile  
 
PostPosted: Mon May 07, 2018 3:11 am 
Offline
User avatar

Joined: Thu May 15, 2008 8:13 pm
Posts: 10135
Location: Caught between the moon and NYC
In my experience the minute a Time Machine backup volume starts getting corrupt it will subsequently keep getting corrupt despite all attempts to repair it. The only way to stop it from getting corrupt is to format & start over fresh. Or replace it with a new, fresh backup volume, if you want to maintain the old backups.


Top
 Profile  
 
PostPosted: Wed May 09, 2018 1:04 pm 
Offline

Joined: Thu May 15, 2008 8:05 pm
Posts: 399
The best next step I have come up with is to blow away the hardware RAID on the Thunderbolt appliance and then put it in JBOD mode and then rebuilt the RAID in SoftRAID.


Top
 Profile  
 
PostPosted: Sun May 13, 2018 9:25 pm 
Offline
User avatar

Joined: Thu May 15, 2008 8:07 pm
Posts: 2569
Location: Inside Flatus Maximus
Little known secret: There are no fully Time Machine compatible devices outside of the Time Capsule line. None. Sure, there are plenty that claim compatibility, but don't actually meet the specifications for Time Machine. Namely, they all use an incomplete version of netatalk for their AFP implementation. This is why WD MyBook/MyCloud devices fail so frequently on TM backup/restores. I have no doubts that the RAID part of the equation is throwing a few monkey wrenches into the mix as well. Backups usually aren't done to striped raids, even if they're in an 10 configuration. 50, yes. 10, not so much.

If your thunderbolt device supports RAID 5 or 6, I'd use that mode instead. With RAID 10 you have zero parity protection. You do with RAID 5 at least.

_________________
Official Mac Tech Support Forum Cookie™ (Mint Chocolate Chip)
Guaranteed tasty; Potentially volatile when dipped in WWIII Forum Syrup®
Caution: This cookie bites back.


Top
 Profile  
 
PostPosted: Mon May 14, 2018 9:19 am 
Offline
User avatar

Joined: Thu May 15, 2008 8:13 pm
Posts: 10135
Location: Caught between the moon and NYC
Depends on whether it implements RAID 10 as a pair of RAID 1 volumes and RAID 0's across them or if it implements a pair of RAID 0 volumes and RAID 1s across them.


Top
 Profile  
 
PostPosted: Mon May 14, 2018 10:39 am 
Offline
Master

Joined: Sun Apr 20, 2008 5:24 am
Posts: 9906
Location: North of the State of Jefferson
You have redundancy in each case: with 01 you have a mirrored set of RAID 0 arrays. A drive can fail in either of the mirrored arrays and you won't lose data, although you will lose that sub-array (which will be rebuilt from the spare). If a second drive fails in the already broken sub-array, it does no more damage. If a second drive fails in the remaining sub-array the entire array is destroyed.

In 10 you have a striped set of mirrored arrays. One drive can fail in each of the sub-arrays before you lose data, but losing two drives in a single sub-array will destroy the entire array.

With a larger number of drives and varying numbers of hot spares the reliability calculations between the two get more interesting, but with four drives you can safely lose any one, and might-or-might not survive losing a second. RAID 10 is probably a better choice than 01 on average because it's usually faster, and I suspect rebuilds could be more efficient.

RAID 5 is usually slower, but you only lose one drive to parity. On the other hand, exactly one drive can fail at any one time. The next one kills the array. Rebuilding the array is also more intensive, so you're possibly increasing the odds of a second drive failing by stressing all the remaining disks. RAID 6 is even slower for writes, but you get to lose any two drives. It usually requires an expensive RAID card. With four drives RAID 10 is a reasonable choice on several levels if you're OK with losing some space. For backups, the extra speed probably doesn't matter.

But anyway back to the topic at hand...I've had very good luck with directly connected local Time Machine backups to reliable hardware. I haven't had reliable remote Time Machine operation in the past, never figured out why it corrupted itself every few weeks, and finally gave up and went back to rsync + MySQL replication + nightly DB dumps. I wrote an rsync script to handle incremental backups so it worked pretty much like Time Machine, without the friendliness, routine corruption, while being much slower. That was several years ago.

Too bad Time Machine has kinda sucked the life out of the remote backup market on the Mac. In Ye Olden Days there was Retrospect, which sucked donkey ba...well, you know, and now there's...still Retrospect which still sucks?? :roll:

- Anonymous


Top
 Profile  
 
PostPosted: Mon May 14, 2018 2:33 pm 
Offline
User avatar

Joined: Thu May 15, 2008 8:13 pm
Posts: 10135
Location: Caught between the moon and NYC
Back in System 7/8/9 days I kind of liked Retrospect. I may have been lucky and ended up on versions that didn't introduce new features and instead just fixed the existing features (you know, like 10.6 did for OS X). I certainly didn't upgrade on every release.

I've found RAID 10 implemented by some not-quite-mainstream vendors to actually be 0+1 and not 10. I don't think anyone at the company above the engineering level understands the difference.


Top
 Profile  
 
PostPosted: Mon May 14, 2018 4:43 pm 
Offline
Master

Joined: Sun Apr 20, 2008 5:24 am
Posts: 9906
Location: North of the State of Jefferson
Retrospect: giving you enough rope to hang yourself since System 7, but will take 18 hours to scan your incremental backup for restore so you'll definitely be ready to hang yourself. Also, the rope is kind of weak and moldy. Or don't bother. It lost contact with the backup server and hasn't backed up in 38 days, and the only way to make it regain contact is to reboot every computer in the backup group. Why did it fail? Unclear. Why doesn't rebooting a single computer (or God forbid, clicking something like "find backup server") help? Unclear. Why does rebooting every computer help? Unclear. Oh goody, a random assertion error! Let's rebuild the catalog. Two weeks later: hooray it's fixed! Next day: oh %*&# me. I'll just start a new backup set, throw out everything, and start over...all I need to do is figure out how to configure it...five hours later: please give me that rope! Oh goody, now I'm getting another random numeric error. I'll just click the client's "backup now" button. Oh, that just doesn't do anything. Why do they even put it there!? Just to taunt you? WHY? I'll just upgrade. Great, I can't import old backup history.

Retrospect was OK in the System 7 days because it was much simpler (you had some vague hope of figuring out how to configure it to do what you wanted, and the documentation bore some crude resemblance to the software), Macs were much less finicky, had many fewer files, and we knew nothing better. Since then it's burned a lot of bridges with me, usually while I was trying to cross them above cold raging floodwaters.

- Anonymous


Top
 Profile  
 
PostPosted: Mon May 14, 2018 8:43 pm 
Offline
User avatar

Joined: Thu May 15, 2008 8:13 pm
Posts: 10135
Location: Caught between the moon and NYC
To be fair, I never really did network backups with Retrospect, it was always local backups to local media. Worked with a couple tape libraries and a string of individual tape drives but never really gave me much pain. I kind of remember setting up a two system backup with Retrospect once and don't have anything bad to say about that setup, but it's all around the same era of OSes.

Now if you want horror stories I can start talking about ARCserve and Backup Exec.


Top
 Profile  
 
PostPosted: Sat May 26, 2018 11:40 pm 
Offline

Joined: Thu May 15, 2008 8:05 pm
Posts: 399
Sorry, I have been utterly swamped at work for months.

I finally had some time this weekend to look at this in some detail and after breaking up the hardware RAID10 and installing SoftRAID I found one of the drives was failing the SoftRAID verification but did not generate a SMART error or even fail the Seagate diagnostic on Windows (guess the model, I dare you).


Top
 Profile  
 
PostPosted: Sun May 27, 2018 2:23 am 
Offline
User avatar

Joined: Thu May 15, 2008 8:13 pm
Posts: 10135
Location: Caught between the moon and NYC
I've never met a 3TB drive that didn't fail within a couple years. Seagate, WD, whatever, they all died young and of course nobody kept a backup. :badteeth:


Top
 Profile  
 
PostPosted: Sun May 27, 2018 5:19 pm 
Offline

Joined: Thu May 15, 2008 8:05 pm
Posts: 399
We have some Toshiba 3TB drives at work in some Sonnet storage that have been trouble-free (I think we have six of them and no failures after ~3 years or so) but the Seagates have just been shit.


Top
 Profile  
 
PostPosted: Tue May 29, 2018 10:28 pm 
Offline
User avatar

Joined: Thu May 15, 2008 8:07 pm
Posts: 2569
Location: Inside Flatus Maximus
MonkeyBoy wrote:
I've never met a 3TB drive that didn't fail within a couple years. Seagate, WD, whatever, they all died young and of course nobody kept a backup. :badteeth:


My Seagate 3TB data drive works fine nearly four years after I got it. But I also don't power it up and down all the time either, which is the primary factor in failing drives. I have a spare and I'm very likely to clone the drive to the spare or just get an 8 TB drive instead so I can stick all my DVDs onto it at once. If I ever get enough money, I'll invest in a 12 TB drive as storage for my disc collections and one as my data drive.

I suspect my drive logevity has to do with my killing shitlight so it never gets a chance to index and having as few power/load cycles as possible.

_________________
Official Mac Tech Support Forum Cookie™ (Mint Chocolate Chip)
Guaranteed tasty; Potentially volatile when dipped in WWIII Forum Syrup®
Caution: This cookie bites back.


Top
 Profile  
 
PostPosted: Wed May 30, 2018 3:05 pm 
Offline
User avatar

Joined: Thu May 15, 2008 8:13 pm
Posts: 10135
Location: Caught between the moon and NYC
Well my pair of 3TB disks were originally used for my Mac's data storage (I stick everything that I care about on it) and a normally turned off backup drive. The drive was literally never turned off and I have a little process setup to touch a file on the disk every minute or two, to stop the case from putting the drive to sleep. The backup drive is normally left off and I periodically turn it on to clone the changes over to it from the data drive. Data drive failed first. Sent it in for warranty with Seagate and bought a WD 3TB drive to replace it. Put the warranty replacement drive in my PC for backups. I do put to that guy to sleep about once a day. Next to go was the backup drive. I didn't even bother with the warranty, just bought a couple 4TB disks to serve as my data/backup drives. WD went over to my Mini for backup and data storage. I don't really use the Mini much but it is running 24/7, and in theory Time Machine should have trickled data to it often enough to stop it from sleeping. Next to go was the drive in my PC, which was now out of warranty. At about the same time I noticed the drive on my Mini had stopped showing up. I coaxed it back to life but then it disappeared again. Tried different cases tried different interfaces different systems different operating systems, still mostly dead with occasional signs of life. By this point I was fed up with 3TB disks and bought some 4TB disks on sale to replace them.

Now this is just me. At work I've lost count of how many 3TB disks have died, either in cases that other departments bought, that employees bought on their own, that students bought on their own, that I bought and was using for storage when 3TBs were still the only option available without dropping down to <7200rpm but before the death rates were well known. This is where I've really seen every manufacturer under the sun. I'm not saying 3TB Seagates are stellar, they're definitely the worst of the bunch, but I've seen drives from everyone else. Now the ones that weren't in my hands may have very well died from misuse. Students and staff aren't known for being particularly graceful and seem to take great delight in torturing devices by knocking them over while powered on, yanking them off tables, etc. but the non-3TB disks have been surviving their excesses, at least for now.

I really don't want to jinx myself but I have a good hundred 1TB Seagates of HD103 vintage up to current rolled out to roughly the same people and in 5 years I've only had a single one fail. I'm waiting with trepidation for the first 4TB drive to fail.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group