New PC with lots of storage
Discussion
Thought this might be interesting to some of you guys.
While I have Netflix and other subscriptions for some TV shows etc, I do like to keep my film collection "owned" - so I am "that guy" who still buys blu-rays. Handy for when we're away and have no wifi - I can take a portable drive with some films on. The quality is better and I'm generally not at the beck and call of the subscription services.
Anyway, this started to present a bit of a challenge with storage requirements - so I recently built a new PC to be both my "normal" PC, but also have enough storage for all the films & TV Shows on it, along with some work backups.
Some challenges I had were:
Need it to be resilient to losing a disk or 2 - the odd bad block or that type of thing
Portable - so if the whole PC dies - I can plug the disks in somewhere else and still have the library. I’ve tried NAS drives before and they were so so. I know they’re better these days, but when I can just add drives to a PC and have it configured how I want. Even better.
Anyway, came up with a decent spec in the end:
MSI Z690 motherboard
4x 20TB disks, connected to the motherboard
4x 8TB disks (older ones), connected to a 9260-8i controller - configured with RAID 0, but with only 1 physical disk in each logical volume.
Then running drivepool with 3x way duplication of all the files - so if I lose a disk I still have 2 other copies of the file. This is perhaps overkill and I will reduce once it’s all backed up to the cloud.
So finally, then running backblaze cloud backup - unlimited storage. Will take a while to get it all backed up offsite, but it’ll get there.
While I have Netflix and other subscriptions for some TV shows etc, I do like to keep my film collection "owned" - so I am "that guy" who still buys blu-rays. Handy for when we're away and have no wifi - I can take a portable drive with some films on. The quality is better and I'm generally not at the beck and call of the subscription services.
Anyway, this started to present a bit of a challenge with storage requirements - so I recently built a new PC to be both my "normal" PC, but also have enough storage for all the films & TV Shows on it, along with some work backups.
Some challenges I had were:
Need it to be resilient to losing a disk or 2 - the odd bad block or that type of thing
Portable - so if the whole PC dies - I can plug the disks in somewhere else and still have the library. I’ve tried NAS drives before and they were so so. I know they’re better these days, but when I can just add drives to a PC and have it configured how I want. Even better.
Anyway, came up with a decent spec in the end:
MSI Z690 motherboard
4x 20TB disks, connected to the motherboard
4x 8TB disks (older ones), connected to a 9260-8i controller - configured with RAID 0, but with only 1 physical disk in each logical volume.
Then running drivepool with 3x way duplication of all the files - so if I lose a disk I still have 2 other copies of the file. This is perhaps overkill and I will reduce once it’s all backed up to the cloud.
So finally, then running backblaze cloud backup - unlimited storage. Will take a while to get it all backed up offsite, but it’ll get there.
Griffith4ever said:
I love 8 spinning platters and then..... A "be quiet" CPU fan :---)
Looks nice and tidy.
There is a bit of noise when those drives are busy - but as it's mostly media on them, I am generally not in the room, I am elsewhere in the house when we are watching a film. Anyway the case is very good at noise attenuationLooks nice and tidy.
On the old build (which I used the same case model for), the loudest thing was the CPU fan - hence the upgrade. It has an extra fan in this build too - so temperatures are lower on this one, which is good. There still a couple of cables that annoy me, but it gets to the point where it's behind the panel and will never be seen! Give it 12 months and it'll all dusty and crap anyway.
Be careful with backblaze. Unless it's changed, they delete everything if the drive goes off line for a period.
I found this out the hard way, disc failed and they started deleting by backup.
I'd have thought a backup should keep the data until instructed to remove it.
I now have a proper cloud backup that does this.
I found this out the hard way, disc failed and they started deleting by backup.
I'd have thought a backup should keep the data until instructed to remove it.
I now have a proper cloud backup that does this.
sociopath said:
Be careful with backblaze. Unless it's changed, they delete everything if the drive goes off line for a period.
I found this out the hard way, disc failed and they started deleting by backup.
I'd have thought a backup should keep the data until instructed to remove it.
I now have a proper cloud backup that does this.
Yes, I am mindful of this. I'm currently on the cheap 30 day policy - but if it's good and I like it, I'll upgrade to the 12 month policy.I found this out the hard way, disc failed and they started deleting by backup.
I'd have thought a backup should keep the data until instructed to remove it.
I now have a proper cloud backup that does this.
Totally get your point - but at 30TB of data - things such as AWS S3 start to be quite expensive for personal use.
So I felt backblaze was an adequate compromise. TBH, I've not had it backed up for the last decade - but then in that decade, I have had to re-rip the library, when it was smaller, twice.
Once when my NAS with Raid 5 died. The disks didn't die - the OS did! But I couldn't get the data back.
Once when I did have an actualy drive die in my old PC.
I've had the odd "close call" on the Raid 5 I used to run as well - but in those cases, it always rebuilt OK, albeit took an age.
So this build aims to mitigate all those risks.
tafkattn said:
That's what I would have thought too. Why does it need to be a full blown machine and not just a NAS?
Having said that, NAS enclosures do get expensive once you're above 4 drives and I guess throughput could be an issue if you have a number of clients.
I needed a new full blown machine - and I just designed it to also have 8 big disks in as well.Having said that, NAS enclosures do get expensive once you're above 4 drives and I guess throughput could be an issue if you have a number of clients.
Did consider the seperate NAS thing, but why have 2 boxes when you can have one.
The old WD NAS I had before was also a nightmare, although I believe NASs are better these days, but just couldn't see any benefit to having one when it was easy to build a single machine that meets all requirements. Could have even self built a NAS using an old motherboard and case. But still 2 boxes then, not 1...
I like the fact that on all 8 of these drives, if this motherboard died today, I could just plug the drives in to a different machine, windows or linux, and all my data is still there. Not striped and impossible to re-assemble withouth a phd. And if I totally lose a drive, I have other copies of the files on different drives - easy to reassemble, either inside or outside of drivepool.
Absolutely. The issue with Raid 5 that I had "last time" was that a single bad block on a disk caused the array to go into rebuilld mode which would take up to 26 hours - it wouldn't just patch the bad block, it was full rebuild. And then if we had a single bad block on another disk in the array that was detected during rebuilt - that destroyed the array. I did consider RAID 6 which gives me 2 disk redundancy - but still the same risk, just reduced. This was running the Intel RXT raid setup. Never Again.
Also looked at (and have) a proper hardware raid controller - an LSI Megaraid 9269-8i jobbie. And in that case, it does have a bit more flexibility - the raid config is more portable. However, it still has the same bad block challenge.
The drivepool setup - if I get a bad block that corrupts a file, the likelihood of a block going bad on another disk at the same time, on a copy of that file is very very small. The biggest risk now is me going Ctrl-A => Delete. Or Format D:. I think.
If I didn't want/need a full size PC and just had a laptop for normal use - then building this using FreeNAS or similar would have been the way. Might have still ended up with similar hardware, although lower spec CPU, Memory etc.
But the fact I'm fine with full size PC makes it easy to just integrate all together.
Also looked at (and have) a proper hardware raid controller - an LSI Megaraid 9269-8i jobbie. And in that case, it does have a bit more flexibility - the raid config is more portable. However, it still has the same bad block challenge.
The drivepool setup - if I get a bad block that corrupts a file, the likelihood of a block going bad on another disk at the same time, on a copy of that file is very very small. The biggest risk now is me going Ctrl-A => Delete. Or Format D:. I think.
If I didn't want/need a full size PC and just had a laptop for normal use - then building this using FreeNAS or similar would have been the way. Might have still ended up with similar hardware, although lower spec CPU, Memory etc.
But the fact I'm fine with full size PC makes it easy to just integrate all together.
Edited by Arnold Cunningham on Thursday 4th August 10:42
Yes. Hard to find now - had to get this one in from Germany.
Is my 2nd - still have the old one too. They are well designed and well made. I don't like the newer designs they have so much - don't come with all the trays and was a struggle to find stock of the extras I would have needed.
Is my 2nd - still have the old one too. They are well designed and well made. I don't like the newer designs they have so much - don't come with all the trays and was a struggle to find stock of the extras I would have needed.
Arnold Cunningham said:
tafkattn said:
That's what I would have thought too. Why does it need to be a full blown machine and not just a NAS?
Having said that, NAS enclosures do get expensive once you're above 4 drives and I guess throughput could be an issue if you have a number of clients.
I needed a new full blown machine - and I just designed it to also have 8 big disks in as well.Having said that, NAS enclosures do get expensive once you're above 4 drives and I guess throughput could be an issue if you have a number of clients.
Did consider the seperate NAS thing, but why have 2 boxes when you can have one.
Drive Blind said:
Good to see a Fractal Design case being properly populated,, Define R5 ?
I love the Fractal cases. My Define XL holds 14 HDDs, an nvme and two SSDs on a PCI bracket. Plenty of room and lots of cooling options make it easy to work in and everything stays nice and cool despite so much in there. I've also never cut myself on a Fractal case which is a bonus xeny said:
Seems odd to go high end on CPU and GPU and then single DIMM?
Actually - they're not. They're all the bottom end of current generation stuff - does what I need it to for now, didn't blow the bank, but does provide upgrade options if required. The DIMM is a single 32 - so easy to plug more in if required.ecs0set said:
I have an ITX machine running VMWare as a media/general server. Having an Intel iGPU means QuickSync transcoding for Plex Server, plus it has the horse power to also run VMs for a desktop OS (so no need for WOL over Internet), an mDNS router, a next-gen software firewall and Active Directory. I'm sure some NAS boxes can be specced to do the same but a standard PC is a viable option.
Professionally, I use linux, so I'm quite happy with putty. I intentionally stay old school - never know what I will find working with a customer so make sure I can still use the basics/essentials.I think I had a look at Plex years ago. However - this platform is part of my Kodi setup - been using Kodi all the way back to the 1st generation xboxes they build it for. So I have a 100" screen, D-ILA projector, and Odroid N2+ on a 1Gbps lan all hooked up. With a decent amplifier and OK speakers as well. It works well and when I crank it up, you can feel, as well as hear, the films (explosions etc).
That reminds me - have started switching to ripping 4K films (from my own blu rays that I buy - I don't pirate) now in anticipation of upgrading the projector. All the other kit already supports it in theory, but I still get a bit of buffering. Must investigate if I've left a Cat 5 cable somewhere in it all.
Edited by Arnold Cunningham on Thursday 4th August 17:04
Edited by Arnold Cunningham on Thursday 4th August 17:04
Gassing Station | Computers, Gadgets & Stuff | Top of Page | What's New | My Stuff