Is Filling hard drive to maximum 99.9% is a terrible idea

@jonesjr, I catch your meaning. Your insights are over our heads.

To support your position, I found a jpg file that is suitable for your profile picture:

Please wear it, proudly.

1 Like

Whatever you said there, good for you! Me too! But a question first > could u use a Spell Check / Grammar Check? On mine it’s fairly automagic and helps reader comprehension on many layers of abstraction. Heaven knows we need more of that, don’t you agree?

1 Like

What is he trying to say???

The drives are all independent (No RAID, no JBOD that would be a bad idea for plots!), only my IMPORTANT DATA which is stored in the two StableBit Drive pools (go Google it) has redundancy. All the Chia plots are not important data, and therefore does not need backing up or redundancy, why waste space which could be earning Chia???

I have a system in place for automatic off-site backup of my IMPORTANT DATA, so on site duplication, and off-site backup.

I’ve already said what software I use, and putting plots on a JBOD or RAID is just stupidity, there’s absolutely no point - just replot if you lose a drive.

You clearly need to go and learn some more, perhaps come back when you can put senior at the end of your name.

2 Likes

With a bit of risk, you could make the most use of your space with JBOD or RAID 0.
If 5 drives each have 50 GB left unused, then with JBOD or RAID 0, you would have 250 GB available for 2 more K32 plots.

I would only consider doing so if I had something like 80 GB left over on my drives.

And if you lose your RAID, then replotting will take some time, but should not be a big deal.
Of course, it depends on how many drives you put in your RAID. I would keep it small, if I were to consider it at all.

You could also just sprinkle a few K33 or K34 plots, and use less K32 plots, to reduce unused space.

1 Like

Please read more about plotting - 3.2 Proof of Space | Chia Documentation

Especially the third paragraph that talks about plot sizes based on k-values, and fourth paragraph that talks about the number of hashes based on k-value.

My understanding of that doc is that your advice is actually the opposite of how to get the highest proof density per TB.

Thank you for correcting my understanding.

I will read the info from your link a little later.

My brother stitched together that remaining space in Windows, he had issues, and even today has still found some problems related to it.

Best way in my opinion is just to distribute K32 and K33 to fully utilize the space, across my 19 didicated Chia drives I have just 64GB in total - I haven’t used K34 plots on these drives.

1 Like

When you’ve read it, let me know your opinion as I’d be interested.

As some have already mentioned, stop promoting JBOD or RAID for storage of plots which is just a VERY BAD idea as when you loose one HDD you just lost pilots on all HDDs belonging in that JBOD or RAID0. The only exception is if you already have RAID with redundancy for your IMPORTANT DATA and you just want to use some remaining free space to store some plots and planning to delete these plots if you need more space for your IMPORTANT DATA.

Many instructions out there on how to format drives for max capacity on both Win and Linux machines but my 2c is to use NTFS with large cluster size on Win while on Linux use Ext4 with no journaling and large files. Avoid using NTFS on Linux as that adds a significant CPU overhead especially when you are using IoT boards like RPi for your harvesters.

All, my drives are filled to their edges (>430TB farm) with various combinations of K32-K34 plots to maximize filled space and there is absolutely NO reason not to do that. Yes, technically sectors at the edge/end of the HDD plotters will take slightly longer to read than sectors at the beginning due to the mechanics, but you are talking about microseconds differences if that and this may only be impactful with many random reads (ex in case of drive used for OS) of data that may be fragmented between beginning and end of the drive (hence the need to defrag to improve IO speeds). Honestly none of this matters with Chia plots as very likely you will only get 1-2 scans on your 14-18TB HDD each 10s if that as Chia harvester is VERY efficient with the filter and plot header caching. Plus, HDDs used just for plots do not typically suffer from any fragmentation as when you plot you move these plots to the final resting place on the HDD writing sequentially to the disk. Honestly, with all the optimizations in Chia your HDDs are likely to enter sleep state because the HDD is not even being read from most of the time and that is the real performance killer as you have to prevent the sleep which cause harvester to wait >5sec before kernel wakes and spins up the drive.

From my experience HDDs falling a sleep due to no activity is the BIGGEST performance issue which requires attention as you need to ensure your plots are scanned in <5s.

1 Like

coulnt agree more. ridiculous non mission critical data were holding its replicatable.

this softwares just new to me.

sounds like it takes care of everything tho like leaving space on drives themself.
it probably wouldnt even mention that its holding onto a spare 5gb per disk. min,
theres really no need too… its common knowledge. with physical spinning disks.

in every step from the factory to at ur home with your software.
and every step in between leaves a little space… even fresh from the manufacture.

they deliberately build in extra space. space on the disk that is unusable usually.

what im saying is. the manufactrure leaves sdpace on a hdd that cant be used.
has to do with bytes and byte sectores and the way data is acgtually written… its imperfect or somthing…

just look it up

also some disk will even have a hw cache built onto the disk for faster journaling mostly/
but so it
depends greatly on the format u r using or vdevs vs devs.
how many tb do you have that are your disks like what did the package say and how many tb do u have read in software?
ill bet its under
because no software foolish enough to write data offf the edge of a disk. on everything leaves a lil extra space.

but hey we can prove what im say simply with linux
and you can try at home too.

ill setup a lil script tonight to currupt a hd.
forcing write to its maxium capacity.

and show how the drive becomes unreadable… ez

tharts all im tryan to say here.

that if truly filling up a disk… to its maximum… maximum. ur doing uself a diservice and risking your data… no doubt

i wish i was better art explaining… im just new to forums. this my first one ever.

whos promoting raid:? and jbod is a perfectly acceptable solution.

and why would u use a filesystem that u have to modify so heavily like ext4…
taking away journalings like… really dum…

bru just use ext2. hahaha

better yet. xfs… is a pertfectly designed filesystem for plot storage… litereally perfect in every way…

and u journaling doesnt automaticlay mean slower performance as u elude
can actually accelerate read preformance greatly (wich is what we doin on chia)

jbod with every disk individually formatted with xfs. mount fused together with mhddfs is a perfect in every way solution to plot storage.

you sound like you only barely know what your talking about when it comes to a jbod…

LOL, slow down your replies as your typos are hard to read :slight_smile:

This is what I was referring to, not a good idea as I had a 4xHDD JBOD formatted as 1 volume where 1 HDD failed and had to re-plot 4 HDDs. When you have a lot of drives the odds of one of them going bad significantly increase and you are not buying yourself as much space as you think.

Ext2 is obsolete but probably the best part of your comment as it will use less space than Ext4 without any tweaks while I would still reduce inode count to free up space used up by metadata as you will honestly store <200 plots on your 20TB HDD (based on my rough math). Honesty, Ext2 is irrelevant as you can get the same amount of storage when tweaking Ext4 by basically stripping the Ext4 protections, so basically back to Ext2 is obsolete.

XFS is a good file system for partial use but requires tweaking just like Ext# to maximize capacity for plots. Just look at this, it can cost you close to a plot of space if not tweaked.

Journaling helps to prevent data loss during writes and as far as I know does not improve reads. With Chia plots journaling is irrelevant as data is written once and read many times so protecting data during write from accidental power outage is completely irrelevant in this case.

Refer to my JBOD comment above, enough said, you learn by listening or by experiencing :wink: Many beginners do not have large JBOD hardware so they need to know the risk of formatting all disks in JBOD as 1 volume. What you are talking about when formatting each disk in JBOD separately is better but not every JBOD hardware allows that. Some hardware treats JBOD as RAID0.

HDD will work fine even if it’s filled up with 0 free space as long as it is not your OS drive or drive used for any types of writes by OS or any app. Some free space wiping tools actually fill up the drive to the max to cleanup data left in no longer used areas of the HDD. Erase of free space is only relevant to spinner HDDs and not SDD/NVMe, for these lookup trimming.

The main idea with Chia plots is to utilize maximum amount of space to increase your chances of finding a block. Some of my HDDs are left with 1GB of free space and no performance issues AT ALL and I do monitor my Chia plot scans times.

LOL, my experience is decades in computer space so I’ve seen a lot. No need for insults…peace!!!

2 Likes

Pure gibberish, not bothering to read any more.

2 Likes

About halfway thru video. They mention. What happens when u fill Somthing.

There are numerous other videos proving what I’m saying g

Why should anyone sift through a 21:09 length video, searching for whatever you claim is in there?

You wrote “about halfway”. What is “about” to you can mean anything, and we should not have to sit through where we think that you think is “about” halfway.

The video has a built-in clock. It shows the minutes and the seconds.
So instead of writing “about halfway”, you should specify the time mark.

“g”

Why do you type letters in your comments that have no meaning?
6

G stands for gangster. First of all

Second

It’s a video with real knoledge in it that’s real.
Ur right who would wanna watch that :joy::rofl:

It’s at 12:44 in video…. About half way thru like I said.

There’s plenty of others I could show you.

U pretend to care about the quest for a higher lvl of understanding about computers if u don’t just want the whole video

Knowledge is power

You posted a video about filling “Somthing”, and that “Somthing” has nothing, whatsoever, to do with mechanical drives that store plot files.

Granted, the video is computer related, and that is where the similarity ends.
The video is might as well be about using object dependencies in MS Access. That, too, is computer related.

Yes. The internet is full of videos that have nothing to do with storing plot files.
r

Look.

Filling anything to max is a bad idea.
It’s a joke.
People who say otherwise are screwing with you.
Data loss is possible. And preformane tanks. When I run out of room for anything to happen on the drive. The drive won’t work well

U don’t understand anything.
Not my windows reference
Not me plainly saying it
Not even Linus can convince you

Ur a clearly a sheep