Can Chia plots "go bad" over time? Mine seem to!

Might ZFS be a solution? ZRAID2 you lose like 136 plots, but can lose two disks without data loss. Might speed up disk writes a bit too. Supposedly, it’s also good at fixing bit flips based on ecc.

Just ran a check -n 5 on one 8-drive nas with 515 plots… only 4 were bad, each from different days, 3 in April 1 in May.

edit1: Second harvester box, another 8-drive nas… 750 plots. 11 bad plots, 4 from april 28, another from april, 6 from different days in May.

edit2: Third harvester box, a 5-drive nas… 320 plots… 3 bad plots, 1 from may 14, 2 within 1 hour on may 16th.

Total of 18 bad plots deleted out of ~1580.

1 Like

Just plain zfs with default enabled checksum, no raid no recovery. You are filling drives to farm. If a drive fails. Let it fail. When it fails, chances are you can not find a good price drive to replace it. Do not risk the whole array.

1 Like

I’m guessing by that point you have monthly HDD orders going out. I think it’s reccomended to keep disks on had for swapping into ZFS pools. Of course it depends on how quickly you can fill an HDD. I’m assuming once one disk goes a bunch of others will die a few days later.

Ok I had another 2 plots on the farm seemingly go bad… from harvester chia.harvester.harvester

ERROR    Error using prover object Src size is incorrect
ERROR    File: C:\mounts\hd40\plot-k32-2021-05-12-20-10-9ad036d8c1f1a17826e9b13b52816c6764128586a57d72295ee769b0807757b0.plot Plot ID: 9ad036d8c1f1a17826e9b13b52816c6764128586a57d72295ee769b0807757b0, challenge: ef2de7f1780887bbf2fc8a9dc2a20e9851e6d3fe9e63456ad61aca834c23e9e0, plot_info: PlotInfo(prover=<chiapos.DiskProver object at 0x00000165D86E5130>, pool_public_key=<G1Element b59e5c3f507029ac4bbff11e00aac05588effc49d210273080ee17f0fcbdf09c70501ce9aa2686411d00ed3bc8d77814>, pool_contract_puzzle_hash=None, plot_public_key=<G1Element 8d71091218f2c3cb6efbebf341ac098637b21161772b8f4cfb7524027b8cdee65820ed12753fe92e828ad192f1c098b4>, file_size=108823403175, time_modified=1620924290.2937818)

Let me plot check it… yep, the plot check fails on it as well. This is so, so strange. The error is different though; it’s using prover object Src size is incorrect.

I am 100% sure that I scanned this disk and all the plots on it before! Also, I’ve been monitored the logs every few days for any ERROR and WARN entries, and cleaned up any reported bad plots as they happened, a while ago.

All the plots are marked read-only as well. :man_shrugging:

@farmerfm (or anyone else) let me know if you start to experience the same thing.

Hmm…any other software touching them?

Anti virus scanning?

1 Like

Good idea, but no – definitely not. And all the plot files are marked read-only. It is very strange, like bit-rot at scale.

1 Like

Have you tried turning it off and back on again? :joy:

Seriously tho - I’ve noticed the same problem and I’m not kidding. After a reboot, sometimes the plots check as fine again. I think I might have a power supply issue and maybe a disk randomly shuts down for a few seconds cause of lack of power. This seems to cause Windows (or the disk controller) to freak out a bit (even tho the disks are hot swappable). Sometimes I’ll be able to read other files on the disk just fine, but other times not. I’ve found that sometimes the disk seems fine, all files are listed in Explorer, the GUI is happily farming away with no errors. but if I just try to create a new folder on the disk, I’ll get a disk access error.

I’ve found that disks act very weird when they don’t get as much power as they need. I’ve got a 1200 watt server power supply on the way. :sunglasses:

1 Like

Yeah it’s very very suspicious. I mean, I know these plots checked out as good in the past. Absolutely they did. And they’re marked read only, so… :confounded:

1 Like

The only thing that I can think of that would cause that after knowing they are good is some sort of hardware issue causing corruption. I assume you have looked to see if they are on the same drive or some other hardware that is common between them?

1 Like

Did you try removing the read only filesystem to the drives?

No, I have only marked the files read-only. I may look into making the mount read only if these weird “plots going bad” things keep happening…