Good luck in testing, that’s how we all learn. You may find that most folks have a machine to plot and a machine to farm to make life easier. keep us updated.
1 Like
SAS cards normally work just fine with SATA drives.
2 Likes
It does basically work. The problem I seemed to have is this. When you are plotting to one of the drives while the the Chia Farmer/Harvester is trying to read from them, the write process looks like it is hogging the action. This resulted in failed lookups for the read and hence a pile of error messages in the log. There is a special firmware version to fix this for SATA drives, but I don’t know how to install it. Nothing I tried seemed to get anywhere. I assume that SAS drives work in a different way so that simultaneous read and write operations aren’t an issue.
Initial tests indicate that life is never that simple. Looking at the 8 SATA connectors on the motherboard, four of them are already connected to something. The two ODD ports (I do have a DVD drive) and the first two of the others. I did try connecting a 24TB Exos drive to one of the spare motherboard SATA ports, but it didn’t even show up at all. The computer has 1 x 2TB SSD for the OS, 1 x 1TB SSD used as temp storage for plotting and 2 x 18TB Exos HDD. When I configured it the during the purchase process it would not allow me to add more HDD. The Exos drives at that time were only 4TB. So I assume that these motherboard SATA connectors are not independent, but all part of the internal storage allocation.
Back to the original configuration and created a Bladebit C7 plot in about the same time as the C1 plots of 16 minutes. But this does include the time to write the plot to the final directory which it seems to do during the final stages of the plot creation. Clearly not as fast as your setup, but a lot faster than the 40 minutes it takes me to use the computer CPU to create a k32 plot in RAM.
All my 16 internal drives are SATA, and 14 are connected to a SAS expander, I’ve had no issues with look up times whilst replotting.
I did have a weird Windows issue with having drives both mounted and assigned a drive letter, but that’s been resolved and replotting continues without any further issues.
This may show what I am talking about:
https://www.truenas.com/docs/hardware/notices/componentarticles/lsi9300timeout/
Managed to do the Firmware update on the LSI card and that seems to have made a difference. No error message in the Chia log file so far.
Now for an issue that is a bit worrying.
I created about 100 Bladebit Cuda files using the NVIDIA T1000 graphics card. All the plots look ok, but running the Chia plot checker, 6 of the plots have errors (Typically: Src size is incorrect).
Now I am getting these errors periodically for the other files. Am I to assume that all of these files are useless?
I am farming with the computer CPU and not the graphics card.
Any clues as to what is going on, much appreciated.
I have had bad plots in the past, when running the checker, just kill them and make news ones, is what I did way back.
The plot checker only tests with one challenge and will weed out a certain number of plots. But I am now getting more errors on the Chia log file as more of these plots indicate a problem. Up until now I have always used the traditional madmax plotter to make k32, k33 and k34 uncompressed plots. I have never had any errors like these before.
I still have some more disk space to fill up and will do more tests.
I will see if it makes a difference if I:
1: Run the Bladebit cuda plotter from within the Chia GUI.
2: Run the normal Bladebit RAM plotter to see if this is a cuda issue.
All of these plots I am creating are uncompressed (C0).
I am still using a nvidia T1000 GPU with the Dell Workstation and Windows 11.
Is is possible that the cuda plot creation is getting screwed up because this graphics card also drives the computer monitor?
The BB cuda plotter did have a habit of producing duff plots, so much so that they built in checking, and it would discard plots that scored less than the set threshold.
Why are you plotting c0 though, pretty much any cpu could cope with c1?
My advice, use Gigahorse, it’s much more reliable.
I am just testing to see what is possible with my hardware setup. At the moment nobody knows what is going to happen with the upcoming hard fork and how it is going to be implemented. Will the concept of k32, k33, k34 etc. plots still exist or will there just be one new generic plot type with various compression levels. When the plot filter reduces, how much load impact will that have on the GPU or CPU. I am going to max out at about 1.3PiB and it doesn’t really bother me whether or not I am compressed or uncompressed. My current ETW is 6 days, but for the last few months I have been doing way better than that. About 75% of my space is k34, so you ask yourself is that significant. Is 1 k34 better than 4 x k32. I still have 4 drive slots left with the surrounding infrastructure. I could fill them with 4 new Seagate exos 24TB drives. That would cost me here in the UK about £1700. From a purely financial point of view totally a not starter if you look at it from a ROI point of view.
I’m also in the UK. From the sounds of it the plot format will change, and any plots we have now, whether compressed or not, k32 or k34 will all need to be replaced.
It’s very easy to test what your hardware will support after the filter change, with GH you use the ProofofSpace program, BB has its own version as well.
I’ve no idea in practice if a k34 is better than 4 x k32, but it’s been debated before and I think statistically it shouldn’t make a difference.
However I have 323TiB of storage, with c0 that’s still 323 TiB, I’ve replotted to c30 and now have 755 TiB effective, that’s 2.34 times the earnings of c0, I also pool so less of a luck factor involved. Yes there is a small electric cost, fee’s with GH and capital expense as I bought the GPU last year when I replotted to BB c7
You mention £1700 to buy 4 drives, now cost a GPU and how much capacity that would give you instead. I added 432 TiB for the cost of a 3060, which I bought new for about £300 last June, significantly cheaper than buying the equivalent HDD space.
Interesting idea to check out a 3060. My computer motherboard is PCI 3, these cards are PCI 4 so I assume there would be a loss of performance. If I can work out if it is actually compatible with my Dell Workstation, I might try one. When I google NVIDIA 3060 there seems to be a lot of slight variations. Are they all suitable for Chia plotting or is there something specific I should look out for? I am new to all this graphic card stuff so my knowledge is limited.
On your note about pooling. I was with Spacepool the day it started. It was certainly beneficial up to the point when my storage size got to 500TB. After that it was better to be self pooling and I left when I reached about 700TB.
My PCIe slots are 3.0 and I’m making 2.8min plots. Back when this started I was making 45 min plots… Start the plots and go make coffee and go shopping.
I use an old Dell T7910 for plotting, with a 3080 and a Tesla P4, works a treat. PCIe 3 is not a problem, unless aiming for very quick plotting, and it doesn’t affect farming at all.
Obviously plotting is a consideration, but you need to be looking at what you can farm, and at what C level, a 3060 will just about cope with my 323 TiB physical at c30 after the filter change.
For a 1.3 PiB farm you’ll need either a far more powerful card, or a much lower c size. 3090 would do 1.3 PiB at c29 and give you 2.11 times the farm size, but then is the increased expense and running costs worth it???
See the chart here for an idea, halve the size to take into account the filter change.
And spreadsheet here for GH 3
And GH 2.5
1 Like
A lot of useful information to digest, thank you both. I will continue to test and experiment.
1 Like
That’s going to be true for any drive you plot to, other than SSDs.
I see the same thing on the SAS drives I have.
Latest update.
Now got a NVIDIA RTX 3080Ti 12 GB. Obviously better than the T1000 8GB, so doing more testing with this.
However I now have another problem which may be more of an issue.
My ETW hovers around 5/6 days with about 1.3PiB of plots. Up to the 9th April I was winning blocks at about that rate of slightly better. Since that date I have not had a single win. On the 10th April, Windows 11 did one of those monthly cumulative updates. The Chia GUI was closed down during this process.
Was this significant, who knows.
I have resync’d the main database from a version had I saved in January 2024 just in case that had got corrupted.
I realize that statistics may be playing a part here and that I have just been unlucky, but something in my bones tells me that I may have a problem.
These are some of the warning messages I have been seeing in my Chia debug log file. Any clues?
2024-04-28T23:31:19.088 full_node chia.full_node.full_node: WARNING Block validation: 2.69s, pre_validation: 2.61s, CLVM: 2.53s, post-process: 0.02s, cost: 446976190, percent full: 4.063% header_hash: e3c66216403d3edac7613af022b36541e301edefd91e7035a462618bf4367719 height: 5286146
2024-04-28T23:58:08.532 full_node chia.full_node.full_node: WARNING Block validation: 2.25s, pre_validation: 2.08s, CLVM: 2.02s, post-process: 0.06s, cost: 877019851, percent full: 7.973% header_hash: 5c44c04c87e66f6bdc2ca1fa17e572f35476d86792ddcf9e63e9bc4025b1de8c height: 5286246
2024-04-28T23:58:11.797 full_node chia.consensus.blockchain: WARNING slow path in block validation. Building coin set for fork (5286245, 5286246) 2024-04-28T23:58:11.812 full_node chia.full_node.full_node: WARNING Block validation: 2.22s, pre_validation: 2.20s, CLVM: 2.00s, post-process: 0.00s, cost: 877019851, percent full: 7.973% header_hash: 90cb60a9be6532b72f1d029bb63734e7450c94fe50beceb9f93adab587a326ee height: 5286246
2024-04-30T19:15:11.173 full_node chia.consensus.blockchain: WARNING slow path in block validation. Building coin set for fork (5294575, 5294576)
2024-04-30T19:15:23.373 full_node chia.consensus.blockchain: WARNING slow path in block validation. Building coin set for fork (5294575, 5294577)
2024-04-30T19:15:23.389 full_node chia.full_node.mempool_manager: WARNING updating the mempool using the slow-path. peak: 10833d86d4cfac42b45c77039fa85a0e3f9d065b614f5717db577e0449f936ee new-peak-prev: 8c3e4df4a009cda63fdff704ef2b26362e4e43e1f1089b73910da624197b44ae coins: set
Just updated Chia client to V2.3.0, so we’ll see if that makes any difference.
Are you in any pool??