Swinging for the fences (the super-close ones)

Chia Plotting
1

For those that have been following my ramblings, my $5k build produces about 22 plots/day.

Not great, BUT I have had an absolute blast learning about all the different configurations/techniques via these forums to try and squeeze more out of the machine. It’s like this entire new realm of computer arch research / failure conditions / stress conditions that we are learning how to optimize our way through together.

That’s pretty cool.

ANYHOO… I was watching YouTube this morning and Sloth Tech was showing how to put together a trash-bin build of 20+ super cheap 10k SAS spinning drives to mine Chia.

The idea got me thinking… I’ve got this 12-bay JBOD sitting here, idling and waiting to hold my plots, but otherwise doing nothing.

On my host plotting machine I have 6x drives for storage - right now I’m mostly plotting this way:

  • NVME1 → 3 Parallel Plots → SATA1
  • NVME2 → 3 Parallel Plots → SATA2

That gives me 22.3 plots/day.

Then it dawns on me, I should be using the drives in the JBOD to do work and not just sit there like expensive idiots.

So I just popped up mdadm and created 2x 6-disk RAID0 arrays.

My plan is to try this out now (remember, an 8-core/16-th CPU):

  • NVME1 → 3 Parallel Plots → SATA1
  • NVME2 → 3 Parallel Plots → SATA2
  • RAID0-1 → 1 Plots → SATA3
  • RAID0-2 → 1 Plots → SATA4

NOTE: I realize this would occupy all 8-cores/16-threads of the CPU which could potentially hamper farming, BUT, I figured it would be fine because the spindle drives will be so much slower than the NVME, that by definition the CPU will have cycles to spare. If I had 4x NVME drives in there, I don’t know that I would try this configuration and potentially starve out the farmer.

I realize that my plot times will probably be 12 hours/plot on the RAID0 arrays on the spinning drives, but that should give me 4-8 more plots/day which would be cool.

Just waiting for a handful of plots to finish before firing this up… maybe it fails instantly, we’ll see!

EDIT: Changed the planned plots, originally had NVME 2x and 2x on SATA, I’m going to go with 3x NVME and 1x on SATA for this first run.

3 Likes
2

Perhaps the master himself @SlothtechTV also wants to share his view :wink:

3

Ok it’s running…

image
image772×841 33.8 KB

4

I didn’t realize he was active here, that would be awesome!

5

This actually makes me happy - I’ve NEVER seen my CPU utilized this heavily before (even trying silly things like 4x parallel plots each with 4 threads or 2x parallel plots with 8 threads)

Also looks like the spindle drives (a+b+c+d+e+f && g+h+i+j+k+l) are just chilling… maybe some overhead capacity there…

image
image944×1191 245 KB

6

30mins in and all the plots are keeping pace with each other… interesting. I think I have some overhead there.

image
image742×756 31.3 KB

7

I don’t understand why you are running it this way. You have all your plots running at the same time. When I say that, I mean you are not staggering them at all. If you would just run Swar’s and stagger those plots, you would get a lot more per day. You are throwing all that at the system at once. So all are in phase 1 at the same time and so on. I think you have pushed your testing as far as it is going to go. It is well past time to step up to a plot manager. You will hate yourself for waiting so long once you see the difference.

That system is capable of a lot more.

8

Setting aside the fact that it seems like youre plotting on all of the drives at once… not sure if you’d get a performance gain from staggering (you likely would) as WolfGT has stated above.

You sir are doing it the same way i am… I have some type of SSD in all of my rigs then i have a handful of 10k drives to fill in whatever is left over … Gross oversimplification: machine can do 40ish plots at a time… 32 plots are occupied by small SAS enterprise ssds and then 4-8 10k drives… i usually plot on them individually (1 plot per drive) and it works great.

The only major issues I’ve ran into were HBA related… ie: don’t use an HBA that has limited performance (IOPS wise) it will kill the performance of 10k plotting – or just limit the total # you can do at once…

But yeah – fill in the leftover cores with 10k drives… it works great-- well done.

5 Likes
9

I’m definitely a master of nothing… The most profitable part of Chia for me personally has been the journey itself and learning from my mistakes :slight_smile:

TY for your support!

3 Likes
10

Staggering you plot start times is important for a couple of reasons. From my observation only phase 1 is multi-threaded. In addition phases 2 and 3 do not seem to take up the full 260 gib. I have been able to run 4 plots to a 1 TB ssd as long as phase 1 and 4 don’t run at the same time on all the plots for a single drive. With a 10850K I have been able to run over 12 plots at the same time using 3 1 tb ssd drives. My cpu stays in the 75% to 100% utilization now as opposed to having waves before when I was using 4 threads per plot.

1 Like
11

What manager are you using to make sure that phases 1 and 4 don’t overlap?

12

No doubt - it’s not that I don’t believe using a plot manager won’t be better, it’s that I’m trying to learn my way into Chia and get baseline readings for the hardware so I know what is min/max/etc.

Once I know what this hardware can do, then dialing in a manager I KNOW I’m doing it right if I’m getting higher counts/day.

FWIW - I installed Swar and started reading through the config weeks ago and NOTHING clicked with me - I just didn’t get it or what it was asking.

That was a sign that I needed to learn my way into this and not just use tools I don’t understand what they are doing. I have old-man brain :slight_smile:

1 Like
13

hey man a lot of us started there when it comes to chia – great philosophy – its the journey after all :slight_smile:

1 Like
14

I am still using the GUI. I just have my start times staggered so even if one starts a little before the other is finished only 2 are at stages 1 and 4 as the other 2 would be in phase 2 and 3. Using a plot manager it could be optimized even more. Depending how many available hhds you have I would try staggering and than offloading the plots to the hhds after phase 1. This will allow you to run 12 plots at a time instead of 6 or 8.

15

I want to understand this in more detail - how would I go about doing that?

Right now I just run the GUI and it does it’s thing moving through the phases… the I/O on Phases 2, 3 and 4 still seem really heavy so moving anything off to a SATA drive before it’s complete seems to me like it would dramatically slow down?

Any links or info to fill me in? Maybe I have the nature of these runs all wrong in my mind.

(100% agree at staggering - I’ve tried before - but the length of execution time becomes completely unpredictable so after it runs long enough - usually around 24hrs - I inevitably have too many at the wrong phase adn the drive fills and they all stop)

16

I know, I keep saying this. (Plot manager!) This observation is the exact example as to why the GUI is not up to the task of running parallel plots. It only takes into consideration what you tell it in the beginning. Once it starts running, the resources of the system dictate what actually happens. And a day or so later, the plot phases sync up because of the resource bottlenecks and boom. Again, a plot manager is the only way to do this right. You are learning the hard way.

1 Like
17

I am an old man too, back in the days I worked with assembly, basic, Delphi.

I never used python before. I used the video of @SlothtechTV together with the @swar GitHub as a lead on how to install.

But as soon as ‘Pip’ came along all went wrong. Finally I found out that in my case I have to put ‘py’ in front of the commands. I am proud I got it to work and learned something new.

2 Likes
18

Ok I promise I will look at getting SWAR running in the next 24hrs - I know this is causing you a lot of anxiety :slight_smile:

That said re-read @nontechguy 's post - he said he’s still using the GUI and then mentioned he’s staggering to avoid phase overlap… which as far as you and I know is pretty much impossible and I’m guessing he’s headed for a frozen plotter here shortly.

I was trying to channel you and tell him the ways of the PLOT MANAGER!

ALL HAIL PLOT MANAGER!

1 Like
19

Not causing me any grief at all. Just trying to help relieve your grief. I was always told “be the benevolent man, not the wise man”. A wise man learns from his mistakes, a benevolent man learns from other people’s mistakes. We have all made the mistake you are making. I’ve just been trying to help by pointing this out early, but my efforts didn’t work because here we are.

Glad you are going to at least try it. I’ll leave you alone.

1 Like
20

Pip also got me hung up first time. The only way I could get it to run was to not skip the “optional” step of creating a virtual python environment. Once that was done everything worked.