Plotter buffer memory usage

Chia Plotting
1

Hello,
I have been doing some testing with threads and buffer memory on a large server.
Ubuntu 20.04
Chia 1.1.5
72 threads, 137GB available memory.

I am curious to know that what phase EXACTLY does a plotter utilizes the allocated buffer fully?
I know multiple threads are ONLY used in Phase 1!
I set 7 plots with 10 minutes stagger, 6 threads and 10GB memory.
It has yet to hit the 10GB limit in utilization.
Below is the htop output and the plotman status.

 plot id    k                           tmp                                  dst   wall   phase    tmp    pid   stat     mem   user    sys     io
6043e055   32   /home/ubuntu/chia/plot-temp   /home/ubuntu/chia/farm-vol2/farm-2   3:30     3:1   234G   6334    RUN   12.0G   5:13   0:21   0:08
f2db36b6   32   /home/ubuntu/chia/plot-temp   /home/ubuntu/chia/farm-vol2/farm-2   3:40     3:1   220G   5861    RUN   11.4G   5:23   0:21   0:08
f7729257   32   /home/ubuntu/chia/plot-temp   /home/ubuntu/chia/farm-vol2/farm-2   3:50     3:2   204G   5387    RUN   11.4G   5:32   0:21   0:09
87f4657f   32   /home/ubuntu/chia/plot-temp   /home/ubuntu/chia/farm-vol2/farm-2   4:00     3:2   203G   4879    RUN   11.4G   5:42   0:22   0:08
94c2a3be   32   /home/ubuntu/chia/plot-temp   /home/ubuntu/chia/farm-vol2/farm-2   4:10     3:2   187G   4386    RUN   11.4G   5:50   0:23   0:09
fc5297c5   32   /home/ubuntu/chia/plot-temp   /home/ubuntu/chia/farm-vol2/farm-2   4:20     3:2   183G   3890    RUN   11.4G   5:59   0:25   0:08
e81b3fe6   32   /home/ubuntu/chia/plot-temp   /home/ubuntu/chia/farm-vol2/farm-2   4:30     3:3   166G   3243    RUN   11.4G   6:07   0:24   0:09


Mem[13.2G/137G]   Load average: 7.61 7.64 7.66 
  Swp[51.5M/16.0G]   Uptime: 05:33:14

    PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
   4879 ubuntu     20   0 10.6G 1932M  7376 R 100.  1.4  6h01:50 /home/ubuntu/chia-blockchain/venv/bin/python /home/ubuntu/chia-blockchain/venv/bin/chia plots create -k 32 -b 10240 -n 1 -r 8 -t /home/ubuntu/chia/plot-temp -2 /home/ubuntu/chia/plot-temp2/ -d /home/ubuntu
   4386 ubuntu     20   0 10.6G 1937M  5680 R 100.  1.4  6h10:29 /home/ubuntu/chia-blockchain/venv/bin/python /home/ubuntu/chia-blockchain/venv/bin/chia plots create -k 32 -b 10240 -n 1 -r 8 -t /home/ubuntu/chia/plot-temp -2 /home/ubuntu/chia/plot-temp2/ -d /home/ubuntu
   5387 ubuntu     20   0 10.6G 1937M  7420 R 99.5  1.4  5h50:42 /home/ubuntu/chia-blockchain/venv/bin/python /home/ubuntu/chia-blockchain/venv/bin/chia plots create -k 32 -b 10240 -n 1 -r 8 -t /home/ubuntu/chia/plot-temp -2 /home/ubuntu/chia/plot-temp2/ -d /home/ubuntu
   3890 ubuntu     20   0 10.7G 1702M  5720 R 99.5  1.2  6h21:26 /home/ubuntu/chia-blockchain/venv/bin/python /home/ubuntu/chia-blockchain/venv/bin/chia plots create -k 32 -b 10240 -n 1 -r 8 -t /home/ubuntu/chia/plot-temp -2 /home/ubuntu/chia/plot-temp2/ -d /home/ubuntu
   3243 ubuntu     20   0 10.6G 1937M  5756 R 99.5  1.4  6h28:56 /home/ubuntu/chia-blockchain/venv/bin/python /home/ubuntu/chia-blockchain/venv/bin/chia plots create -k 32 -b 10240 -n 1 -r 8 -t /home/ubuntu/chia/plot-temp -2 /home/ubuntu/chia/plot-temp2/ -d /home/ubuntu
   6334 ubuntu     20   0 1837M 1223M  7560 R 94.2  0.9  5h31:43 /home/ubuntu/chia-blockchain/venv/bin/python /home/ubuntu/chia-blockchain/venv/bin/chia plots create -k 32 -b 10240 -n 1 -r 8 -t /home/ubuntu/chia/plot-temp -2 /home/ubuntu/chia/plot-temp2/ -d /home/ubuntu
   5861 ubuntu     20   0 11.2G 1985M  7536 R 92.2  1.4  5h41:47 /home/ubuntu/chia-blockchain/venv/bin/python /home/ubuntu/chia-blockchain/venv/bin/chia plots create -k 32 -b 10240 -n 1 -r 8 -t /home/ubuntu/chia/plot-temp -2 /home/ubuntu/chia/plot-temp2/ -d /home/ubuntu
2

As far as I have observed, A plotting process does not use more than 3GB in reality even if given a very high amount of buffer memory. This observation is based on ubuntu 20.04.

1 Like
3

The plotter process uses just over 3 GB when using the default 128 buckets. If using a HDD for tmp the documentation suggest using fewer buckets. This increases the amount of memory used. 64 buckets doubles memory usage. It does improve the plot performance to HDD but only by 15-20%.

Buckets should align on powers of 2. Specifying a none power of 2 seems to round down to the nearest power of 2.

4

So that means setting a value of say -b 6800 does not have any effect and it will still be using around 3GB due to having 128 buckets.

5

That has been my observation. It only uses what it needs which seems to be driven by the bucket size. The memory option is just a limit that says don’t go over this amount. In the logs, I’ve seen it fall back to a different sort algorithm if it doesn’t have enough memory for the preferred algorithm so there is some flexibility there.

6

Yeah I agree. I’ve accidentally given parallel plots 8gb ram, and way too many of them that would have exploded the machine if they had actually used anything close to that in reality…

I’ve stopped giving plots extra memory, I just add 4096 to keep it an even power of two and leave it at that. Extra memory doesn’t speed anything up any more.

7

So I did another experiment with bucket size -u 64 and that’s where the plotter starts to take up more memory to do all its stuff in 64 buckets instead of 128.