CPU: 5950x PBO enabled. (running 4.7Ghz most of time)
RAM: 32Gb x 4 3200Mhz cl18
SSD: Samsung 980 pro 2TB x 4 with Asus Hyper M.2 (raid 0 using mdadm, filesystem: ext4)
plotman.yaml
tmpdir_max_jobs: 20
scheduling:
# Run a job on a particular temp dir only if the number of existing jobs
# before tmpdir_stagger_phase_major tmpdir_stagger_phase_minor
# is less than tmpdir_stagger_phase_limit.
# Phase major corresponds to the plot phase, phase minor corresponds to
# the table or table pair in sequence, phase limit corresponds to
# the number of plots allowed before [phase major, phase minor]
tmpdir_stagger_phase_major: 2
tmpdir_stagger_phase_minor: 1
# Optional: default is 1
tmpdir_stagger_phase_limit: 5
# Don't run more than this many jobs at a time on a single temp dir.
tmpdir_max_jobs: 20
# Don't run more than this many jobs at a time in total.
global_max_jobs: 21
# Don't run any jobs (across all temp dirs) more often than this, in minutes.
global_stagger_m: 20
# How often the daemon wakes to consider starting a new plot job, in seconds.
polling_time_s: 20
plotting:
k: 32
e: False # Use -e plotting option
n_threads: 4 # Threads per job
n_buckets: 128 # Number of buckets to split data into
job_buffer: 6144
In fact, I just want to utilize my huge memory that’s all.
I don’t think there’s bottleneck since CPU I/O wait time is under 10 sec. 4x 980 pro raid 0 rocks!
I have a 5950x, 64gb with a single Force MP600 2TB, due to limited SSD availability.
So I am running 8 plots in parallel, staggered, through plotman. Allocating 4 threads each.
I can confirm 20-22k seconds per plot.
What amazes me is that I easily get 28 plots per day, with just one NVME drive.
I am expecting to receive two more drives tomorrow. But I believe this shows that plotting is less bottlenecked by the nvme drives than I originally thought.
I tried xfs for temp but I thought it has stability issue. Basically I solved the issue with RAM spot cooling & changing file system from xfs to ext4. problem solved. I did those actions simultaneously, so I can’t tell what the real problem was.
That’s quite surprising. I’ve seen so many people having trouble plotting 8 on a 2TB disk
Would be really interesting to know what happens if you have two of those drives and plot 4 on each instead of 8 on one.
I’m currently on the thinking that both CPU and Nvme can be bottleneck at the same time.
Some parts of the process seem to go faster with a better CPU, regardless of the drive
Some parts of the process seem to go faster with a better Nvme, regardless of CPU
I dunno the more times and setups I see coming by the more difficult it becomes to figure out what affects plotting speed and to what degree. But in general, yes it seems that CPU speed and overall system balance are key and that nvme is a smaller part than you would think based on many articles out there
Yes I know the video, but that’s why it’s so surprising to see someone with rather good times on a MP600 with 8 plots on 1x2TB, that has a pretty low sustained write speed.
I see, the lack of 128 bit long double support in Microsoft’s C++ compliler is what is hampering us. Maybe someone could do an adaptation of the code for the Intel C++ compiler for Windows that does support it? My C++ is way to rusty to attempt this myself.
thanks for sharing this!