I just finished my first plot with the MadMax plotter and it only took 40 mins! However I am mainly here to ask if it is possible to run multiple instances so that you would be parallel plotting. I understand the software is new, but I was wondering if anyone has done that yet and if they had any results that they would want to share.
I havenāt run it myself, but i was reading all the topics and wondering if there might be a diminishing return somewhere because it just tries to use everything the system has.
So I was thinking maybe you could run a VM, split up yur system resources and try running two like that.
Just brainfarting here btw
1 Like
Ohhhhh, I didnāt even think about using a vm. That would be quite interesting, although idk how much the overhead from the vm would effect the plotting time.
Itās not really designed to be parallel, it is designed for maximum throughput and continuation, i.e throw your entire systems power behind making a single plot (thus there is no system resources free to make another in parallel) this is pretty much why itās so fast at producing a single plot, then you simply run it again - looping after each job completes.
3 Likes
You need to track which resources it uses (memory, threads, CPU) then time the parallel plots to āflatten the curveā as it shift from resource to resource - i canāt imagine how any VM can help - they just add more overhead in the emulation
Not a lot different to how plot manager work now - just the resource/timing parameters are different
1 Like
I just ran two in parallel to see what happens, and the second one was bottlenecked so hard, I couldnāt even see many read/writes to the assigned NVMe, while the first one was rolling. I am sticking to consecutive plots for now as I managed to reach the same daily throughput as when using 16 plots in parallel (35 minutes per plot on a 5900x with 980 pro as tmp drives).
2 Likes
Anyone tried parallel dividing the threads? Like max 12 so use 6 threads each .bat (for 2 bats on parallel)?
I seeā¦ Thank you for that info! Iāve been reading that it might help if you stagger the plots at phase 1 and 3. I might try that out see if that will give me any better results, but so far Iām just baffled by how fast this plotter is.
I tried 8 and 8, no good. Like, before something would take 10 seconds, and after each one took like 30, so much worse.
1 Like
Daaam, I hoped it would work, so my 64G ram is useless, I wish I`ve 128G right now.
Not quite useless. Linux does use the for buffers and caching and they end up altering the end result. I tried something to make an hybrid ram/nvme disk and pass it as temp2 but i ended up with worse results even though iām pretty sure it was working. Luckily my 64gb is 2 sticks so i just ordered the same kit i have and iāll have 128 (and probably save 3 nvmes from certain death)
So what u r saying is this fast plot sw is actually equivalent to the āofficialā sw. It works harder on each plot, but canāt do multiple plots. In the end one=the other. I guess itās like bragging about 0-60mph times in a sports car, when the actual goal is fastest time around the whole track.
PlotNG v0.23
New experimental support for MadMax Plotter
I was thinking this as well at first, but the (really) good thing here is that if you use ramdisks or a combination with fast sas hddās, there is no wear, no more burning up nvmeās
I was expecting fury road which I now finally have working, to max out my system, but I cannot get it to go much over 60-70% processor, I just donāt think my ramdrive is fast enough. So Iām spending 3000 seconds on phase 1, then I would, for now, manually start another instance. Iāll try it now.
That is not the point. If you are plotting yourself you know that it you stagger by 30 min it takes 8 hours for your system to reach full usage if you run 16 plots in parallel. And if you want to turn off the computer to do some maintenance or add some hardware, it will also take you around 6 hours from stopping the creation of new plots until all running plots are done.
Also, if your system crashes, you lose only 1 running plot with madmax vs 16 with plotman. That is a lot of wasted read/writes on your nvme. Finally, by using 110Gb ram as tmp drive 1, you offload 75% of the read/writes to the ram, sparing your nvme. That is if you have that much ram
2 Likes
Yes, those are valid points. I would mention, though, that plotting is a race, not a sprintā¦meaning itās not really necessary to be in a terrible hurry so a couple hours getting up to speed (if u need to) full plotting isnāt the worst thing. Crashing is thou, but that shouldnāt be happening if uv got (made) a stable system. I do like the ram idea, I have 144GB so it would be fun to use it like that.
About staggering, Iāve found I can pretty much start 15 plots (3 x 5 nvmes with max 40 min delay (wi/each group) and itās fine. Long staggers smooth things, but are a PITA to monitor. I like to sleep too, so less stagger means plot groups end every10-14 hrs that way.
Last, was reading that these fast plots are apparently not exactly like regular plots, theyāre studying why, even thou they pass the plot tests. So at ur own risk. Personally, I have only 120 plots before no more disks. So nothing new is so easy to finish those up. Have another PC I may try them on for fun, just no where to put them!
Wow, anyone tried out?
I tried a variety of combinations using madmax-furyroad 0.0.4
System is E5-2650v2 dual 16 cores 32 threads, 8 x 3TiB SAS drives, no SSD/NVME, 320GB DDR3. 280GB ramdrive (might be able to cut this down as the plotter doesnāt seem to use as much ramdrive as suggested).
1 plot 28 threads, 128 buckets, temp1-SAS temp 2-ramdrive 13000 seconds approx. for 1 plot. 7/day
2 parallel plots with a 50 minute stagger same settings 13000 seconds for plot 1 and 14000 for plot 2 which started 3000 seconds later. So 2 plots in 17000 seconds. 12/day
This seems fairly reasonable, the fastest guys with AMD Epyc and DDR4 ram are getting 20-25 minute plots which is 7-8 times faster as their processor is about 8 times faster than mine.
I have tried plotting using SAS for temp drives and it seems to be around 4 times slower. So no gain there. I will switch them to raid 0 pairs (they are individual drives atm) which might help but I will keep using the ramdrive for temp2 and striped SAS for temp1.
I am fairly sure if I had 12 drives connected and plotted 12 in parallel which takes 16 hours using these drives as temp, as I do not want to damage any nvme/ssd, I could plot around 18/day, but with the issues that any shutdowns or problems wastes a whole bunch of partial plots. However, as I only have 8 drives on this machine (I tried adding 6 more USB in parallel but something crashed the system, so I am not sure if that will work).
Latest thing now, I have two staggered processes, I will see how many plots this produces and whether there are any problems. Iāve set 20 threads for each process I donāt know if this is the right number it seems to make not all that much difference.
Just out of interest this seems very bottlenecked by disk speed, hence the recommendation for ram drive. I am currently plotting using this plotter on my laptop to a USB HDD and itās only managing 10MB/s my processor is at 5%. So it seems as though it can only utilise the processor fully if the disk can keep up.
I have 2x E5-2680V2, total 20 cores, 40 threads. 128GB RAM.
With 110G for temp2 and NVME for temp, I found most efficient was -r 16 with plot time of 36mins. Fewer or more thread will increase plot time, go figure!
Then I tried running another instance, since there are still 24 threads available, with temp on a second NVME, no t2 (not enough RAM). First instance plot time increased to 39-40mins. 2nd instance was 60mins. But after a couple of plots, both processes died. Donāt know why. Most likely RAM since only 18GB left after the 110G ramdrive.