help just sais
“-w, --waitforcopy Wait for copy to start next plot”
Does this mean that from 0.07 it does not wait for copy by default and -w activates it or do I use “-w 0” or “-w n” to make it parallel run while copying?
Thanks
J
Enabling -w tells the script to stop when the plot is created and wait until the copy to the final destination has completed before beginning the next plot.
That is my understanding.
1 Like
OP - What normally happens is that plot 1 completes, then begins to copy, this is essentially a separate process, plot 2 begins plotting immediately. With any type of really fast copy, such as to another directory on the same disk, this is more or less instant. Setting -w or not makes no difference.
With the -w option the enabled, plot 1 plots, then copies, nothing else happens until this is complete, then plot 2 begins.
I can only assume it is for situations where the speed of copying, lets say its over a slow network connection, is too slow to keep up with plot creation, which will cause the temp drive to fill up and then plotting to stop/crash. That being said you could solve this problem by reducing the core count, to slow plotting down, as plotting at maximum speed is only worthwhile if you can move the plots to the final drive at least as fast as you can plot them.
1 Like
Good evening everyone.
After plot complete, my final directory is in my harvester computer.While second plot starts immediately, the previous plot file is being copy at the same time over network to my harvester, and I never had any interruption on my plotting machine, or slow down in plotting times.
I’m using madmax.
1 Like
I am running Chia version 1.2.11 on Windows 10.
When I run, for example:
chia plotters madmax -n 10 … etc…
…when a plot completes, the next plot does not start its creation process until the completed one copies to its final destination.
In other words, it runs as if -w is present.
Is this happening to anyone else?
It can happen when the copy of the previous plot (that is, the one before the one that just completed) is not finished yet. That is because madmax can only manage 1 copy and 1 plotting : when there is still one plot copying to another drive, the copy of the finished plot cannot take place, and the plotter waits.
Then what is the purpose of the -w option? madmax seems to conduct itself exactly the same way, whether you use -w or you do not use -w.
The -w is one of those that don’t work as intended in 1.2.11 - see Madmax command line syntax - #12 by billy
To never transfer a file while plotting. What I described is an edge case, make sure your drive is fast enough.
I do not understand.
If I use the -w option, then the madmax plot waits for the copy to complete before starting the creation of the next plot.
If I do not use the -w option, then the madmax plot waits for the copy to complete before starting the creation of the next plot.
Using the -w or not using the -w results in exactly the same operations. So what is the point of it?
If you do not use the -w option, madmax will start working on the next plot, except in the situation I described.
What is not discussed here, but a question on my mind… My destination Dir is a USB3 external drive - it will easily take the K32 before the completion of the next plot so filling up the plotting drive(s) is not the main concern. If I don’t wait for copy though, will I strain the I/O on the plotting drive and wear it out quicker? In my case T1 is a RAID0 of 5 HDD, and T2 a RAM disk. Where does the file come from exactly? T1 or T2?
In my experience multiple normal RW on disks (e.g. copying movies), especially HDD… grinds things to a halt and takes much longer than the some of consecutive copies.
Edit: With -w disabled copy took 50mins (according to the MM CLI logs). When I copied a plot raw (without plotting) via Ubuntu GUI, it took 40mins. So it seems there is a bit of overlap.
You should try using bpytop, as it shows not what “mv” or whatever does, but rather the actual io to the drive. Linux will happily grab the file and push it into the RAM cache (if available), and report back to you that copying is done, where the physical copy may be taking few more minutes. You should also disable swap on all drives.
Since you are using SAS RAID, you should think about MM process as three parts, where the extra part is the staging / destination / offloading of the plot. Maybe you should do one more SAS RAID only with 2 drives (should be good enough) that will be used just for that part. This way, you offload t1 from parallel reads / writes of different files.
Using -w I have found my plots taking longer over time (from 55 mins to 1hr 45 by about plot 5 consecutively). It must mean there is IO conflict.
I wonder, if -G (toggle) would help this. @Jacek do you know if the plot copy creates IO on T1 or T2? And what of T1 or T2 is most strained under the initial phase(s), when the copy is finishing?
My understanding is that -G toggles t1 and t2 when the next plots starts not during a single plot run.
bpytop shows physical IO on drives. Also, long time ago, someone posted a chart outlining disk access through out the plot, you would need to search forum (although, it may be that it was based on chia plotter, so will be kind of irrelevant). In general, t2 takes 3/4 of writes comparing to t1 (that’s why we use RAM disk for t2 first).
I haven’t had a chance to use bpytop yet but turning on -w (wait for copy) has stopped my performance leak. Plots are consistently ~55mins (plus copy time of around 40mins), whereas with -w they get longer and longer. I’m not too fussed TBH. Stability is more important. I just want to fill up a drive as I accumulate them.
-G, --tmptoggle didn’t improve anything