Please help… What flags and arguments of "create plots " command are used for making parallel plots and staggering them while using CLI?
You can use the argument -h on any command to get the available options.
chia plots -h
#Options:
# -k, --size INTEGER Plot size [default: 32]
# --override-k Force size smaller than 32 [default: False]
# -n, --num INTEGER Number of plots or challenges [default: 1]
# -b, --buffer INTEGER Megabytes for sort/plot buffer [default:4608]
# -r, --num_threads INTEGER Number of threads to use [default: 2]
# -u, --buckets INTEGER Number of buckets [default: 128]
# -a, --alt_fingerprint INTEGER Enter the alternative fingerprint of the key you want to use
# -c, --pool_contract_address TEXT Address of where the pool reward will be sent to. Only used if alt_fingerprint and pool public key are None
# -f, --farmer_public_key TEXT Hex farmer public key
# -p, --pool_public_key TEXT Hex public key of pool
# -t, --tmp_dir PATH Temporary directory for plotting files[default: .]
# -2, --tmp2_dir PATH Second temporary directory for plotting files
# -d, --final_dir PATH Final directory for plots (relative or absolute) [default: .]
# -i, --plotid TEXT PlotID in hex for reproducing plots (debugging only)
# -m, --memo TEXT Memo in hex for reproducing plots (debugging only)
# -e, --nobitfield Disable bitfield
# -x, --exclude_final_dir Skips adding [final dir] to harvester for farming
# -h, --help Show this message and exit.
Example:
chia plots create -k32 -n1 -t/mnt/nvme -2/mnt/nvme -d/mnt/8tb/plots -b 4096 -u 128
-r 4 -a <your_fingerprint>
To run in parallel, just run 2 instances of the command, and you’ll have parallel plots.
3 Likes
Thanks. How about staggering? Is it the buffer flag?
No, you need to combine that command with the sleep command in linux, or Start-Sleep if you are in windows (powershell).
Bash
sleep 1800;chia plots create <args>
Powershell
Start-Sleep 1800;chia plots create <args>
If you want to launch plots based on the stage they are, you’ll need to Parse your plotter logs and launch the next plot based on that output.
2 Likes
Great. Thanks for your help
1 Like
I run the following in a linux shell script.
This starts a total of 24 plotters with an interval of 15 minutes, which is basically your stagger.
I am not passing the public and pool keys because I have the harvester initiated with the private keys of the main node.
for i in {1..24}
do
today=`date +%Y-%m-%d.%H:%M:%S`
echo "Starting plot no. $i"
chia plots create -k 32 -b 3400 -n 1 -r 2 -t <temp1> -2 <temp2> -d <plot final destination> plot_${today}_$i.log &
sleep 900
done
exit
2 Likes
Thanks, this is great help.
I am not passing the public and pool keys as arguments because I have the harvester initiated with the private keys of the main node.
If you’re using Linux:
I don’t have an overly complex setup, so I created the following bash script that executes on my plotter(s) via cron every 5 minutes.
Since the multi-threading is only done in Phase 1 and not really time based, this checks for phase 2 progress and starts new plots based on that.
Note: if you use this you’ll need to update the directory paths etc…
#!/bin/bash
echo "Starting Plot Manager script"
#Set number of parallel plot process this host can handle.
maxPlots=2
#Plot Log path:
plotLog="/home/user/.chia/mainnet/log/plot-process"
#Time Run log:
timeRunLog="/home/user/.chia/mainnet/log/timerun.log"
#Set host.
host="Plotter-1"
#Get current date/time.
dateTime=`date "+%F-%T"`
#Function to define the command for creating a new plot.
#
#This activates the chia python virtual environment
# . /home/user/chia-blockchain/activate
#
#This is the command used to create a new plotting process.
# chia plots create -k 32 -r 4 -b 3416 -t /mnt/chiatemp/ -d /mnt/chiaplots/
#
#Pipe the output to a log file for active monitoring of progress
# > $plotLog-$dateTime.log
#
#Once the plotting process is complete, append the start date/time and total time of process run to a log file.
# && echo "Date:"$dateTime-"Host:"$host"-"$(grep "Total time" $plotLog-$dateTime.log) >> $timeRunLog
#
#Delete the plot process log once it's all complete.
# && rm $plotLog-$dateTime.log
createPlot() {
. /home/user/chia-blockchain/activate
chia plots create -k 32 -r 4 -b 3416 -t /mnt/chiatemp/ -d /mnt/chiaplots/ > $plotLog-$dateTime.log && echo "Date:"$dateTime-"Host:"$host"-"$(grep "Total time" $plotLog-$dateTime.log) >> $timeRunLog && rm $plotLog-$dateTime.log &
}
#Get number of running plot processes.
runningPlots=`ps aux | grep '[c]hia plots create' | wc -l`
echo "Number of running plots:" $runningPlots
#Check for current running plots. If there are max...exit.
if [[ $runningPlots -ge $maxPlots ]]
then
#No Action needed. exit the script.
echo "Max plots already running."
exit 1
else
if [[ $runningPlots > 0 ]] #If there's at least 1 running plot do the following.
then
#If there is at least one plot, check the process logs for progress on computing.
phase2processes=`grep 'Starting phase 2/4' /home/chia/.chia/mainnet/log/plot-process*.log | wc -l`
echo "Number of plots that reached Phase 2:" $phase2processes
if [[ $runningPlots > $phase2processes ]] #Check if there are running plots in phase 1.
then
#Wait until all running plots have reached at least phase two.
echo "Not all plots have reached phase 2 computing...waiting."
exit 1
else
#All running plots has reached phase 2, kick off another plot process. (Via function above)
echo "All running plots have reached at least phase 2 computing. Plot process started"
createPlot
fi
else
#There are zero plots running, create one.
echo "Plot process started as there were none active."
createPlot
fi
fi
3 Likes
This looks great, thanks for your help
1 Like
I noticed that it staggers a proccess for 15 mins after finishing plot nr 1 to start plot nr 2 and so on.
So this is not exacly what I meant.
Although, If you put more instances of the command in separate terminal windows and stagger them every 900 ( or whatever suits you), it should work fine together. What do you think?
@cyb3rPhilomath, looks like for optimal usage of your script it’s also required to specify -2 parameter with dedicated SSD (defines a secondary temporary directory for plot creation). If I understood it correctly, you’re only waiting for the ending of Phase 1 and then start the creation of new sequential plots. So, you will have some plots running in Phase 3 and some in Phase 1 using the same SSD, which potentially will get overwhelmed in case of very precise use of resources (the most optimal maxPlots number). Does it make sense?
You just need to use sleep in the beggining, so it waits before starting “next window” plots. Then it should work fine, if you limit the iterations to max number of plots your sytem is capable of.
Am I thinking correctly?
I dont seem to have these folders by default. Does it create them?
#Plot Log path:
plotLog="/home/user/.chia/mainnet/log/plot-process"
#Time Run log:
timeRunLog="/home/user/.chia/mainnet/log/timerun.log"
Hi @zeitler,
Yes you’re correct. I implemented it with a single temp directory. (I typically have 2-3 parallel plots per SSD.) I think you could probably squeeze out another parallel plot or 2 using the -2 parameter since the temp directory would then be less utilized. (Although I haven’t tested that myself.) Good info here: 2nd Temporary Directory Experiments - #12 by WolfGT)
The script initially was meant to manage automatic plot creation and staggering to prevent over utilization of 1.) the temp directory and 2.) CPU threads during Phase 1 (8 CPU threads in my case). After Phase 1 is complete each plotting process drops back to a single thread, so if my system has 8 CPU threads I figured always having one in Phase 1 (4 CPU threads), then as many as 4 plots in Phase 2/3/4. (Maxplots=5) (I’m no expert though, just figuring this out as I go. lol)
I think I’ll try the -2 parameter though. Thanks for mentioning that.
Hi @witwitkowski,
When I installed the Chia blockchain app from https://www.chia.net/ on Linux it always had the log directory at /home/<user>/.chia/mainnet/log/.
Be sure to replace <user> with your actual username on the system.
So if you do a ls -al /home you should see a directory for your username.
Nah, it staggers 15 minutes after start of first plot. the & at the end of the start plots moves it to the wait. So this script will launch 24 plots with a delay of 15 minutes each.
And if you want them to continue plotting then you may do -n 2 in which case each plotter will start a new plot as soon as it finishes the first one.
There is no requirement to launch 1 script per terminal window as the script itself launches the plots passed in the for loop at 15 minute intervals.
Well, it didn’t seem to work as you expected. It was deploying another plot only after finishing the previous one, with stagger ontop of it.
This is on my system, I hope it works fine for you and the others.
I got the Plotman instead. So far it seems the best solution.
This is very odd, here is the full script.
cd ~/chia-blockchain
. ./activate
# run in plot logs folder
cd /home/ubuntu/.chia/mainnet/plot_logs
# machine name to append
MACHINE_NAME="m5.16xl_1"
# start plots
for i in {1..32}
do
today=`date +%Y-%m-%d.%H:%M:%S`
echo "Starting plot no. $i"
chia plots create -k 32 -b 3400 -n 1 -r 2 -t <temp1> -2 <temp2> -d <plot final destination> plot_${today}_$i.log &
sleep 900
done
exit
When you run it, run it like this
./startplots.sh &
Hi, I have tried your script. It seems to start up 3 plots at each 5 minute interval the script is called, then it stops at 3 plots (as I have set up no more than 3 parallel). The check for no plots in Phase 1 (before starting up another plot) … doesn’t seem to work …
I have tested the line that loads “phase2process” and that is “0” so not sure what is wrong …
Each time it started one of the 3 plots, it printed the line: “Plot process started as there were none active”
Cheers