Ploto: A PowerShell Module for Plotting

Chia Tools & Scripts
1

I’m a Windows Farmer and needed to automate my plotting process.
I found plotman and was blown away by it, but there was no way switching to Linux solely for plotting (Its my gaming rig :P).

I also did not want to hustle around with Python and WSL, otherwise I could have gone with Swar’s implementation, which is a great tool!

So I decided to develop a plotman inspired alternative using Windows PowerShell.
All of this ended up in a PowerShell module that some of you may find helpful:

Features at a glance

  • Automatically starts Jobs on the bets fitted temp and outdrive
  • Sends notifications when a jobs is started (more to come)
  • Define max amount of plots to be run in parallel
  • Get active Plot jobs details (PerfCounters)
  • Ability to stop a Plot job and automatically deletes .tmp and . log files
  • Ability to automatically move finished plots to a UNC path or other drive.

Feel free to contribute, adapt and re-use :slight_smile:

7 Likes
2

These are some idea.
If we can list out all drive, find best combo and plot all of them would be great

2 Likes
3

Very cool. Will check it out as soon as all my crappy .ps1 files finish plotting!

Thanks!

5

I did some updates regarding the Discord reporting. You now can configure an intervall upon which you’d like to get information about your jobs in progress, and completed jobs in the defined intervall.

image
image509×855 68.9 KB

EDIT: I did not want to reply to you alphaito. Sorry.

1 Like
6

I’ve just published another release that focuses on fixing various issues of spawning.

Changed

  • The core logic has been refactored which makes Ploto much more understandable, readable and ultimately; more reliable
    * It does not wait anymore for a given TempDrive to become plotable. It checks all TempDrives peridodically upon definable Intervall (default 5 min)
  • Ploto now uses a ConfigFile instead of Parameters to control spawning of jobs (Yes, you can edit it on the fly, meaning when Ploto is running)
  • Moved some Write-Hosts to Write-Verbose

Added

  • Ploto now checks for abortedJobs and deletes them upon every iteration
  • Some more detailled -verbose logging

Fixed

  • Several issues

Known Bugs

  • #48
    -InputAmountToSpawn is blatantly ignored for now. Why? I focus on successfully spawning jobs as fast as possible rather than complying with that param. Overcommit should not possible anyway due to checking OutDrives. And also If a jobs fills a drive, you may copy/move stuff from on that drive and te job can continue. Will be fixed though!
7

And another one. Sorry for the spam. :slightly_smiling_face:

Added

  • Ability to use -“2” param for chia.exe (temp2 drives)
    Assumes dedicated temp2 drives. A raw implementation.
    If a t2 drive is available it uses it, if not, it tries to launch a job without it.

  • Ability to define Window Style of spawned plot jobs (chia.exe)
    Cause I closed all parallel plots today trough fatfingering.

    Accepts values:

    • normal
    • hidden
    • maximized
    • minimized

Changed

  • Switched from WMI to CIM. Should now work for a broader range of PowerShell versions without adjustments and also more reliable (Could not get TempDrive issue)

Fixed

  • An issue where Ploto would not want to stop generate plots (ignoring “InputAmountToSpawn” param)
  • Some other issues
8

In eagerly awaiting pool protocol support:

  • Ploto now supports the specification of Famer and Pool Public Keys for the plots created with it.

And to let you know what to expect within the next couple weeks, heres the raw roadmap I layed out for myself:

Upcoming releases will include (next few days probably):

  • More dynamic staggering

    • Limit max parallel amount of jobs in phase 1 on a single disk
    • Limit max parallel amount of jobs in phase 1 across all disks
    • Ability to start a new plot, when phase 3 is completed of another

  • Enhancements for temp2 usage

    • Only calculate jobs in phase 3 & 4 towards AmountOfPlotsToTemp on that t2 disk

  • More disk stats

    • Show off temperature of drives in report (Get-PlotoJobs -PerfCounter and Discord notifications)

  • Daily summary reports by Discord

    • Amount of plots created last 24 hours
    • Plots created along with stats (StartTime EndTime, Times for Phases 1-4)

Midterm Roadmap (next few weeks to a month or two)

  • More configurable notifications

    • PlotJob finished
    • Plot moved
    • PlotJob status unknown
    • PlotJob aborted
    • OutDrive Overview
    • TempDrive Overview

  • Config generator

    • Automatically try to generate a config based on specification of host and best-practises

  • PlotoFy farmer monitoring (separate Repo); notifies in Discord about what your farmer is doing

1 Like
9

I just published module version 1.0.9.4.3 which brings the following:

Added

  • Ability to define pool & farmer key
  • Ability to define max parallel jobs in phase 1 per each disk
  • Ability to define max parallel jobs in phase 1 on all disks
  • Ability to launch a new plot when a given phase is completed (EarlyStart, set to 3 advised. So when a job completed phase 3, the next can be started)
  • PlotJob Phase on Get-PlotoDisks (Temp/Out/T2)

Changed

  • Waaay less noisy verbose. Finally is usable to trace what Ploto is doing
  • Merged Spawner and Alert config in one
  • The most appropriate OutDrive is now specified by the least amount of jobs it has

Fixed

  • Improper Job Count when Alarms enabled
  • Some bugfixes and refactoring
10

It’s me again with some updates.
Also started a Discord Server. So if you have any questions, issues, comments or just want to chat, you may join: Discord

Added

  • Ability to start PlotoFy when Spawner is launched
  • Remove-AbortedJobs/Stop-PlotoJobs now also does clean up of temporary t2 files
  • PlotoSpawner now analyzes the output of the function that loads the config for known errors. If an error is known (missing ", or , and or using \ instead of /) a hint is displayed
  • More resilience for Remove-AbortedJobs/Stop-PlotoJobs trough less generalization
  • Some more output for console

Changed

  • Removed MaxParallelJobsInPhase1OnSameDisk. Its a useless param.
  • Adjusted some Write-Host/Verbose outputs all around Ploto (except mover)
  • Split of “plots create” from Get-PlotoJobs output
  • JobLog generated by chia GUI or CLI Plotter are now ignored by Ploto (No need to delete)

Fixed

  • fixed a bug that prevented spawning (MaxJobsOnDiskInP1 was calculated wrong)
  • a bug where Get-PlotoOutDrives could not get the right amount of plots to hold
  • surpressed error fom Stop-PlotoJob is its the known one
2 Likes
11

Heiho

Theres been an update to Ploto. In order to prepare for pool plotting, it now features support for P2SingletonAdress and -c option for chia plots create.

Added

  • Ability to pass -c param to Chia plots create (portable pool plots)
    Be advised, this is a beta. It is based on the demo we saw from Mariano at the pool operators call. Might not be working correctly yet.

  • Ploto now checks for NVMEs to spawn the jobs on. If an NVME is found and defined as a TempDrive, it will be used first to spawn a plot on.

Changed

  • Added an additional check for OutDrives, if they have JobsInProgress.

Removed

  • ProgressBar for PlotoMover using Bits
12

Me again.

I’ve just release version 1.0.9.5 which brings the following:

Added

  • An Installer script that correctly installs/updates Ploto Module and guides trough the setup of the config.
    Also copies config to where it needs to be.
  • Ability to launch Ploto commands globally without need to import first (Thanks to installer)
  • Replotting capabilities; A watchdog that looks for jobs that are about to enter phase 4 and has IsReplot set to true. Then deletes a plot on defined RepotDrive closes
  • Ability to Define ReplotDriveDenom

Fixed

  • A bug that did not set -f param when usin a P2Singleton to plot
13

A beta is available that supports Madmax (Stotiks version of it) in Ploto.
You may get it from, the testing branch here: https://github.com/tydeno/Ploto/tree/testing-stotikmadmax

If you want to test it, make it sure you launch Install-Ploto script and define your config to update it.
This one also breaks existing logs. So if you have existing logs in your plotter folder, delete or remove them.

Please be aware that I did not cross-test if chia official plotter still works (it should as separate logic). This will be tested later on, as I focus on getting Stotik/Madmax supported.
Also make sure you adjust the config to how Stotik works (only 1x parallel etc.)

And the most important note: Do not hesitate to create or update an issue in Github when find a bug or error.

14

Release 1.1 for Ploto is here!

It now supports using custom plotters of stotiks/madMAx43v3r.

Adds

  • Support for madMAx43v3r and stotiks blazing fast custom plotter.
  • A few improvements on the installer
    • shows selected drives by denom in cfg mode
  • A config handout when Ploto Spawns
  • A cleaner config
  • Notifications when Remove-AbortedPlotoJObs detects an aborted Job
  • Add Size of plotfiles on t2 drive in Get-PlotoJobs

Changes

  • Ommits -p and -f in output of Get-PlotoJobs and shows “YourKeys” instead for those params.

Fixes

  • Completed Job is considered “Aborted”
  • When T2 is same as TempDrive error on Remove-AbortedJobs
  • and various more

How to update (make sure you follow that procedure)

  1. Make sure you flush/move your existing logs in your plotter folder (C:\Users\Tydeno\.chia\mainnet\plotter)
  2. Launch a PowerShell session
  3. cd to downloaded source ( for example cd C:\Users\Tydeno\desktop\ploto )
  4. Launch Install-Ploto.ps1 (As Admin)
    4.1 Update Module
    4.2 set a new config using Install-Ploto (Introduces some cfg changes)
    4.3. Let install script copy the config
  5. In PowerShell launch Start-PlotoSpawns

If you use different Users for administrative Tasks and normal usage of your PC (restricted normal user under which context you run chia), you have to abort the script after you defined your config (Where its asks if you want to get it into production)
The values you defined are then saved in downloaded config in the source folder. Afterwards, simply copy the config from sourcefolder to
C:\Users\Tydeno\.chia\mainnet\config

How to use Stotik Plotter

If you want to use Stotik Plotter, make sure you set PlotterUsed to “Stotik” and that you specify the fullpath to the .exe in the PlotoSpawnerConfig.json

15

v1.1.1 is available!

This release focusses on moving from the former Drive Denoms to using distinct driveletters and paths for your Temp, T2 and OutDrives.

Added

  • Support for Ramdisks closes #88

Changed

  • Now using Driveletters and or UNC paths instead of a denominator to specify your drives closes #94

Sample disk config:

   "DiskConfig":  [
                       {
                           "TempDrives":  "B:,D:,E:",
                           "Temp2Drives":  "A:,Y:,Z:",
                           "OutDrives":  "H:\\Plots,K:\\Archiv,I:,P:,Q:,R:,S:,T:,X:",
                           "EnableT2":  "true",
                           "ReplotDrives":  ""
                       }
                   ],

How to update

  1. Make sure you flush/move your existing logs in your plotter folder (C:\Users\Tydeno.chia\mainnet\plotter)

  2. Launch a PowerShell session

  3. cd to downloaded source ( for example cd C:\Users\Tydeno\desktop\ploto )

  4. Launch Install-Ploto.ps1 (As Admin)
    4.1 Update Module
    4.2 set a new config using Install-Ploto (Introduces some cfg changes)
    4.3 Let install script copy the config

  5. In PowerShell launch Start-PlotoSpawns

If you use different Users for administrative Tasks and normal usage of your PC (restricted normal user under which context you run chia), you have to abort the script after you defined your config (Where its asks if you want to get it into production)
The values you defined are then saved in downloaded config in the source folder. Afterwards, simply copy the config from sourcefolder to
C:\Users\Tydeno.chia\mainnet\config

16

image
image701×697 70.5 KB

v.1.1.2 is available

This release focusses on bringing PlotoMover into the PlotoSpawnerConfig.json aswell and make it startable upon launch. Until now, it was necessary to start another PowerShell Session to fire up PlotoMover. Alongside that, it now also supports MountPoints and Network drives! It also brings some new capabilities within the replot mechanism. Ontop of all, it includes the new logo and new illustrations for the readme.

Added

  • Support MountPoints as OutDrives; Ploto now support the use of MountPoints as outdrives and final drives for mover.
  • Support of Network Drives as OutDrives; Ploto now support the use of Network drives as outdrives and final drives for mover
  • "ReplotPlotsOlderThan": "20.6.2021 in config parameter. Allows you to define which plots shall be replotted and which not by specifying a date as seen above. This is a mandatory param if you want to replot, as its the only way to determine which plots shall be replotted
  • Replot mechanism now picks those drives that have final plots on it for replotting. If there is no drive that has plots to replot, its chooses the best suited OutDrive by free space. If there is none, it aborts.
  • Mover is now able to handle several final drives. If there are more than 1 final drives, it picks the one that has no BITS Transfers in progress. If there are non with BITS Transfer in progress, it selects the one with the most available space.

Changed

  • PlotoMover cannot be started anymore with params. It uses definable Drives and Paths in config. closes
  • Changed some outputs to proper meaning and source (function)
  • Mover now solely relies on BITS to transfer plots
  • Mover now transfer plots async and waits 30 minutes between each move of a plot
  • mover ignores plotsolderthan to be moved if replot enabled
  • reduced sleep of Watchdog intervall to check for jobs to 1 minute instead of 5

Fixed

  • An issue where -p and -f were not ommitted correctly.
  • A couple of potential bugs