Windows Plot Mover - also deletes old plots to make space

[Ronski]

Warning

Please review the code to make sure you are happy with what’s its doing (the code is available on my Github account), and please use a cold wallet - running programs, especially an EXE where you can’t see what’s its doing is dangerous, and whilst I’m not doing anything dodgy (but I’m just someone on the net so you shouldn’t trust me) it doesn’t mean someone else’s code isn’t. You should ALWAYS take precautions to protect your XCH, this has already happened!

The program and source code is listed on Github, anyone can download the code, and build it themselves, this also means anybody can alter it and post the program online. PLEASE ONLY DOWNLOAD from my Github account, unless you trust other sources.

About me

I’m not a programmer, I’ve done the odd bit over the years, starting with BASIC on the ZX Spectrum back in the early eighties, then a bit of machine code on the Spectrum, and sometime around 2012 I co-wrote a program which was used by many people for logging VDSL modem stats. So the code may not be efficient, and there may well be odd bugs, or events not handle cleanly, but it worked for me.

I’ve never used Github before, but wanted to make the code public, so have uploaded it there.

Download.

Releases · Ron-ski/Windows-Plot-Mover

and code repositories, so people can check and build from source.

Ron-ski - Repositories

Description

This program is designed to run on Windows, it is written in Visual Studio 2022 VB Net, depending on your system you may need to update Dot Net.

I wrote this for my own use, but I can see that others with Windows systems struggle moving plots, so have decided to make it publicly available via Github. There are other systems available, but they tend to be complicated scripts, mainly for Linux.

Performance

How it performs is entirely dependant on you’re hardware. There will always be a bottle neck somewhere, drive read speeds, drive write speeds, network speeds etc. If using a network please be aware of the limitations of 1GbE which will only manage one move at a time, I use 10 GbE and have easily moved 6 plots at the same time.

Installation

There is no installation required, simply download the zip file, and extract the two programs. Double click “Windows Plot Mover” to get started, the other program is run automatically from the plot mover GUI.

Usage

The program uses drive letters as its much easier on the programming, if you have mounted drives just add a drive letter from disk management (they can stay mounted as well), you can remove the drive letter once you’ve finished with that drive. If you’re drives are on another machine then you need to map them to a drive letter.

I have recursive plot scanning turned on in the Chia GUI (Settings/harvester), its much easier, the majority of my drives are mounted in folders, so I just need to add the root mount folder to Chia and it then automatically picks up all plots in the sub folders.

On each drive I have a folder with the existing plots, I rename that to something like DeletePlots (it can be anything), then create another folder for the new plots, I normally name this so I know the drive.

Here it is mounted

Here it is with a drive letter assigned.

Repeat the above for all the drives you want to use.

Adding drives/destinations to the program.

On the drive locations tab, select the plot source drive first by clicking the browse button.

Then you can add the destination drives, by clicking their browse button. If you are deleting plots, add the delete plots folder, and tick the “Delete Plots” check box next to it.

“Disable and clear settings” will clear that lines settings, and disable it.

If you want to enter a nickname for the drive then enter that in the nickname box, I would use a reference name that refers to that specific disk.

Reserved space is used if you don’t want to completely fill a drive - I have a drive which is used for data and TV recordings, so I keep 700GB free.

So once you’ve added drives you will have something like this.

Now switch back to the Process Plots tab, so you have this, if there are plots on the source drive they will show up in the list at the top left.

You’ll need to change “Maximum consecutive moves” to suite your hardware (I suggest starting at one and increase it as you get a feel for it), then press start. The more plots you are moving, and the more drives you are writing to will affect Chia’s response time, so monitor that as well.

As each move completes it will cycle through the drives, deleting plots to make space if required, and moving plots.

Here are some drives now shown as full, when a drive is full you can switch to the destinations tab and select another drive to replace the full one.

When moves are started the standard Windows move dialogue comes up, but I couldn’t work out how to get them to open in different locations on the screen, so they all appear on top of each other, you can hover over the task bar icon or manually spread them out.

Eventually all the drives will be full, and I’ve just noticed an error in the message at the top - its not waiting for more plots, the drives are all full!

Currently the Chia Bladebit plotter will crash if the destination drive becomes full, I believe this will be fixed in the next release.

All moves are written to logs in the log folder, the last two numbers are the size, and the time it took to move.

Plots are renamed whilst being moved, and then renamed once complete. This is a screenshot from my Linux plotter, the files being moved are name .movXX, where XX is the drive number they are bing moved to.

[Ronski]

After you have finished filling a drive you should always run Chia Plots Check on the drive, and remove any invalid plots. You can use a command window as well.

How To Check Chia Plots Using The Command Line Tool In Powershell

You can use a command window as well.

Open an explorer window and enter %localappdata% in to the address bar and hit enter.

Then navigate to the following folder

C:\Users<yourusername>\AppData\Local\Programs\Chia\resources\app.asar.unpacked\daemon

Then type cmd in the address bar

then press enter

You can now run chia commands from the cmd line.

[jack6070]

Will Chia software detect invalid plot, automatically?

I am asking this because the Chia software “plots” section displays how many bad plot.

[Ronski]

Yes it should do, but I think it is always better to check as you go.

I would also suggest using Notepad ++ to search the debug logs for “Invalid Proof of space” after that message will be a 64 alphanumerical string such as a0d4e0be0c69791d6cc8cc1f15b0502f19a7c73197ab88109d60123ec94ad0bb which is the plot ID.

If you do chia plots check -g a0d4e0be0c69791d6cc8cc1f15b0502f19a7c73197ab88109d60123ec94ad0bb it will then check that plot, it may pass but have a very low score with lots of warnings, simply delete it and copy a new one in. I’ve had about a dozen of these ‘bad’ plots.

[Ronski]

You need to temporarily add drive letters to the drives.

Just go in to drive manager and add a drive letter, it can remain mapped to the folder as well. Once you’ve done moving plots to the drive remove the drive letter.

It was very complicated to work with mounted drives, and using drive letters was a lot easier to program, not ideal but it’s much easier for me to program.

Sorry about the lack of resizing, I’ve not learnt how to do that yet.

You should be able to move the window to the side to reveal the max moves. Left click on the title bar as you would any other window and drag it to the left.

[Ronski]

I’ve added the ability to move NoSSD plot files, selection is by the two drop down lists on the main page.

One selects the format to move, the other the format to delete if applicable.

The filenames in the below screenshot are wrong, I just changed the file extension for testing purposes, I’ve only done limited testing but it seems to work fine.

[jack6070]

Thanks, Sir. I haven’t really used this program, but I have followed your instruction from the early time regarding data transfer from Linux to Windows. I got 10GB SPF cards and cables. They were cheap. Two DELL 720 with 1080 GPU are sending plots to a Windows harverster. 500 MB/sec.

[Ronski]

Nice, good to hear, and glad its all working.

[RobertH]

It seemed to work well for a couple of days then Windows produced this error and quarantined the file.

[Ronski]

That is most strange. If you’ve downloaded it from my GitHub account then it doesn’t contain any viruses, it’s written and compiled in Visual Studio 2022, Microsoft’s own program.

Please upload the file to VirusTotal and see what that says - I’m at work at the moment.

The only other thing you can do is submit it to Microsoft as a false positive, they will then check the file and clear it.

I will check tonight, and see what comes up on my system.

Thanks for the heads up.

[Ronski]

Just done a scan against the GitHub URL as I’m at work on my tablet.

[RobertH]

i did one too. Virus total showed it clean. I downloaded it direct from Github. Just windows doing it’s thing.

It is good to use this utility once you get the drive mapping sorted for a remote server.

Some feed back

1. found that it kept dropping out doing 4 transfers to windows harvester over 10G direct connect. - Fixed it by turning off indexing for the hard drives.
2. if the harvester is rebooted with no files being transferred it comes with an error and crashes out. Could be a bit more graceful and just say no network connection.

[Ronski]

Thanks, I’ll submit it to Microsoft tonight, hopefully should solve that problem.

Not sure what would cause issue one.

With regard to issue two, if I understand correctly the program running on the plotter, and when the harvester is rebooted it crashes on the plotter?

There is a bug I’m aware of, and that is it still shows the list of plots if the plot source drive becomes unavailable, although it didn’t cause a crash. I’ll add this to my list of bugs and see if I can replicate it when I get chance but will likely be a while before I get chance.

[Ronski]

I submitted the file to Microsoft, and they found no problems, so hopefully no more false detections.

I’ve released another version with some minor fixes.

• Minor bug fix - plot list wasn’t cleared if the plot source drive became unavailable.
• Add some error catching.

I’m hoping the error catching I’ve put in will catch the error @RobertH experienced, but I couldn’t replicate the error.

Releases · Ron-ski/Windows-Plot-Mover

[Ronski]

I’ve also had to submit last nights release, as MS is picking that one up as a PUA.

Virus Total also has one positive, but the company (Max Secure) that detected doesn’t even seem to have a way to submit files, and also their contact us form thinks my email address is invalid!!!

Seems I need to check prior to each release, what a pain for such a basic program.

[Jacek]

I had a similar thing happening to my exe(s) (downloads blocked, and if permitted, file was killed when touched). It is not really a virus, but rather over aggressive Defender to basically be on par with other anti-virus packages. Still, it is rather hard to explain the problem for those that are trying to use such binary.

When you use Edge and try to download such binary, you can click on download progress highlight the warning message, and to the right of it you will get more options. One of those options is to say that file is safe for downloading. When you click that one, you will have a page where you can select that you are the owner of the file, and there will be a bigger form to submit to MS.

I did that, got an automated reply that someone will be in touch, but unfortunately no one did. As I only shared that file with one friend, I gave up on that process (that download page is locked just for two of us, so MS may have problems testing it - saw that they tried to access that file, but of course failed to do so).

In your case, when you are submitting the binary, you are just trying to make an exception for that one file, so it is a nuisance. Also, it takes some time for MS to address the download part, and a bit more to make file Defender exclusion. Potentially, when you go through that Edge submission page, they will make an exception for your github download page, thus all future versions will be good to go (MS owns github).

Again, I didn’t really pursue to finish the process, but I think it may work.

[Ronski]

Thanks, antivirus will look for signatures, and also try and work out what the code is doing, in this case it takes a dislike and flags it as something, which seems to be different each down.

You’d think that MS would scan available downloads on Github, wouldn’t you, perhaps they are but as you say, it’s just not worked through the system.

I’ll try the download via edge from Github, and see if that does anything, thanks.

[Jacek]

@Ronski

You’d think that MS would scan available downloads on Github

Well, one can only scan for what one understands (known threats). Your false positive is not because it tripped a well know rule, but because they have some extra heuristics that are just pulling restrictions from a thin air (what may work for a lot of scam binaries). Also, most likely Defender is more aggressive for the first few downloads, and quiets down if there are no “that’s not safe” reports. So, once you submit a new version, download it with Edge, and mark it as safe (through that extra menu), maybe that will relax those heuristics.

By the way, when submitting that MS form, I would also link to this thread to show that you have users that appreciate what you are doing and are asking for changes / updates to the binary.

Maybe contacting Github would also not hurt. They are part of MS, thus may be able to reach to Defender team faster.

[Ninevah]

This looks great. Does it only allow for 1 source location? I have 8 plotters, so I’m usually moving plots from 8 different mapped drives to my farmer.

[Ronski]

Yes, it only allows for 1 source location.