ChiaGarden - A Toolkit for PoST Farming on Linux

ChiaGarden: A Toolkit for PoST Farming on Linux

Hello community,

I’m efnats. I’ve been working on a project named ChiaGarden, which I hope some of you might find useful. It’s a collection of tools I put together to simplify certain aspects of managing a PoST (Proof of Space and Time) farm on a Linux system.

Features of ChiaGarden:

  • chiainit: A tool that can assist with wiping, formatting, and labeling multiple drives.
  • gardenmount: Aims to reduce manual effort by automatically mounting CHIA-related HDDs. It comes with a systemd service for boot-time automounting and supports MergerFS.
  • plot_counter: Displays the number of plots for each C-level. It’s been tested with gigahorse and uncompressed plots so far.
  • plot_mover: A straightforward script to assist with moving plots, handling one at a time currently.
  • plot_over: Helpful for replots. Supports gigahorse and uncompressed plots for now.
  • plot_starter: This script and its associated systemd service can start plots automatically upon boot.
  • taco_list: An attempt to save time by wrapping chia mountpoints, reducing the need to manually type them.

You can explore ChiaGarden in detail on GitHub.

I genuinely hope this can be of some help to the community. I’m around for any questions, and if you have suggestions or feature requests, I’d be grateful for the feedback.

Happy farming,
:seedling: efnats

2 Likes

Update Okt 11 2023:

Hello again!

Here are updates for ChiaGarden.

  • Gardenmount

Introducing the ‘slack’ feature
This feature is specially designed to enable optimized space utilization across hard drives. It creates a btrfs raid0 from the leftover or ‘slack’ space on each HDD and mounts it to a pre-defined mount point (default is set to /mnt/slack).

Now, to clear any misconceptions: This isn’t a tool to establish a complete raid0 over all your disks. I understand that a full raid0 isn’t ideal for Chia farming. Instead, what this does is to allocate the remaining ‘slack’ space on your hard drives, connect them via loop devices, and cross-mount these as a raid0 to your chosen mountpoint.

A few things to note:

  • While a disk failure will affect the integrity of the entire raid0, it won’t impact the plot files that are written directly to the disk.
  • Should there be a mishap, the raid can be quickly reconfigured, and what’s even cooler is its ability to automatically expand with the addition of new disks.
  • I’ve also introduced a maxsize parameter. If the slack file creation surpasses this set threshold (which should ideally be a bit more than the smallest expected plot size), the HD in question is being skipped. This is to ensure you’re not caught off-guard shifting an entire drive’s plot space to the raid0.
  • Bonus! I’ve integrated the raid0 into the mergerfs mount point, which is set at /mnt/garden, further streamlining your farming process.

Older updates (Sept 23)

  • Chiainit

long awaited: drive ranges

  • To make things even more convenient, I’ve added support for drive ranges. Now you can easily manage a range of drives with patterns like sda-sz or sda-sdbz and so on.
  • New argument --all: Operate on all spinning drives (excluding system drive and mounted drives)
  • New argument --exclude [drive1 drive2]: Exclude the specified drives from the operation

If you have suggestions or questions, please don’t hesitate to reach out.

Here’s to efficient farming!
:seedling: efnats

1 Like

Gigahorse 2.5 is out and I wanted to say that the entire ChiaGarden toolset is compatible with the new plot format (and naming scheme).

Here is a screenshot of the replot helper script plot_over in action.
I think its self explanatory. It will gradually remove plot files making sure that you always have enough free disks to offer enough bandwith to offload new plots from your SSD thus making sure you can continue to farm during your replotting phase.

The plot_counter will give you insights on the current status and how fast you’re plotting. (Yes, I am replotting to C18) :slightly_smiling_face:

I am happy to help if you guys and girls have questions.

:seedling: efnats

1 Like

heyhey… looking at the screenshot of plot_counter there obviously was a bug recognizing the plot levels. Fixed :wink:

I’ve just updated the plot_counter to include new metrics:

  • eTiB (effective Size)
  • Average Seconds/Plot in the given period

So are you also working on the Windows version?

I am sorry, no. But if you want to try out Linux to test Chiagarden I’ll help you get set.

edit: I see in you Bio that you probably wont need help for that since you have a lot of experience with various types of systems.

Yea, but ever since I went on vacation 8 years, I try to give up the cmd line :rofl:

Looooots of updates since last time. Here is just the most visible one

The plot_counter looks like this now.

  • Removed plots levels with zero count
  • added MB/s to check wether your combined hdd write speed keeps up with your plot creation speed

.

Happy (re)plotting,
:seedling: efnats

1 Like

At least you are a Windows professional :joy:

This is such a cool package of scripts for farmers. Deployment at scale made easy.

2 Likes

Thank you very much, @DigitalSpaceport!

Yes I use them for my own and tweeked them a little bit for my setup.

1 Like

I’ve done some work on the plot_timer. Run it next to plot_counter to give you comprehensive info on where the bottleneck of your plotting efforts may be.

image

usage example:

plot_timer --last 60

Currently the requirement is, that you’re using plot_starter, or at least the systemd service plot-starter.service. (We need time information in the logs to do all the calculations.)

I’ve made a major update to the Chia profitability calculator cropgains

Have a look:

^^
This is me plotting and farming.

cropgains supports automatically retrieving energy readings from tasmota enabled smart plugs. Tasmota is open-source software that you can install on smart plugs, allowing you to control them directly over your home network, without depending on the manufacturer’s internet-based services. Tasmota pre flashed plugs can be found on Amazon, but lots of existing devices can be flashed quite easily.

If you dont own a tasmota smart plug, you can still manually add your power draw readings per device in the config file and cropgains will do the math.

Have questions? Do ask me!

to always green profits!
:seedling: efnats

I’m trying to use chiainit and running into an issue. I’ve used it on my 2nd chia machine to test and was able to init 24 drives. The other machine has 126 drives. I want to init 10 drives at a time and then I will replot from C8-C18 with gigahorse.

I am wanting to init and replot the first 10 drives. I have unmounted them, but when I run ./chiainit --fstype ext4 --init sda-sdl
I get Drive sda is mounted, sdb is mounted, etc.
image_2023-11-17_194955547

I unmounted the partitions and the disks are mounted…what am I doing wrong?

Hi @drewscho ,

thats indeed odd.
I have tried the same procedure on my test server here and I am not running into this issue.

Did you clone the latest repo?

Upon checking this I noticed there was a bug in chiainit where the drives that are in the excluded list were listed properly to be excluded, but then this wasnt handled. It is more aestetical thing for system disks and mounted disks since none of the commands wipe, mkfs etc are actually working on these. However, if someone would have made use of the --exclude function chiainit would actually had ignored it.

It is fixed now in the latest version. Thank you very much for pointing me into looking at this part of the code - even though it was coincidentally.

1 Like

I’ve got several jbods, some Netapps, and some EMC STL3, and the disks were showing up weird even in disks…so I ran updates and rebooted (1st time in about 2 months) I picked 15 drives, unmounted them, chiainit worked fine. I’ve got 126 drives to replot from c8 to c18 so I’ll let you know if I have any more issues! Thanks!

So, I am trying to go ahead and just relabel all the rest of the drives and still getting errors. Like I said I have several different jbods, over 100 drives. So I tried:
./chiainit --label --label-prefix CHIA sda-sddq
and get:
df: Warning: cannot read table of mounted file systems: No such file or directory
for everyone of them.

I accidentally unmounted the sys drives…rebooted and I’ve relabeled.

NOW, I would like to use the gardenmount --print-fstab but it’s not working. I’ve tried as root. ./gardenmount --print-fstab