My journey on the "use till the last bit" thing

General Chia Discussion
1

You can safely ignore this thread, since it’s just my complain on technology and how people trying to be smart …

Apparently it was a great article, BUT …
I HATE the overhead of MS storage pool, it’s slow and used almost 30GB just for the 1440GB pool, theoretically i just lost a plot to the storage overhead, so i started my journey to find a better way …

  • At first, i thought a .vhd file mounted as a physical disk, which could be converted into a dynamic disk, then you can just make a spanned volume across the vhd files to make a storage pool, but then i realized that Microsoft outsmarted me by prevent the vhd file from mounted as dynamic disk, you can try to use another tools to make the vhd file a dynamic disk, but then it will failed to mount using windows way.
  • The linux way would be easy since you can just make loop devices from the excess space, then merge them using either mdadm/zfs, but i had to shake it off because all plot drives are NTFS and there’s no easy to convert them into linux format, and NTFS reading from linux is such a pain to your CPU.
  • I looked for the 3rd party solution, but there’s pretty much … none. Some software like ImDisk only emulate a partition, not a disk, not a spanned volume. Some others can actually emulate a real harddrive but have their own limitations :
    • OSFMount can emulate 9 physical disk and no more, guess some people in their company is very smart and decided to support 9 instead of 999 drives just because they can. Theoretically the driver code don’t care if you mount 1 or 100 disks.
    • TotalMount has the same limitation.
    • DAEMON Tools is not free, but their demo version can’t mount dynamic vhd
    • ArsenalImageMounter can mount 1 fully functional vhd, the rest are readonly.
    • A lot of crappy tools.
    • A lot of untested commercial tool.
    • A light shown up : winspd can mount as much vhd as you want, but quickly fades away : their user mode code ate a lot of cpu and are quite slow, <10MB/s.
    • Another bigger light shown up, a tool that microsoft made since they introduced the VSS : the vstorcontrol/virtualstoragevss that is available on every WindowsSDK, but then also fades away - Microsoft outsmarted me again by limit the volume size at 512MB.

In the end i still stuck with stupid storage pool that had became readonly because i accidentally made the virtual volume size bigger than the pool size and there is no way to shrink it because Microsoft outsmarted the whole world by decide that disk shrinking is not necessary.

There are some 3-way intersections on my journey, like using linux/windows VM to handle the excess space, but they’re all stupid idea …

2

You should be able to shrink the volume using Disk Manager (right click volume and choose Shrink). Keep in mind that it can only shrink the volume using free space at the end of the drive so you must defragment the volume first to be sure that all the files have been moved to the beginning of the drive. Secondly, I’ve found that I’ve had to run the Shrink command a few times sometimes to get down to the size that I know it should be. For example, I had created a 16tb volume for a storage pool that only had 4tb. I had to run Shrink on it several times before it was able to shrink down to the actual size.

I’m not sure where that “overhead” is coming from - I don’t see nearly that amount of overhead in my storage pool. Are you using “Simple” resiliency instead of “Mirror” or “Parity?” You don’t need mirror or parity for Chia plots and those options will slow down access by a bit over “Simple” and they also add overhead/duplication.

3

Of course you can “shrink” a volume, but not a virtual disk created by the storage pool. It’s ridiculous when you can create a “thin” disk that is bigger than the pool, but once created you can’t shrink it, and once the disk usage reached the pool capacity, you should see something like This disk is offline because it is out of capacity and you can’t even turn if online with DiskMgmt. Only diskpart can make it online with readonly flag.

1 Like