I’m opening a thread to gather statistics around the forum users for the new bladebit disk plotting software coming out soon, and which is currently in unreleased alpha state. Some people at Keybase are starting to compile and run it on their linux systems.
At the time of writing, the performance is on par with Madmax, and it’s expected to surpass it on alpha/beta releases. It’s unclear to me if the ssd writes will be lower than with madmax using ramdisk. One of the stated goal is to speed-up plotting on higher core systems. The ram usage should come down too.
Feel free to share your results here if you’re already trying bb disk.
I’ll update this post when more information comes out.
In the context of k32, there is a benefit of Bladebit v2 that many aren’t aware of: it no longer drops entries over 2^32.
Madmax and Bladebit v1 drop these entries to improve performance, but Bladebit v2 is able to achieve similar or better performance without dropping these.
This means that Bladebit v2 k32 plots are now equivalent to plots produced by the original chiapos plotter.
The final BBv2 k32 plot sizes will vary slightly more than you’re used to with MM and BBv1, and a set of plots on a disk will take up slightly more space overall. More space taken = more proofs.
So even if BBv2 alpha is performing the same or worse for you, you’re still getting slightly better k32 plots with it. It’s certainly not worth replotting MM and BBv1 plots as you’re getting the same amount of proofs per byte, there is no additional compression (that’s coming next year).
How is any of this meaningful and/or important? Or something most even care about. Most just want something that works better in some significant way. End of story. Don’t want to be too, too, critical here, but help me out please. Specifically…
I don’t understand. Any new version plotter should be at least producing plots equivalent to the OG plotter, or they will not be valid, yes?
Taking up more space? … that is not a positive, is it? I want them to take less space, i.e., be compressed, if anything.
Just how is a BB2 plot better? Are these plots better because they are bigger? I thought that what k33, k34, etc plots were for?
Everthing mentioned in the graphic seems only of any interest to programmers, except ‘low RAM’ perhaps.
Something far more useful would be if this new plotter did k33, k34 plots. Currently all this kerfuffle about BB2 seems to be just reinventing the OG wheel, or trying to match MM performance. So called benefits anyone can muster seem fairly vague, at least at this point. Even faster plotting of k32 isn’t needed much a year post mainnet and when plotting is reduced greatly for obvious reasons.
Since you’ve been using this plotter in development, what is actually anything relevant to the general Chia farmer that is better in some significant way? Thanks, and again, I’m all for this project, just looking for a ‘why’.
so the nice thing about no dropped entries is an easy validation of plots created, since the same memo will produce the same plot byte for byte and can be easily checked with a hash. The bad news, the performance hit was a bit larger than we expected. Actually looking at going back to the dropped entires for k=32. For higher k values this won’t matter since it won’t fit into 32 bit anyways.
goal is to be faster on all systems. On high-end systems BB2 disk is beating madMAx by 2x. On low end systems with the right settings we are seeing a few % improvements. Trying to make that meaningful enough on everyone’s systems. The disk writes and WAF are absolutely lower and those have been measured. Cache is still a little busted in regards to performance, there are some changes coming there too.
Sure, that is all good in the lab to do purity test, but in the field everyone is using MM.
Actually, this is how Chia started bashing MM almost to oblivion as their asses were handed back to them by MM, and apparently all that was a nonsense. So, are we starting the same process again?
Could you be more specific about that, please.
I would say that this should be the primary goal. Talking about “The disk writes and WAF” and not mentioning the RAM looks like a mistake to me.
I don’t follow. Chia has good relationship with madMAx chia plotter, as we use it in the official Chia CLI and GUI.
We are posting some early results in this spreadsheet here
I have a 7 minute disk plot on Ice Lake Server (64 core) and 10 min disk plot on Cascade Lake Server (32 core). Harold is seeing similar 10 min time on a 32 core Threadripper pro with single NVMe.
Not from the very beginning, though. I think that we can still go back to MM github page, and try to dig those statements out, if needed. I was also reading those statements and rebuttals, and was hesitating to use MM initially.
Is there any reason that someone would not want to get enough RAM for k32 plots to do all plotting in RAM, if they spent about $5k just for two CPUs on the first system (Xeon 6338)? Actually, the last system (Xeon E5) is the same, it is using DDR3, so the cost of ram would be even smaller. Also, I don’t see k-values mentioned in that table, so all of those are for k32? Also, one thing that is not being mentioned, and maybe is relevant to Chia green approach is that the more RAM is used, potentially the less power per plot is needed.
I assume, the only reason to use drives would be to use BB for plots with k > 32. And as @Ronski mentioned, BB should use all the memory available on that box. Isn’t this the goal with BB-disk (to use whatever memory is on the system, not just a fixed amount)?
Assuming that this is for k32, how that would compare to the original BB running completely from RAM?
Actually, I have more questions. Would it be possible for Chia to consider deemphasizing OG plots in the UI? There is quite a bit of newcomers that start with those plots, as the UI defaults (at least up to v1.3.3) to those. It would also be nice, if plot UI would start with the best settings for a given plotter / system (e.g., using max number of threads, …). Finally, not sure whether you have stats for that, but are there people (except of newcomers) that are using Chia plotter? Maybe to make it easier for people MM should be the default one?
I know you know what you are talking about…but the fun part of the above paragraph is that I can confidently say I have virtually no (well maybe only a tiny bit ) understanding of a single phrase there
Capital idea - the OG plotter has one playing a game of whack-a-mole with the resources available, disk, memory, cpu, and provides lots of self-punishment in the process and only begrudgingly provides reward. It needs to be de-emphasized if at all possible, lest it discourage even newcomers.
chiapos reference plotting - produces good plots of any k value, slow
bladebit memory - produces fastest and most efficient plots, k=32 only and requires a lot of memory (416GiB)
madMAx - used by majority of farmers, good performance and features with temp storage (including offload to ramdisk). Does not scale beyond 8-12 cores, need to run parallel instances to get maximum system output.
bladebit disk - goal to be fastest disk-based plotting with tons of features and options to perform well across OS and system types
Is that a positive or negative? To me, it looks like inventing issues that are at most minor and not really worth mentioning. Also, with multi processor systems, using NUMA and parallel instances can have advantage over a single process that reaches across processors / drives.
Also, in your writings, you are switching between threads and cores, where from S/W point of view that is rather irrelevant (all are threads, and have no relationship with CPU architecture). Maybe it would be better to use phys cores instead.
I think that this statement is at least an understatement or rather misleading. For those setups below 512 MB RAM, MM so far is the gold standard and performance king yet to be dethroned.
On the other hand, for those setups with more RAM BB is the king.
MM and BB are best in the space they address, so there is no point to emphasize one more than. the other.
I would assume that those are the goals for any software that is written. However, not always what is the intention of a given dev translates to what is the final product (e.g., Chia’s plotter). Wouldn’t you say that Max (MM owner) didn’t think like that? One route he outlined was to move part of the plotter to GPUs, and that is potentially missing (at least so far) with BB.
Again, some concretes would be nice to have.
Actually, there is one more thing that you may want to address. Here is the official Chia page with percentage of k-value plots out there (lower right) - Grafana.
Over 97% are k-32 plots. From what has been written on this forum, some / most of the people that plot anything else than k32 are under the impression that:
k33+ plots offer more proofs / GB
they are afraid that Chia will one day pull the plug on k32 plots, and they will not be able to replot fast enough.
The logic behind using less ram, is that you can use less ram per unit of output. You can, of course, upgrade your ram and plot more, even switch to only ram-plotting.
I think that the point was a bit different. It was not really about using more or less RAM, but rather be able to use whatever RAM is available, not to stick to some fixed sizes we have right now (e.g., MM 128, BB 512). At least, I read that comment this way.
When I started using MM, I only had 32GB RAM in that box. With that amount of RAM, I tried using PrimoCache to put it in front of -2 folder, and that already made a big difference. That experiment made me upgrade RAM to 128 GB. Although, some systems can only hold 64 GB. This is even more obvious for BB, as it needs 512 GB. So having a bit more flexible code would benefit potentially majority of smaller farmers.
Although, (IMO) BB target was basically bigger farms, so it was natural to stick with that requirements. I guess, the main reason to go with BB-disk is to have a second source for smaller farms (that MM owns right now, and looks like he is not that interested in improving - focusing on his blockchain project).
Actually, one feature that actually a lot of people that don’t have dedicated plotters asked is to be able to specify “below normal” priority. Even if a plotter will always run with that priority, for those that have a dedicated plotter nothing would change (no competing processes), but that would be a big difference for the other group.
I suppose it depends how you take the statement “The ram usage should come down too.”, better efficiency or just use less and use storage more. By what your saying its the former with the option to use as little or as much ram as you have.
I have a dedicated T7910 with 512GB of ram for plotting, so I’m hoping when plotting a K33 or above it will use as much ram as possible, and then use storage when it doesn’t have sufficient ram, rather like we used to do with Primocache on Windows, but intelligently use the ram to the max.
I just want to post an update to my earlier criticisms of this plotter (in this thread) to take virtually everything negative I said about BB Disk plotting back … now I’m familiar with BB Disk v2.0.0 Beta 1.
Why? Because I’ve been able to produce the last 100TB or so of K32 plots in record time… compared to even the excellent MM plotter … and certainly light years faster compared to the lame OG plotter. And I’m using Windows, for those that think Windows is no good.
How good is good? Best I’ve done on 16c/32t 108GB cache w/one plotting SSD is 14 min. Most are more like 14.2-14.6 min for whatever reason. I save them off to a large 4-6TB ssd before sending them off to HDs. If I do the copy to HD whilst plotting, that seems to add a min or two to plotting. Big deal.
Before BB, and using MM, best I could ever do was 21 minutes, but more often 22-25 minutes, even using two separate SSDs for t1 & t2… or a RAMcache. I’ve tried all the tricks I could manage.
In any event, BB Disk is freaking wonderful the amount of time saved over prior plotting is phenomenal. If only I could have the month or more 24/7 of my life (sort of) wasted plotting earlier without BB Disk!
The final version, sometime, should be even better … maybe k33, k34s? or maybe faster yet?
part of the above paragraph is that I can confidently say I have virtually no (well maybe only a tiny bit 
) understanding of a single phrase there 
(20 chrs)
… now I’m familiar with BB Disk v2.0.0 Beta 1.
the amount of time saved over prior plotting is phenomenal. If only I could have the month or more 24/7 of my life (sort of) wasted plotting earlier without BB Disk!