Nah. I don’t think I did anything in particular. I use 1866 Samsung DDR3 RAM and output to an nvme buffer drive. I experience fantastic times like this. It is worth noting that I did some testing with Bladebit on Ubuntu. My times on machine with Ubuntu installed were a full 5 or 6 minutes slower than the same machine with the equal version of Debian installed. I brought it to Harold’s attention, but I don’t think he ever figured anything out. I figured it had something to do with the Ubuntu libs vs Debian libs maybe??? Idk. I know it’s a pain in the ass, but back up and try a clean install of Debian and see if your results are any better. I’ve had 1 other user approach me and confirm this on his v2 server also. It is the same on all 3 of my v2 servers. What kind of RAM and CPUs are you using? On my dual socket e5 2690 v2 & my dual e5 2697 v2 times are around 14 minutes per plot. I was experiencing about the same times with madmax and all the numa plotting stuff.
I have dual e5-2695 with 1,886 Hynix RAM. I get about 17:30 combined with MM, and over 20 min with BB. Actually, the best times I got when both tmp and dst were in RAM, and bash script did the final xfr (whether to NVMe or directly to HD).
On those v2 chips, it is possible that starting around e5-2690 the memory becomes the bottleneck, thus you don’t see much gain with your 2697.
I will try Rocky LInux first (CentOS creator) and if that will not be much better, will try Debian (I am more familiar with CentOS than Debian/Ubuntu branch).
Lol im plotting on a ryzen 5 5600 with 16 gb ram (my farming rig)
I would think you should be able to gain performance on the centos distro. I’ve heard of people using it and it being fast. I only get about a 1 minute gain on the 2697 vs the 2690 if that. Please let me know what happens with your tests.
Nothing wrong with that! I assume that’s what a lot of people do …
By the way, this is my cmd line to run BB RAM:
b-v2.0.1/bb -f fff -p ppp ramplot /mnt/nvme1/mmx/xfr
If you recall, are you using the same, or add some extra params (it is for a single run, so don’t care about the final xfr, thus let it go with all logical cores).
To be honest, I think I’m still using the older version of BB. Iirc it’s faster in RAM than the new version. I guess I was doing some specific things lol.
The latest (v2.0.1) should be just a wrapper around v1…, so no code changes. Only the Disk portion should be new /modified (and that one is really slow on my box). I will try the old version, just to make sure those two are the same.
From what Harrold wrote we can infer that if things are not logged on their github page, we can assume those are not tested at all, as the MO is to release something, and see whether it sticks, and only fix if there are blockers (e.g., chia v1.6.1 has bb v2.0.0 which has broken Disk part - thus v2.0.1 was released, but is not included with the main chia package).
I went through few different distributions (Ubuntu, Rocky Linux 9.1, Debian 11.5, CentOS 7), and tried to run both MM and BB RAM. Starting with Ubuntu, my MM plots were around 18 mins (tad less), and BB was over 20 mins. Switching to Rocky, both dropped to tad below 17 mins, so about 5% or so speed gain for MM, but a huge gain for BB. Debian was basically the same but was killing me with RDP. I used CentOS 7, as v8 was made by Red Hat to hit the dirt last year and 9 should be basically the same as Rocky Linux 9. However, that was also no fun. I did manage to compile MM, and times were about 30 sec worse than on either Rocky or Debian. I couldn’t compile BB, kind of gave up on it.
So, I can confirm your observation that BB has some issues with Ubuntu. Also, it looks like Debian and Rocky 9.x have basically the same performance (at least on my box).
Not sure what box you have, but I think that there are potentially 4 things that may keep my box from going down to ~14 mins:
- I have e5-2695 v2, and maybe yours is e5-2697 v2 that potentially could be 5-10% faster what could maybe drop my times to ~15 mins in exchange for power consumption (I am on ~$0.5 kWh in California, so maybe not worth it).
- My RAM is not as fast as yours (it is 1,866 MHz, but still may be not as good / fast as yours)
- My box is Dell t7610, maybe the mb architecture has some limitations
- There is something in the BIOS that I don’t understand, and don’t know that it needs to be checked on/off
On my box, MM runs at about 80% load, where BB at 100%. What that implies to me is that going with a faster CPUs may not help with MM (as looks like it already suffers - RAM bandwidth?) but may give a nice boost to BB.
Actually, on Debian (only, so far), the box started to throw machine check errors (either CPU, or RAM, or bus - couldn’t get mce to work properly, didn’t want to spend time on that), virtually every time I run BB. On the other hand, it happened just once while running MM. All my temps are rather low (CPU ~50 C, RAM ~60 C, so not that much)
I have also tried to compile BB v1.2.4. It had exactly the same times as v2.0.1 (as expected). However, v1.2.4 compiled basically without any compiler warnings, where v2.0.1 had a shit load of those (like 2 different engineers would be working on those). For a company that produces a shit release after a shit release, to still have a policy where compiler warnings are permitted in the checked in code, especially in the production branch just says that they really don’t give a damn about the quality of what they release or how that works for farmers. Most of those warnings were simple cast warnings that take 5 mins to fix 20 of those, but just too many to scrutinize all of those for potential shit. Knowing that v2.0.0 (production, included in chia v1.6.1) didn’t work out at all in Win (github bug, thus v2.0.1 fix shortly after), just adds another layer to that (they didn’t bother to make a single run on Win before releasing it to production).
My dual 2690 v2 are actually faster than the 2697’s. I think it might be the higher clock speed. I never really experienced any time issues with MM. Only with Bladebit. Glad to hear though that you saw a nice improvement on the BB times though! I’m not really sure what could be making it behave that way. What’s the BIOS setting that you don’t understand?
Just get an Samsung PM9A3
Also do not buy the M.2 SSDs but the U.2 ones.
You can get U.2 => M.2 Adapters for 10-20 bucks.
eBay is a good source for used ones:
Yeah, performance of a specific program (e.g. BB or MM) may differ from synthetic comparisons, so there is really no way to tell, unless running both CPUs on the same box for a given program.
I have also never had any issues with MM plotter. So far, BB (whether RAM or Disk) is either equal or much slower than MM on my two boxes, so I am kind of giving up on it.
I guess, the next step would be to get a couple of 2690. Although, potential improvement would be around 10%, but there would be about the same percentage penalty on power consumption, so kind of a wash.
My BIOS has some NUMA settings, so I tried both, and for each did one plotter run with or without NUMA, so it is a pain to run it through all those combinations. MM just runs as is and doesn’t really require that much tweaking on the BIOS side.
Why buy new for this purpose. Just think of what you could do with those drives in real life. But for the purposes of a POW blockchain? What a waste of technology. Buy 10k or 15k sas drives and be done with it. And feel better about what you’re doing for the environment at the same time.
Not sure how me buying a sas drive over an nvme is saving the planet…
Listening to all the conversations, it’s obvious that plotting is like cooking…somewhat of an art. We can all start with similar ingredients, but the results at the dinner table vary greatly, from super good to not very. So I will say>
…
“I have also never had any issues with MM plotter. But so far, MM (whether using RAM as disk, RAM as cache, or nvme) is always slower than BB on my box by a significant amount typically 20-50%”
…
So you just never know. Is it hardware, operator, or configuration differences? Likely some of all three, as there are so many variables and beyond that fine tuning is often required for best results.
Isn’t the whole point of this forum to have us be better cooks by helping each other? So, kudos to @RobbieL811 for pushing Debian and letting us know about it. That pushed at least me to try both Debian and Rocky (which I am more familiar with).
Not sure whether you have noticed, but my BB RAM results are potentially on par (or very close) to what @RobbieL811 has, but the missing part is most likely the CPUs difference (give or take RAM difference and mb / BIOS architecture).
On the other hand, my MM results are potentially same as his BB results (factoring in the CPU difference). Although, BB on my box is maxing out CPUs, where MM runs at 80-90%, what may imply that BB will benefit if CPUs are upgraded, where MM may not (as it is potentially taxing RAM more making it the bottleneck on this system).
What happens when you buy new off a store shelf? When inventory drops, they order more. Thus, they manufacture more. Last time I checked, manufacturing (and the fossil fuels to ship product to market) is not green and has nothing to do with renewable anything. Simple math. Oh well.