Minimum RAM requirements for madmax to create K34 plots -- and other stuff

Chia Plotting
21

Just a misuse of hardware.

Laptops are not intended for prolonged sustained loads. Neither thermally, nor hardware endurance. Especially battery - you will burn through its endurance very fast.

The portable SSD is not intended for sustained io load. When they advertise “high performance” they mean burst loads or sequential file transfers (e.g. backup your photo library). If your laptop doesn’t even get hot during plotting, then it means the process is I/O bound (bottlenecked by slow disk performance), so it is not capable to utilize CPU at capacity.

If you want to test overall performance “for science”, why not start with just one K32 plot? If it takes way too long, it means trying higher K’s are pointless exercise.

2 Likes
22

Intel I5 8250U. The “U” series is the low end, and the “H” series is the performance end (I believe).

My internal SSD was not being used. I was using a Samsung T5 USB drive for temp processing, and it stayed cool (and it has an activity light, so I know that it was in use – and I saw all of the .tmp files that were being crunched).

However, when I used that T5 on my AMD Ryzen 9 5950X as a temp directory, it got warm to the touch. Apparently, my laptop was not fast enough to heat up the T5.

Nothing was heating up. I checked all around the case. I do not think that I was risking damage.

Alas, twice the laptop hung after 4 to 6 hours (can’t be sure of the exact time, because I was not checking on it more than once every 1 or 2 hours). And since the screen was blank, I have no idea what error might have been there. Nothing I pressed or clicked would wake it up. I had to hold the power button for several seconds to turn it off.

I disconnected the “Live USB” flash drive, and turned on the laptop. It was like nothing ever happened (which is how it should be). Crummy old Windows started up, and all is well.

Perhaps the “Live OS” was not a good way to test the laptop?
I will fully install MX Linux on a flash drive, and boot from it, and try again.

23

It is in the 50s outside. I keep the laptop near a window that is cracked open, and cold air is constantly flowing around it.

It is designed to throttle down, accordingly. I do not feel that there is a meaningful risk. And if I feel it getting hot, or someone can tell me how I can check the thermals in Linux and I see that it is running hot, then I will discontinue the experiment.

The laptop is plugged in. The battery was not being used.

How do you know that?
I have used the T5 to create hundreds of plots on my AMD Ryzen 9 5950X. The T5 is fast, and has never throttled. It also never gets hot (just a bit warm).

That is true for mechanical USB drives, and for other SSD based USB drives (including the T7). But the T5 was designed to be hammered, sequentially and randomly. It will out-perform any consumer SSDs (including NVMe’s) that are bundled with computers.

The T5 will hold its own against the Samsung 980 Pro (it will lose, but it will stay in the race).

The T7 is designed to be much faster than the T5, but only until its cache gets consumed (which Chia will do, quickly – it then becomes a snail). But the T5 is uniformally fast.

Something was bottlenecked. But my Linux skills are minimal.
I ran “top”, and it showed that madmax was using 79% CPU cycles. But it also showed that some ntfs process was using 89% CPU cycles. So that readout confused me.

When I get MX Linux running, I will install “htop” (I hear it is very good). Or I will install something else, if anyone has a suggestion.

I will know, either way, whether it is worth the effort.
If a K32 plot completes, I will just delete (I am replacing my K32s with K34s). I would rather not create a file that I will not be using. I might as well see how it does with a K34. If it drags on forever, I will just kill it and forget the whole thing. That will probably be the case.

By the way, I have an ancient Dell laptop with a Core Duo CPU. It is slow. No, no, no. What you are thinking is wrong. It is slower. :slight_smile:
I use it to backup my iPhone (I do not back-up to the “cloud”).

I got it for free from my cousin, when he could no longer deal with its limping, and he purchased a new laptop. Whatever possessed him (his wife) to purchase that slug is a mystery (his wife). :wink:

It has only 2 or 4 GB of memory (I do not recall right now). Even if it supports more, I will not spend a nickel on it. But if it had enough memory, I would have it create a K34. It would probably take 3 -6 months. Now that is a textbook case for misuse of hardware. In fact, booting up that laptop is a misuse of hardware. :wink:

24

I believe the battery is used even whenever laptop is plugged in, similar to a double-conversion UPS. An easy way to tell whether or not this is true is to remove the battery from the laptop and plug it in, and try to use it. For example, Dell Latitude laptops work this way.

Well, you can always move this plot to the permanent location on your main farm. But the point I wanted to make is different. Just to be able to judge how capable your laptop is, you can try a K32, just because you know it is a low level of effort. To be able to conclude anything, you need to measure it against something. For example, plug that T5 to your Ryzen 5950x and make a K32. Measure time. Suppose it is 40 minutes. Then plug in that T5 to the laptop and run a similar K32 with the same number of buckets, but adjusting for number of CPU cores. Suppose it is 8 hours. Knowing this information, you will be able to decide if it is even worth trying higher K’s.

Also you don’t need chia software to be able to create plots. You can install MadMax standalone and use it on Windows, or you can install WSL (it is a Linux VM made by Microsoft) and compile MadMax from source code there. I tested plotting time on Windows and it is about 15% slower than on Linux WSL (same system, same hardware).
The best performance would be bare metal Linux, but you really need to have commanding understanding of system tuning to make the most of it. For those who have little Linux experience, WSL is a great way to start. Another benefit is that if your T5 is formatted in NTFS, it will automatically mount to the WSL VM and will have optimal performance.

https://ubuntu.com/wsl

Note you should use WSL v1, if you want to go that route. Do not use v2 (it is slow for plotting use case)

1 Like
25

I am installing nothing on my Windows boot drive. That installation is remaining as clean as I can keep it. I use it for a music server, and nothing else. I never even opened a web browser on it.

Since my laptop will be struggling with plotting, I will not give it the overhead of running Windows with a virtual OS (WSL). Plus, I doubt that Microsoft is able to have their OS run Linux with zero bugs.

If I run into an issue, WSL can complicate finding the culprit (is it a madmax issue, or a Linux issue, or a Windows issue?).

I just installed MX Linux on a 250 GB Micron M.2 drive, that I have encased in an M.2 reader that plugs into a USB port. It seems to be running smoothly. That 250 GB Micron M.2 drive is the one that came with my laptop. But I needed 500 GB, and replaced it soon after I purchased the laptop. So I had the spare 250 GB M.2 drive handy. It is not NVMe. It is SATA, and more than fast enough to boot MX Linux, even via the USB connection.

The Samsung T5 will be taking on the brunt of the madmax temp processing.

I will be installing madmax soon, crossing my fingers, and hopefully all will go well. I also hope to see better performance than I got when I used the Live Pop OS environment. It was probably not a fair test for Pop OS, not having it actually installed.

1 Like
26

madmax processing, via MX Linux, is humming.
I estimate that it will take 30 hours to complete a K34 plot.

The laptop is getting a workout, because its fan is going, and it is rather warm. I took a step to help it stay cool.

I do not think that I will create more than one plot. The heat being generated from the laptop suggests that the amount of energy it is consuming will cost more than it is worth to create the plot.

But I will let this one finish, just to scratch my curiosity itch. And as a bonus, I now have a fully fledged

By the way, can someone please explain to me why htop is reporting CPU percentages that far exceed 100%?

Also, since my Samsung T5 is formatted as NTFS, is that “/sbin/mount.ntfs” process consuming a lot of CPU cycles in order to work with that drive? If so, my overall time might be significantly improved if it were an EXT4 formatted drive?

Also, how do I check the thermals on my laptop? Is there a standard Linux command or application that I should have or should install?

Lastly, I apologize for the distracting background in the photo. I have not yet figured out how to have a plain background, nor have I figured out how to change the default transparency settings for windows.

Linux computer. :slight_smile:

htop
htop1600×1200 275 KB

27

https://www.deonsworld.co.za/2012/12/20/understanding-and-using-htop-monitor-system-resources/

https://ostechnix.com/view-cpu-temperature-linux/

As your 4 cpu cores are kept pretty well busy I don’t think a faster diskformat like ext4 of xfs will impact overall running time much.

1 Like
28

I had the same issue when I wanted to see temperatures on my Linux plotting workstation, I eventually after trying a few different programs used a program called sensors, it’s CLI only.

I’m using Linux Mint, Sensors doesn’t show all my cores, but I do have 36 of them.

2 Likes
29

Yes, the 4 CPU cores are being kept busy. But it looks like “/sbin/mount.ntfs” is one of the processes that is keeping them busy. If those CPU cycles were freed up and were available for madmax, I would have a better plotting time.

30

I remember something about saying
“Yes that should have performance issue.
In windows there is a kernel-level ntfs driver but on linux it’s a user-space driver thats always slower.
On Linux you might consider formatting your T5 as XFS or EXT4 for better speed.”

But quit a lot of the CPU usage you now see at /sbin/mount.ntfs will be spend in the EXT4 driver then, more efficiently though. So … oh I already said that.

31

I assumed that the ntfs driver was a hog, and the ext4 driver would barely show up. Why?
In Windows, taskmgr does not report driver usage that registers near the top (unless it is called something else).

So my assumption was that a native file system would not be resource hungry, at all, but ntfs under Linux is. Ergo, using ext4 for my temp drive, I thought, would use minimal resources, and not make podium presence on htop. But you think that ext4 would also consume a fair amount of CPU cycles, albeit less than ntfs?

32

Yes, that’s my assumption.
I’ll start a plot and look for ‘evidence’ on my linux box…

htop does not show any cpu activity for xfs being the file system on my nvme.
I think that’s because the driver is in kernel-space. However there must be clock cycles involved in assigning tracks,sectors etc from disk to write and read commands.

1 Like
33

My MX Linux laptop hung during:
Phase 3-2 Table 7.

I know it is hung, because the mouse does not work, my Samsung T5’s activity light stopped blinking, and my desktop’s clock stopped updating 63 minutes ago.

I am going to try this, again.
I am going to format my T5 with an ext4 partition, and direct the temp work there.

I am a glutton for for punishment. I am also determined to complete a K34 plot.

1 Like
34

I struggled with gparted (never used it before). But I managed to allocate my T5’s free space (1.9 TB was free) from NTFS to unallocated, and then to ext4 (and I did so without destroying anything :slight_smile: )

My laptop is once again hammering away at a K34 plot. I think that it will go much faster, due to the ext4 partition. Why?

My T5 heated up, more so than when it was used as NTFS. Also, no “/sbin/mount.ntfs” processes is there (unless it is hiding at the bottom of htop – my final drive for the plot is NTFS, so eventually that will come into play).

Also, the madmax process is consuming more CPU cycles. But it is early. It is on the first step on phase 1. I have a better feeling about it succeeding this time.

1 Like
35

@seymour.krelborn
How far along are you?

36

It hung, again, during phase 4. :frowning:

I do not know how far into phase 4, because the screen was blank, and neither the mouse nor keyboard would wake it up. But I know it was phase 4, because it was there on my last visual check a few hours earlier. And I knew it was hung because the laptop cooled off and no activity on my T5.

Perhaps 20 GB is not enough RAM?

But whatever the problem is, it should not hang the OS. If it cannot handle a K34 process, then it should crash or exit the application. The OS should be robust enough to withstand demands from the applications.

I was going to try Linux Mint. But the installer warned that it would wipe “the” disk. But which one? The installer is not clear regarding which disk it is going to wipe. My Windows M.2 drive, or my USB flash drive? (I want the Linux OS to install to and to boot from the flash drive, and leave my M.2 drive untouched).

For what is touted as a beginner friendly Linux OS, its installer is not beginner friendly.
With Pop OS, and with MX Linux, I had no partitioning problems. But Linux Mint is not clear when dealing with partitions with data. I even deleted the flash drive’s partitions, leaving 100% unallocated space. Still, no luck pointing Linux Mint to it.

I am sure it is just a matter of choosing the right clicks. But the descriptions are vague. It should not simply read: All data will be lost. It should state the disk to which it is referring. I was not going to cross my fingers.

I considered removing my M.2 stick (containing my Windows OS), but there are a lot of screws, and I am not in the mood to remove them all, pry open the case, disconnect the drive, install Linux Mint to a USB drive, reconnect the M.2 drive, close the case, and populate the many holes with the screws.

I am not ready to throw in the towel.
Maybe I will try a plain vanilla Debian installation or Ubuntu installation?

Or might I have better luck with Redhat?

37

Hello,

I only plotting K34 and some K33 to fill all left space under ubuntu server.
The RAM amount u need depents on how many CPU threads are used for plotting. My experience shows that u need about 2.5 GB - 2.8 GB per thread at peak for K34. This peak occurs in phase 3 at table 6 & 7.
If u only plott one K34 u normaly need 2.8GB but if use constant plotting, after 1-3 Plotts the used RAM is reduced to 2.5 GB per tread.

I hope this was helpfull

1 Like
38

My laptop has an I5 with 4 cores and 20 GB of RAM. So I am guessing that it has 8 threads? Based on your figures, a K34 is going to consume my RAM.

madmax has a “-r” option to specify the number of threads. I have not used it. It defaults to “4”. If that is the case, then it should be using slightly more than ½ of my RAM.

Maybe I should give MX Linux another shot, and specify “-r 3” or “-r 2”?
MX Linux seemed to be nicely configured, which is why maybe I should try, again with one of the above -r values.

I have never run Ubuntu. So a vanilla Ubuntu distro is also an option. But I suspect that any Debian based system will have similar RAM consumption. I will lower the madmax thread usage via -r and see what happens with MX Linux.

39

windows OS is not the best OS if u try to plott K34 with low RAM, because it uses self a lot of RAM.
but 5-6 threads should work with 20GB.
i had sucsessfull plotted K34 with 4 threads unter win10 with only 16GB (at my starting days with chia :smiley: ) till i build a seperat plotter (it takes about 15h per plot).
but keep in mind u also need more free ssd/nvme space for K34, normaly a little bit more than 1TB.

40

Only way to really know is to look up your CPU on the Intel site
Intel® Core™ i5 Processors