Arc plot gpu plotting 0.6 release, please help test and send feedback

use is almost same as MM, for me tested only CPU ploting 5900x was 19min some 15% faster then MM

1 Like

in tar file only one file how to use that on ubuntu ?!

Exciting to see this.

It would be helpful if you could provide a few examples on how to run it. Testing on a couple of different plotters and took me a minute to notice that GPU was off by default.

Plot time on 11900K with 110GB local ram disk for phase 1 went from 700 seconds to 200 seconds once I enabled the GPU (RTX3080).

I see there is a -R option and wondering how that might differ from using a disk for the temp directories.

Great to see innovations like this.
Couple things. Call these observations or feedback.

First, fix spelling mistake on GitHub. Above the fold spelling mistake makes the project look bush.

Requirements don’t make sense. A 4-core CPU, great! But 128GB RAM? Do tell where you will find old boards with greater than 32GB max capacity. 20 or 30 Series Nvidia GPU? Too back you can’t make use of older GPUs or AMD.

I’m confused about the requirements. If this is meant for consumers, do you not see 128GB RAM as a barrier? Who wants invest that much into this and says to me you aren’t using it unless you’re running a server board.

So to me the requirements don’t match the claim of “focus on consumer hardware”. If you showed this running on Z97 boards or newer with 16GB to 32GB RAM and say, older GPUs that meet the 8GB VRAM requirements, then maybe. If all your data is based on 128GB RAM? I guess I’ll wait this out for some progression.

Not trying to dump on it and I’m sure it’s a project done in your spare time and not as some money making venture at all. GPU plotting in its infancy and I should appreciate that fact. I hope at some point it’s a serious consideration. I just think the goal with anything related to Chia should be about how to reuse old computer gear that needs some “use case” scenario. Maybe a bit slow, slightly less power efficient, but if fully usable for the sake of a crypto POW blockchain. We shouldn’t be plucking new anything off store shelves for the sake of Chia farming.

If GPUs can simplify the plotting? It will be very interesting. Might taint some of the green aspect of Chia though.

-R is used for use 110G ram plotting without setup -2 ramdisk. and all data go through ram directly bypassing file system api, so it’s will be little bit efficient vs ramdisk , but not too much. and save you from create ramdisk.

thanks you for such great feedback.
for hardware requirement:
a.) 4 core cpu for gpu plotting is b/c all GPU can max out it’s performance with 4 thread. when 1.0 release , all load can shift into GPU , so 4-6 core cpu /per GPU should able to fully utilize one GPU’s raw power.
b.) for 128G ram, this is little bit complicated answer. first of all, we maybe all know that when plotting , there are total like 1.3T data need been processed. when speed of phase1 plotting reduced from eg 600sec to 200sec, the bandwidth need for IO is 1.3T /200 sec = 6.5G/sec. if only use 32G ram with SSD, lots of bandwidth pressure will dump into SSD, and there are no consumer level SSD can provide such sustainable IO even use 2 SSD raid0.
c.) for 8G GPU with rtx 20serious plus. this is b/c 10 serious nvidia ( cuda compatibility 6.2 ) have no atomicCAS( uint16_t) support, if go atomicCAS( uint32_t) it will blow the shared memory ( which is 48K). only solution is create a new algorithm to make it happen. ( I 'll try to solve that after 1.0 full gpu function release).
d.) no amd or intel gpu support yet, that’s on todo list, I have to build function one by one. but it also on todo list.
e.) please forgive me , it is a bush project so far :grin:. but I’ll try to make it better and better with such limited resource.

3 Likes

get help:
./arc_plot --help

run:
cpu: 32G + SSD
./arc_plot -r cpu_core_num -n num_of_plots -u 256 -v 256 -t your_ssd_path -d your_distination_path -c your_contract -f your_farm_key

cpu:110G
./arc_plot -R cpu_core_num -n num_of_plots -u 256 -v 256 -t your_ssd_path -d your_distination_path -c your_contract -f your_farm_key

cpu+gpu: 110G
./arc_plot -G -r cpu_core_num -n num_of_plots -u 256 -v 256 -t your_ssd_path -d your_distination_path -c your_contract -f your_farm_key

1 Like

No, I don’t consider this bush at all. It’s just that typos can make something look lesser than it is. Just like a tucked in shirt for example showing up to a business meeting.

Thank you for the explanation in terms of technical reasons for the requirements.

It’s impressive to bring something out first. There is a difference between working on something behind closed doors but you’ve put it out there. It’s a big deal. GPU plotting? It’s very interesting for sure. I’m all ears. I’m not buying a new GPU though and sure I wish a power hungry 1080ti could do the trick rather than buying more hardware to plot with.

I love computers and tech for the very reason you can’t predict how people will make use of it or what they might decide to create with it. This makes Chia very interesting. Community innovations that Chia themselves endorse!

will you be releasing source code at some point in time, or will this remain a closed source deal?

Will be opensource after I done all design and implementation. now I have no time to cleanup the code and the design still drifting too much.

1 Like

Requirements make perfect sense and will generally be universal. 1000 series should work I think but likely would be a lot less efficient.

Maybe you could try it on either Rocky Linux / CentOS or Debian. A lot of people are reporting that Ubuntu is the slowest for both MM and BB (for about 10-20% or so).

Also, there is a bit of plotting activities for MMX project. Maybe you could add support for it (really small code changes - 20 lines or so).

if linux kernel is gcc 11 based, arc_plot should be good to run on all of them. lol

I have tried to run it on Rocky Linux 9.1, but it immediately barfs. Would be nice if you could recompile for RHEL branch and have both for download.

could you send me the err msg?

Here you go:

[jacek@i9 arc]$ ./arc_plot --help
./arc_plot: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by ./arc_plot)

...

[jacek@i9 arc]$ sudo find / -name libstdc++.so*
/usr/lib/gcc/x86_64-redhat-linux/11/32/libstdc++.so
/usr/lib/gcc/x86_64-redhat-linux/11/libstdc++.so
/usr/lib64/libstdc++.so.6
/usr/lib64/libstdc++.so.6.0.29
/usr/share/gdb/auto-load/usr/lib64/__pycache__/libstdc++.so.6.0.29-gdb.cpython-39.opt-1.pyc
/usr/share/gdb/auto-load/usr/lib64/__pycache__/libstdc++.so.6.0.29-gdb.cpython-39.pyc
/usr/share/gdb/auto-load/usr/lib64/libstdc++.so.6.0.29-gdb.py

[jacek@i9 arc]$ sudo strings /usr/lib64/libstdc++.so.6 | grep GLIBC
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_3.4.26
GLIBCXX_3.4.27
GLIBCXX_3.4.28
GLIBCXX_3.4.29
GLIBC_2.2.5
GLIBC_2.3
GLIBC_2.14
GLIBC_2.6
GLIBC_2.33
GLIBC_2.4
GLIBC_2.18
GLIBC_2.16
GLIBC_2.32
GLIBC_2.17
GLIBC_2.3.2
GLIBC_2.34
GLIBCXX_DEBUG_MESSAGE_LENGTH

By the way, this is on Rocky Linux 9.1 (CentOS replacement). Fedora is kind of Ubuntu on Debian side, so it has newer libs. Rocky / CentOS are more for long-term, so unless really massaged, it may be hard to get the latest libs installed. (Just run yum update, and all is up to date.)

The dev has already have two issues open for statically linking libraries.
He closed one, so fingers crossed he just injects the libraries into the binary so it works on more systems.

1 Like

Yeah, both Ubuntu and Fedora always have the latest libs. Those that are running either Debian or CentOS are kind of SOL. I don’t think that asking people to switch to the latest Ubuntu is going to work. I am on the CentOS side, so would have to be really hard pressed to switch to Fedora; however, would not consider switching to either Debian or Ubuntu side.

This plotter is still in a pre-production phase, so using libs that are more common should rather be a norm. I would not mind running better instrumented builds, rather than trying to have a gain of few seconds and going through extra motions to make it run. Of course, statically linked libs would solve this as well.

This is not correct, arch / gentoo have the latest libs.
This is compiled against a ancient version of gcc that only rolling releases still have in their repos.

There is literally no downside to statically link libraries for releases.
If someone is on void or some real weirdo distro, they can set their library path when launching the app and compile against the version it needs.

release 0.6.2 just published. your issue should be solved. enjoy. :grin: