I have a Raspberry Pi 4 that I’m been trying to farm on, and it’s been a huge pain. It takes forever to sync, I’ve copied the blockchain from another faster computer, but even still, it loses sync and then takes a very long time to resync. Are there any recommendations on a Ubuntu compatible PC that will work well for Chia farming (not plotting)?
Thanks.
Why do you need to sync a computer that is only farming?
My understanding is that only one device should be synced, and that is your full node.
Perhaps I am mistaken?
Take a look at the exchange between Stefan-Lange and I during the last dust storm. After reducing the peer count, his RPi started to work without any problems (as it should).
Also, follow the bug link that Stefan opened, as after his RPi started to work there was an official chia person repeating those suggestions as beneficial for RPi owners. I would also want to add that potentially moving debug log to RAM drive (about 100-200 MB) may remove some pressure from the SSD holding blockchain db during those storms.
However, if you are running multiple full nodes, then what @seymour.krelborn stated is correct - you should not be doing that. You should run just one full node, and the rest of your boxes should be just harvesters. It should be perfectly fine to run full node on RPi 4, and harvesters on RPi 3.
The chia network is being dusted again. That said, my pi4 8GB 1TB SSD is coping with it. I have reduced the number of peer connections to 10 a long time ago.
I’ve ordered an hp 8300 with i7 to replace it
I don’t want to seem like I’m advertising so I’m trying to reduce the amount of times I mention it but…have you tried flexfarmer? Farming on a pi or other low spec equipment is what its made for.
Yup. Here is the chart - https://graphchia.com/Mempool.php
Which chia version are you running? Could you also check your debug.log, whether it is saturated with spendbundle garbage from time to time? I am on v.1.2.10, and see bursts of those less than millisecond apart. Maybe it is a good time to drop to INFO level for time being.
I am on the latest 1.2.11 I don’t see a lot of log activity, in fact there are some unexpected pauses with nothing logged for several seconds
There is one node in my connections that has got left behind and is stuck a hundred blocks behind now
@Chris22 I am running with 10 peers only on i5 box. The box is running more or less idle (CPU is around 10-20%, NVMe that holds blockchain is basically at 0%, all partial times are below 2 secs), but there are some stales reported on Flex page for my farmer (not much, but rather significant increase from normal (below 0.1%)). As I cannot point to anything on my box, it looks like those stales are due to the server side. Any comment?
Actually, I have checked the top 10 Flex farmers, and about 5 of them had roughly the same percentage of stales as my farmer (~0.8%), and in about the same places. That would further indicate that the problem is on the Flex side.
You should also see duplicates as there are chain re-orgs happening too (this is flexfarmer)
[2021-12-28 07:01:17] WARN worker: The Chia Network is unstable - Challenge Chain reorganization detected. Duplicate partials may occur. new=3 prev=3
[2021-12-28 07:01:17] INFO worker: New signage point ch=91c66dc96b26 elapsed=4.89542ms eligible-plots=4 index=3 space=219.49 TB
[2021-12-28 07:01:18] ERROR pool: Duplicate partial, ensure that only ONE farmer instance is running with these plots ch=64c2c7279104 diff=1 elapsed=260.346841ms error=pool error: 16 Duplicate partial plot-id=f2e3dba1e2b9 plot-path=/plots12/plots/plot-k32-2021-11-25-17-14-f2e3dba1e2b9c0b4e95e459f0ffa235397d5cd1f7260713ec3c41cbee989eed2.plot size=32
[2021-12-28 07:01:18] ERROR pool: Partial rejected ch=21cc2d36d55f diff=1 elapsed=288.228879ms error=pool error: 1 Requested signage point was reverted plot-id=11c0df38f97b plot-path=/plots11/plots/plot-k32-2021-11-19-07-45-11c0df38f97b79fc87cfa03d588903585db1c688f266364486221a31fe641004.plot size=32
grep -i bundle ~/.chia/mainnet/log/debug.log
gives nothing at all
Just do tail -f debug.log, and let it sit like that for some time. If not those spendbundle bursts, everything else is more or less as normal (farming stuff reported every few seconds). Although, if that burst happens, there are 100 or more of those lines, more or less all at the same millisecond. They should move those lines to DEBUG level
A couple of nodes being behind is normal, as it kind of represents the state of the network (e.g., 10% nodes are behind). I would not worry about those, as long as you have three / four good ones. There is no point to act on those, as who knows what node will connect. Also, I think that if a node will stall for some time (it is completely overwhelmed), it will drop at some point, and a replacement will come.
My understanding is taht those “ERROR pool: Duplicate partial” lines are coming back from the pool. I see them also, but a bit differently formatted.
That “The Chia Network is unstable” is an interesting one, though!
Maybe they moved those spendbundles to DEBUG in v1.2.11, then. The last dust storm, I was on v1.2.6, and those were killing my box. Here is what I have - notice, everything the same millisecond:
2021-12-27T23:08:41.123 full_node chia.full_node.mempool_manager: WARNING pre_validate_spendbundle took 1.2043 seconds for 7eb914fd5f6c045f024b3883f74d06347b2d1853d2f535fcfadd0960ca5798c6
2021-12-27T23:08:41.123 full_node chia.full_node.mempool_manager: WARNING pre_validate_spendbundle took 1.2043 seconds for 7e399e35ec7b3206b2b4e2d32f6edd46141b3f885229bb773348d477c6c60851
2021-12-27T23:08:41.123 full_node chia.full_node.mempool_manager: WARNING pre_validate_spendbundle took 1.2043 seconds for 6297b9601273fedd1890a2450a353194e7d16979d19fa3dc700a6bff2f10e223
2021-12-27T23:08:41.123 full_node chia.full_node.mempool_manager: WARNING pre_validate_spendbundle took 1.2043 seconds for bd02ad64eb110c645ddad640c7f33e5b5c92fbf5b6321c3052f3f095d69afd15
2021-12-27T23:08:41.123 full_node chia.full_node.mempool_manager: WARNING pre_validate_spendbundle took 1.2043 seconds for 53c1472ae97843b59386a4168afcbe9588e15fe7084fbec72684aa5af6366091
I just run grep for that Network is unstable, but don’t have that line in my log. Again, looks like v1.2.11 log line.
I just thought - I am running “chia start farmer” on the command line. Are you on the GUI version?
I am running UI, as I am on i5 box (although mobile version - 10W TDP or so - Gigabyte NUC).
Yes, this could be it:
chia/full_node/mempool_manager.py: log.debug(f"pre_validate_spendbundle took {end_time - start_time:0.4f} seconds for {spend_name}")
I have tried to upgrade to v1.2.11, but wasted half a day, and was going nowhere. Tried v1.2.10, and it was running right away. I was upgrading from v1.2.6, so there was something not so obvious that was making it impossible (I gave up, as I saw another guy mentioning the exact same thing, upgrading from the same version - on github Discussion page). I did remove config.yaml, and all dbs. Nothing was working. The painful part was that there were basically no logs at all, just spinners.
Yup, it got moved to DEBUG level in v1.2.11.
Comparing with the last dust storm, my debug log is really slow right now, even though I am not yet on v11.
How do I report bugs in this? Is it keybase?
(venv) chia@pi8:~/chia-blockchain$ chia show -c
Connections:
Type IP Ports NodeID Last Connect MiB Up|Dwn
WALLET 127.0.0.1 Dec 28 07:23:06 854.2|4.2
FARMER 127.0.0.1 Dec 07 13:00:13 47.4|0.0
FULL_NODE 88.97.46.151 8444/8444 738f078d... Nov 25 09:57:09 0.0|0.0
-SB Height: 0 -Hash: Info...
FULL_NODE 93.51.123.86 8444/8444 05fcbe67... Dec 28 07:24:25 188.9|1743.7
-SB Height: 1341304 -Hash: 4c2313bc...
(venv) chia@pi8:~/chia-blockchain$ chia show -r 738f078d
Attempting to disconnect NodeID 738f078d
NodeID 738f078d... FULL_NODE
(venv) chia@pi8:~/chia-blockchain$ chia show -c
Connections:
Type IP Ports NodeID Last Connect MiB Up|Dwn
WALLET 127.0.0.1 Dec 28 07:24:46 854.2|4.2
FARMER 127.0.0.1 Dec 07 13:00:13 47.4|0.0
FULL_NODE 88.97.46.151 8444/8444 738f078d... Nov 25 09:57:09 0.0|0.0
-SB Height: 0 -Hash: Info...
FULL_NODE 93.51.123.86 8444/8444 05fcbe67... Dec 28 07:25:08 188.9|1743.7
-SB Height: 1341306 -Hash: 9c38677f...
If you are sure that you see a bug, then you should go to github Issues - Issues · Chia-Network/chia-blockchain · GitHub
On the other hand, if you are not (yet) sure, you may post it on github Discussion - Discussions · Chia-Network/chia-blockchain · GitHub
I think more people can potentially chime in on github eventually supporting what you see, and chia guys troll those pages from time to time (especially, when the topic is getting hot). Although, I don’t know keybase at all, so maybe I am wrong.
Although, where do you see problem with that output? To me, it rather looks like your node may be potentially struggling - that connection to 88.97.46.151, as it sits at 0/0 Up/Down what rather implies that your node cannot fully proceed with a full handshake with that peer. Although, if that is just one peer with 0 height, it could be that node not being able to handshake. Just watch Up/Down MB, whether those are changing / you are in sync.
Maybe a good test is to boot that one with 0 height and check the behavior of the new one. If it will also stay at 0 (or rather everyone will stay like that, when you boot one such peer at a time) that would indicate problem on your side. Although, if needed, you can still drop the peer count down to 5 or so at least for as long as this storm lasts.
The peaks for all those last storms are the same - 12k. Maybe that indicate that this is the same kid, running the same script? If that is the case, maybe he is trying to make a point? Otherwise, I would assume that he could scale up his operation by getting a couple of more kids from his class, unless that is his bragging point at school
If you drop a bunch at the same time, you are just giving a bit more breathing room for your node, so it will look like you are helping, but that is only a temporary help until the node will reach the number it cannot handle anymore.
I would not bother disconnecting that one peer, though (it will quickly become a full-time job). If that is just one such node, and you are fully synced, we can assume that it is really behind. Still, you have 9 good peers, and you are syncing, so your node is OK. That node will drop off after some time (yeah, I have already suggested to chia that they shorten the timeout to drop nodes that stalled and add a (dynamic) timeout before a replacement node will be accepted, as that would be one way to control the number of peers in such situation - thus helping the node to cope with dust storms, or those that have less resources).
B450m+2200g+16GB RAM+500GB nvme is good for farming. 20 port sata card+7 port usb pci card is connected to this computer. 43 external+ 20 internal+ 4 PSUs 550W+ are connected. Total 63 hard drives. 515TB. Peers is limited to 8. Windows 10. No problem.
I dont know how you all get away with so little ram, unless it’s from limiting no of nodes.
I have 80 peers, and use 14.3 GiB of ram.
Wallet synchronization is not required for Farm. You can temporarily (or permanently) stop wallet sync until the storm passes. Wallet sync uses a lot of CPU.
Type resmon in windows search. Suspend start_wallet.exe.
Wallet sync is only required for transferring.
Even “light wallet beta” has been released. The full version will be released soon. You do not need to run a wallet on the farmer’s computer.
At the bottom of the page. Download - Chia Network