Syncing taking ages

Hello,
I made a “Chia” farm with my Raspberry pi 4.One week after I started the Pi up it isnt synced I even went to my router and opend port 8444 and still its taking ages.

Please help me,Thx

To sync right now requires download of 30GB for the blockchain, 7GB for the wallet, plus processing time on your end to digest/store it. Is your Pi up to doing that at the speed you require now? … and going forward?

If yes, wait for sync, it takes some time. If not, upgrade and you can speed up the process.

How did you make your setup? Are you running your OS from a SD card or rather from a SSD?

Downloading 30GB is more or less your ISP factor (say few hours the most), not really your RPi write speed problem. However, saving that db by Chia is the problem. Virtually, every other fork provides db downloads to get people up to speed faster. It would be nice to have Chia doing the same.

Also, assuming that you have 100Mbps downstream speed, and 80 “nice” peers (default value), those peers would need to update you with about 1Mbps speed each (most likely below their upstream limits) to saturate your downstream bandwidth (if this process can be partitioned, or is partitioned). Assuming that this is an ideal case, it gives you under an hour download time (if my math is correct). I would think that you could write 30GB to SD card in about that time.

Thx for your Replies i had a extremly slow SD card with only 24 Gb available. Now i am trying to get it running on a 1TB HDD hopefully it will work.

when it comes to my internet I actually have a stable connection for the RPi.

You can forget your HD route. It will just give you a headache down the road (in addition of what you get right now). I am running i5 NUC with an average SSD, and that was still not cutting. I have just added NVMe, and can see huge difference.

Update:
Take a look at this page:

If I do understand it right (cannot say that I do), you may be able to get 90MB/s with V90 type card, what is about what you get using USB HD. Although, I don’t know whether RPi supports that type / speed. If that would be an option, it is a better one than that extra HD.

Thank you for your Reply!
I am using now a Nvme storage device for booting but it didnt work out. I reinstalled the programm wich turns out was a bad idea


I am a bit relived that it works now better thanks for you guys but in the other hand I have to wait a loooooooooooooooooooong time

Edit Nr.2: I tried to calculate how long it should take pro 24 Hours ≈ 317.000 that means it should take 4 Days to finish if nothing goes wrong or goes slower etc.

1 Like

Great that you are on your way now. Hopefully, it will not slow down, but rather speed up.

The next time you will be doing changes, just make sure that you save your ~/.chia/mainnet/db/blockchain_v1_mainnet.sqlite file. That is the one that during the syncing process you are building. It is around 30GB (when done), and can be moved between boxes / installations to jump start / bypass the syncing process. It is good to keep a backup copy of it around.

1 Like

You will sync the blockchain faster on your regular computer, then you can transfer the DB file to your RPI there. You will still need to sync the wallet (you can’t backup the wallet DB) but it will be faster. I believe nowadays you can sync the db in 1 day.

You can also try to find a db file and download it straight from the internet, but I wouldn’t recommend it.

3 Likes

Thank you very much for your suggestion!
I am going to try that, hopefully nothing goes wrong.

1 Like

There are two reasons to potentially not do it. Blockchain db could be either corrupted or poisoned.

Therefore, downloading should be limited to only trusted sites, and IMO that would only by Chia.net. The main problem with downloading is that db could be poisoned (causing harm not just to the person that downloaded it, but to the whole network).

On the other hand, when you get it from your friends, most likely their db will be clean, although it may be corrupted (less likely, as they have their systems running).

So, it is a good route, but one needs to be careful.

1 Like

You don’t have to make the Pi run on HDD. You can bind only the database folders (blockchain and wallet) on HDD. In fact, I would recommend you to run the database on HDD even if you get a TB microSD card on the Pi. The bandwidth of the card reader on the Pi is very limited (around 40 Mbps), so you should offload the syncing to one of your farming HDDs.

You can use the mount --bind command line to bind your local folder to the folder on your HDD, for example:

sudo mount --bind /card/farming/01 /hdd/farming/01

After the binding, right-click > properties in the card’s database folder to check whether it uses the HDD space now.

I fortunately already had installed and synced Chia on my normal PC and I tried it out and it
works! Its still a bit slow going but I will just leave it overnight and it should be finished.
Thank you all very much for all your suggestions.

Take a look at that thread:

OP has a nice link that shows how to point your existing setup via config.yaml changes to use both blockchain and wallet dbs in different than standard locations. I followed that link, and it worked like a charm. No need to touch anything on the OS level.

mount --bind is just one of the mount options. You don’t modify or touch anything at the OS level and the binding is not permanent.

Using the mount --bind command is a much simpler solution since you don’t have to manually modify Chia’s config file. Just a single line of the command and you’re ready to go :sweat_smile:

Edit: Seeing you in another thread saying that HDD is too slow for database files… It’s not. I have no problem farming with my database on the HDD.

1 Like

Yeah, I know/use mount all the time. This config.yaml change is just one more way to do it.

I think that it may be more what the whole setup is. I had my dbs on SSD, and had some issues. After switching to NVMe, I see basically zero activity on it (so looks like an overkill). The box is i5-8250, not that slow.

Although, maybe the main motivation for me to dump that SSD was that I tried to do sqlite backup, and that just didn’t work on the SSD. After switching to NVMe, it works great in off-line mode (with Chia being shut down), but just doesn’t work in on-line mode. Looks like it is not really a live backup after all.

On the other hand, per Chia, nodes that were the most affected during dust storm included RPis. There is no data (as far as I know) to pinpoint what was wrong with those RPis, but most likely slow SD cards. So, my take is that if one really needs to update, why not to jump straight to SSDs. It also may be that depending on what skills a person have, as one option may be preferred over the other.

As mentioned above, I read about various SD speeds, and it looks like V90 can go up to 90MB/s, what is almost the same as a HD. If RPi 4 supports that speed, I would say that such card may be preferred to a HD solution. Although, you have stated that RPi maxes out on 40MB/s on those cards, so maybe SD cards are just no good. Sill, 40MB/s is not that bad, if it can be sustained.

1 Like

Most people on the Pi will just dump their database file on the available space left on one of their farming HDDs while running the OS and Chia on the card to get the most out of device slots for HDDs.

The reason I am farming on the Pi is to minimize the initial investment per node and also the farming cost in the long run. We just don’t know whether Chia will moon. So, the best way to go (in my opinion) is to break even as soon as possible.

1 Like

I am with you. My i5 box is a NUC that was laying around (that I was using for Tizen development). It also draws around 10W or so (it is a low power i5, but again not that bad, it was able to plot, but got too hot for me - small enclosure, bad heat management).

I had an exchange the other day, with another RPi user about using one RPi just for a node, and other RPis, potentially RPi3s, as harvesters. So, really you can make a really big farm with few RPis.

By the way (off topic). Were you affected by Dust Storm? I mean, a lot of boxes got high stale percentages, but my understanding is that RPis went completely out of sync. Looking at my box Connection tab, I saw a lot of “starving” boxes asking for data, so the assumption is that those could be RPis. My box was all the time synced, but my upload bandwidth was fully choked by those starving boxes.

2 Likes

My stale rate only went up for a few days but it had been fixed on the Space (I am pool farming here) side a few days ago without having to change the default settings on my side.

I only had about two hour long jumps in stale counts. I think two on Sat, and three on Sun.

Although, my understanding of those starving nodes is that those were out of sync, and couldn’t keep up with catching up due to the amount of those small transactions. That potentially implies slow media under their dbs. If your node was all the time synced (regardless of stale count), that would mean that HD for dbs is really good enough option (maybe again assuming USB3, …).