Recently, I have had my DB’s get stuck and he unable to sync. A lot of folk seem to hit this issue over time. For me, it wasn’t happening pre 1.2.5 ; now I’m re-syncing for the second or third time.
This is a loooong process. Using my 5950x it takes about 8 hours. The CPU works pretty hard during this time. However, it isn’t at full load.
Im wondering if I can push it harder to make the sync faster. It actually seems far more impactful to add power and overclock the CPU rather than worry about peers and connections. In terms of speed, it is my CPU which is making the difference.(for me - guess I’m not peer constrained)
There appears to be a process called start full mode.exe which does all the work. Once things kick off, there are 9 instances of this process running. Is there a way to increase this . Eg by editing Config.yaml ?
Given my CPU load is at about 30% I’m wondering if I could double or triple the number of these processes running and see the same increase in sync speed.
Your sync is basically downloading and assembling the blockchain using a peer to peer torrent protocol similar to BitTorrent.
Increasing your processing power will not significantly decrease your sync time as your sync speed is mainly based on peer to peer download speed and the process limits itself for each user in favor of delivering the blocks to everybody as quickly and evenly as possible.
The best thing you can do to ensure a complete sync first try is to make sure you are connected to at least one full node when you start the sync.
This should help you get the full chain without getting stuck and having to re-sync, but nothing except downloading the blockchain (which is not really a good idea … another conversation) will speed up your sync.
Guys, I appreciate the comments but I am confident you are wrong and I think I now understand why.
The speed of processor makes all the difference with how fast the blockchain files sync. I have three machines I have tested as nodes on the same network. I can tell you that the difference on a 5950x versus a Ryzen 5 3600 or an Athlon 3000G is night and day. The 5950x goes from zero to fully synced in 6 hours. The Athlon will run at more like 4 or 5 days.
So there are a few things that might be happening here. Maybe the start full mode process (which uses more CPU) is the mini torrent program and having more copies of it running at once is speeding it up. The 5950x runs 10 processes. The 3600 runs up to 9. The Athlon runs up to 4. So that is a difference.
But I’m wondering if there’s something else. This past week there was something else complicating the sync due to these new “blue time lords” which we’re (as I understand) sharing compressed files , to try and speed up the sync. But nodes were getting inundated with lots of compressed files. And to uncompress these files takes RAM and I presume extra CPU resources. So - I think these last few days under the issues in 1.2.5 and 1.2.6, it was VERY significant of you were using a faster CPU and a lot of RAM to boost your sync.
Checking my network, I see some significant traffic, mostly 10-100s kbps, but some even 8-9 Mbps traffic with the system otherwise idling. Perhaps I’m helping others compress (I’m on 1Gb fiber connection), as that’s considerable traffic. CPU is doing up to 20% on a Ryzen 3600, typically should be almost nothing.
Yep. I try to always know of what I speak, but I did not think this fully through at first
After considering your comment I realized you are entirely correct. I based my faulty reply on my i7, 8 thread PC without having a high end machine to compare it to. Assembling the downloaded blocks into the chain takes a lot of processor power and, of course, a faster processor with more threads will do the job more quickly.
SQLite gets slower and slower the larger it gets, hence why syncing is progressively worse as time goes on. Poor choice of storage backend for a blockchain IMO. At least disable indexes until its loaded up and ready to go to speed up inserts.
Thanks for that share @AlexeyK - I had to resync starting yesterday because the blockchain mysteriously got corrupted and reset back to 0. 0 - 400,000 took about 24 hours. At this point I’m gonna let it sync to see how many more days it takes from block 500,000. Disk IO is the main bottleneck on my less powerful system syncing right now.