Chia 1.3 Windows Upgrade from 1.2.11

Hi, I just performed the upgrade on my full node and found the following:

  1. I am no longer connected to the pool I was connected to prior to the upgrade. Instead, the GUI says I am self-pooling
  2. while my node is sync’ed, my wallet is not and the balance is showing zero. While it says the wallet is syncing, I can’t tell if it really is since it seems to be stuck at the same number for the past 30 minutes.
  3. It looks like I am still on v1 db. Do I need to upgrade to v2 manually? How?

Anyone else experiencing this? Should I just rejoin the pool? And will the wallet eventually sync? Is there a way to check? Thank you!

This post might help: Upgraded to 1.3.0 - DB file still large ( +70GB) and can´t connect to pool ( stuck in pending) - #5 by sgroiwes

1.3 has caused quite a few issues for some so you may consider downgrading if problems persist, however, the post by sgroiwes may help with this problem so do try chia upgrade db if you haven’t already.

Thanks. After waiting a bit more time, my wallet sync’ed and the node automatically rejoined the pool. Sorry for jumping the gun…

On the v2 upgrade, do I have to do it manually or maybe I just need to wait a bit longer?

I couldn’t find any specific upgrade instructions for 1.3.0 but based on this comment in the release notes, it sounds like you need to do the DB upgrade manually.

From Release 1.3.0 · Chia-Network/chia-blockchain · GitHub

New v2 DB - improved compression for smaller footprint (the v2 DB is created alongside the v1 DB. Please be sure to have enough disk space before executing the DB upgrade command).

So, it seems like you have to run chia db upgrade manually if you want to upgrade to the new V2 database format. No idea what happens with the V1 database after upgrade or if you need to keep it. I guess don’t delete it too soon in case you need to roll back to 1.2.x for any reason.

If the V1 DB files don’t change after some time, then they can probably be safely deleted. Hopefully we’ll know more about this soon or someone that does know can follow up here.

After installing1.3.0 its said to do a .\chia db upgrade which creates the v2 database from the v1 database and makes changes in config.yaml to the new v2, but no where does anybody say anything of changing the wallet db from v1 to v2, becaus ethe upgrade doesn’t touch it. " would it be that hard to just make a little note on what needs to be done. Is there a current cook-book link?

From looking at the code for the db upgrade functions it appears that this only updates the blockchain database, not the wallet DB. If there are plans to improve the wallet DB this update does not cover it.

Don’t try to run the DB upgrade specifying the wallet DB either, since it will screw up the wallet if it ends up trying to run the upgrade against the wallet DB instead of the blockchain DB.

When you run .\chia db upgrade it creates a blockchain_v2_mainet.sqlite in the \mainnet\ db folder. It also creates a blockchain_wallet_v2_mainnet_xxxxx.sqlite in the \mainnet\wallet\db folder. But only makes changes in config.yaml for the full_node: database_path: db/blockchain_v2_CHALLENGE.sqlite

Thank you for everyone’s help. In case my experience is helpful, this is a summary:

  1. I shutdown my full node running 1.2.11 properly and made sure there were no running chia processes in Windows task manager
  2. Run 1.3 updater
  3. Start Chia and selected full node mode
  4. The node sync’ed pretty much right away, but the wallet took an hour-ish to sync during which time, the GUI showed 0 Chia in the wallet and appeared to be stalled. However, I left it running and in time, it finished syncing and I see all my Chia
  5. Concurrent to this, the Pool screen showed I was not part of the pool I was connected to prior to the upgrade. Again, be patient because after a while, I was reconnected to said Pool and farming as part of the pool and all is working well.

Separately, the upgrade to v2 of the database does not happen with the upgrade to 1.3. As far as I know, that has to be done via the command line and my upgrade to v2 db took about 45 minutes. The upgrade also updated my .yaml config file to point to the new v2 db. This is contrary to some YouTube videos which suggested the config file needs to be updated manually. I’m not sure if the switch happens on the fly or not after update and so I closed the full node and restarted it and now, I can see that the v2 db is the one being updated and used by the full node.

Thanks again for everyone’s help on a successful upgrade!

3 Likes

After I did the db upgrade the mainblock db was changed in config.yaml but the wallet db still shows being pointed to V1 in the wallet section. And would be nice if they came out with a little cookbook for others saying this needs to be done after installing 1.3.0 on top of 1.2.11

1 Like

When it’s out of Beta there will be better instructions. The config is only updated if you do not specify the DB path, so maybe the videos were specifying that manually which would prevent the config from being updated.

Thanks for posting that summary @hammer.

It is out of beta 1.3.0 is the normal release now.

Ah yeah my bad, it was a release and not a beta.

I’m a relatively new farmer and still learning, so all the information posted here has taken me by surprise. Are there any major drawbacks or potentially, dare I say, hazardous conditions on my farm that could arise by not performing the v1 to v2 update immediately after upgrading from 1.2.11 to 1.3.0?

You’ll lose out on improvements in response times due to the v2 db.

1 Like

Do we know what are those improvements? Are those listed somewhere?

From what I have seen on my two nodes, once I run blockchain upgrade (did it offline in both cases, so those nodes were an hour / two behind), I don’t think there were any perceived improvements there. The main start_full_node process choked one core (i9-10900), and the rest of block crunching start_full_node processes were mostly starving.

I guess, what I am trying to say is that there are three/four places where improvements can be done:

  1. Db structure / indexing (partially done)
    2 Switch to a better performing db (not done)
  2. Improvement in the main start_full_node process with respect to task dispatching (doesn’t look like anything has been done). This work would speed up syncing / keeping up synced, as block crunching processes will be fully utilized.
  3. Improvement in the sub start_full_node processes with respect to crunching transactions (doesn’t look like anything has been done here). This would let chia to use 100% of CPU time when the syncing falls behind. Although, this will not help much, if #3 is not done, as that process basically chokes the progress.

Also, the current blockchain db redesign (to get rid of redundant data) is really helpful, as it looks like with the current ongoing transaction traffic, v1 blockchain db grows by around 20-30 GB/month, so hopefully this change will cut it a bit. This is the main reason to upgrade the blockchain db, but it is not really a pressing task, and it can wait until the updated chia will settle down (if the node is/was sound on v1).

I’ve looked at the v1.3.1 release notes, but I don’t know what effects what, if it tells at all.

However, I can plainly see the improvements on my node and my plotter. Let me explain how I came to this conclusion…

My Threadripper is primarily a MadMax plotter using Powershell. So I obtained two 16TB drives to fill… this time wanting k33 plots, thinking I could even out some drive space on my other drives on my farmer. Meanwhile, I fired up the full GUI node on the Threadripper, upgraded from v1.2.1 > v1.3.1, plus the db to v2… to farm those plots made while in process.

I was astounded at the farmr times I was getting, and still am getting, over the course of a week or so on that machine. Sure there are only 110 k33s so far, but 0.0-0.25s (98.3%) below .25 seconds. I was like Wow!

Longest response: 1.10987 seconds
Shortest response: 0.0 seconds
Median: 0.000s Avg: 0.027s σ: 0.068s
0.0-0.25s: 8425 filters (98.3%)
0.25-0.5s: 118 filters (1.38%)
0.5-1.0s: 27 filters (0.315%)
1.0-5.0s: 1 filters (0.0117%)

With that under my belt, and seeing no significant problems with the 1.3.1 upgrade, I did the same on my full node, wondering if I was just seeing the result of the TR on times or was there really such an improvement in response times.

Here’s a chronology on that node with 4890 plots, mostly k32s >

2-22-22 restart - this is baseline from before upgrade to compare

Last 24 hours: 76240 plots passed 8059 filters
Each plot passed 0.99 times per 512 filters
24h Efficiency: 99%

Longest response: 3.2986 seconds
Shortest response: 0.20304 seconds
Median: 0.431s Avg: 0.501s σ: 0.194s
0.0-0.25s: 66 filters (0.819%) <---------
0.25-0.5s: 4922 filters (61.1%) <---------
0.5-1.0s: 2890 filters (35.9%)
1.0-5.0s: 181 filters (2.25%)

farmr output just AFTER restart to 1.3.1 on 3-23-22 6:30pm

4890 plots - 484.7/485.1 TiB (100%)
CPU: 11th Gen Intel(R) Core™ i5-11400 @ 2.60GHz with 12 threads
4883 k32 plots, 7 k33 plots
Last 24 hours: 84041 plots passed 8792 filters
Each plot passed 1.00 times per 512 filters
24h Efficiency: 100%

Longest response: 1.59976 seconds
Shortest response: 0.06249 seconds
Median: 0.403s Avg: 0.463s σ: 0.184s
0.0-0.25s: 374 filters (4.25%) <---------
0.25-0.5s: 5414 filters (61.6%) <---------
0.5-1.0s: 2874 filters (32.7%)
1.0-5.0s: 130 filters (1.48%)

Later that day…

Last 24 hours: 85891 plots passed 8959 filters
Each plot passed 1.00 times per 512 filters
24h Efficiency: 100%

Longest response: 1.60422 seconds
Shortest response: 0.06245 seconds
Median: 0.362s Avg: 0.410s σ: 0.194s
0.0-0.25s: 1934 filters (21.6%) <---------
0.25-0.5s: 4603 filters (51.4%) <---------
0.5-1.0s: 2319 filters (25.9%)
1.0-5.0s: 103 filters (1.15%)

3-25-22 7am currently as I post this

Last 24 hours: 86612 plots passed 9061 filters
Each plot passed 1.00 times per 512 filters
24h Efficiency: 100%

Longest response: 1.47228 seconds
Shortest response: 0.06248 seconds
Median: 0.299s Avg: 0.364s σ: 0.184s
0.0-0.25s: 3204 filters (35.4%) <---------
0.25-0.5s: 3986 filters (44.0%) <---------
0.5-1.0s: 1808 filters (20.0%)
1.0-5.0s: 63 filters (0.695%)

In summary, going from 0.0-0.25s: 66 filters (0.819%) to 0.0-0.25s: 3204 filters (35.4%) on the same machine, the only difference being the upgrade, something major has been improved, at least in the farmer.

1 Like

Yes, I also checked v1.3.1 release nodes, and didn’t find much there. Although, I didn’t check v1.3, and those should have more improvements, as v1.3.1 is just a QA forced cleanup for v1.3, not really a feature improvement release (v1.3 should be suppressed or marked as not-to-be-used release).

Good data (mine was just observing Task Monitor during synchronization / catching up, …), although I am using ChiaDog, and in my case, there is rather not that much difference there (ChiaDog doesn’t provide much info).

I am not that familiar with farmr output. Are those filter percentages a different way to show harvester lookup time distribution?

In case of ChiaDog, it only reports an average lookup time, and how many are over 5s/15s, and all those are more or less the same for me - about 1-3 lookups over 5 secs per day and averages around 0.75sec (node i5-8250U NUC, USB drives). There may be a 30ms (5% down) improvement in averages (before 0.77 sec, now 0.74 sec), or it is just a fluke. Although, those are harvester responses, not really the node (in ChiaDog case). Still, in your case those times dropped roughly by about 30%, what is a big improvement. Maybe, as my node has only USB drives, those have slower access times and are already close to be optimal. Again, I am assuming that your data are lookup times, but maybe those are some network response times (what ChiaDog doesn’t provide), and if so, I don’t know how to factor those in.

No doubt, v1.3.11 is a good and clean upgrade for several reasons (to me, the wallet is potentially the biggest improvement - wallet db size dropped from ~10GB down to 15MB). Although, I was looking for improvements around start_full_node related to transaction processing, as that part helps the initial sync process, and keeping nodes afloat during dust storms, and at least looking at my two nodes, I really cannot say that much got improved, if anything at all (although, my NUC node is rather not affected by those dust storms). Maybe I will need to sync my plotter from scratch to compare it with the previous full sync time.

In any event…I’m more than satisfied with the upgrade to 1.3.1.

1 Like