Hi,
I’m nearly out of diskspace and so i checked the .chia folder. I’m at 11G for /wallet and 84G for /db. This seems very high to me.
- Is this a reasonable size or has something gone wrong?
- Is there a reliable way to shrink db or wallet?
Thanks
Sounds that you have not done the 1.3.1 upgrade. This will shrink those numbers considerably.
Once properly updated to 1.3.1 and synced:
- blockchain db v2 is about 45% smaller
- wallet db v2 is negligible (some 60MB)
A few questions for my version 1.2.11:
- If I upgrade to 1.3.1, will Chia automatically compress the database?
- If compressing the database is optional, is there a GUI menu or chia command to manually execute the compression operation?
- I assume that the full node needs to be down during the database conversion/compression?
How long does it take? - How much free disk space is needed for Chia to have room to perform whatever it needs to perform for the compression to complete properly?
- Working with compressed data uses more CPU cycles. Is it noticeable? Or is it 1% or less of CPU cycles?
My blockchain is 97 GB. But I have 311 GB of free space. And my 1.2.11 is not giving me any headaches. So I would rather hold off on the upgrade, unless someone can offer other benefits for upgrading.
And when 1.3.0 was released, I read about people having issues. Has 1.3.1 ironed all of that out. Is it ready for prime time?
When you do the upgrade from 1.2.11 to 1.3.1 and run the .\chia db upgrade the database files are smaller and load quicker. Just make a backup of both folders before you upgrade. And after the db upgrade change the config.yaml to point to the v2 under the wallet section, it changes the node for you.
1 Like
No
CLI only: chia db upgrade
No you can keep running, after it finishes, you need to restart. (and check if the config.yaml is correctly set for the new v2 db)
No idea, haven’t noticed a big change.
I think I heard Gene mention in the last ama this is actually kinda a soft fork, so please update.
I installed it, everything worked without a hitch, but I have never had any trouble with any of the updates so far.
2 Likes
If you are running out of disk space, it is probably a good time to think about a disk upgrade. The db will still continue to grow and even if you are able to save, say 30% with the db v2, you will be at the same point sooner or later.
I don’t even know if you can upgrade so easily as this will temporairly expand the disk size considerably. Disk space is not so expensive anymore.
It is a totally different story if you are not farming and just use the wallet. Then I would delete the DB and upgrade to 1.3.1 right away. The light wallet mode in 1.3.1 does not need a big saved db and syncs in a couple of minutes with a good cpu / network connection.
@seymour.krelborn I am also on 1.2.11 on my farm still and would rather wait further. I have 1.3 on my wallets at home to access my funds. (The light wallet truly is a relief)
If you look at the github, they stated that many bugs are fixed, so that it’s worthwile to release 1.3.1 but many issues are still unaddressed:
We appreciate all of the feedback that we’ve received from our 1.3.0 release, and are happy to deliver 1.3.1. This point release includes several fixes to issues that were introduced in 1.3.0. While there is still more work to be done, 1.3.1 includes enough important fixes that make it worth delivering to you now.
They probably did not want wait further to release a patch to the most critical issues of 1.3.
You will see smaller db and some performance improvements. If you do not have perfomance / space issues, I would wait until a confirmed stable release is out.
This is widely recommended in the forum from what I’ve read. And people reported roughly 40 minutes - 3 hours
You can not easily make that statement. What truly uses CPU cycles are nested loops and non optimized software.
An overly simplified example could be like:
10 = 1+1+1+1+1+1+1+1+1+1;
10 = 10*1;
Chia is still very un-optimized. So I would expect that code optimisation makes way, way more up than working with compressed data. Also not having seen the database yet, it could mean that they had unnessesary data in it. Cleaning that out would also “compress” it, but have positive effect on performance.
1 Like
I don’t think that there was “a confirmed stable release” yet, and I doubt that it will be. My take is that Chia doesn’t have any QA people there, and just throws stuff out to have us test it (otherwise v1.2.9 or v1.3.0 should not see the daylight - still, those should be delisted from the download page). This is potentially the reason that we see a total crap like v1.3 followed (shortly) by v1.3.1
So far, most if not all people that upgraded to v1.3.1 and took time to not rush it are happy with that upgrade. There are several improvements (e.g., wallet, or startup / shutdown dialog listing processes - really good improvement). So, I would really encourage anyone to upgrade. You can take a fast route (just keep clicking), or more paranoid (offline db upgrade, more backups), but you can be sure that there are people around that will help in case there are issues.
Also, before starting the upgrade, backup everything, and then backup again. If you are not on v1.2.11, it may be that you first need to upgrade to v1.2.10, then to v1.2.11, and then finally to v1.3.1 (maybe the middle step - v1.2.11 is not needed - again, the paranoid upgrade).
As far as performance improvements (e.g., during dust storm), I don’t think that are any. I run sync from scratch on v1.2.11 and v1.3.1 (with both v1 and v2 dbs), and I cannot say that I saw much if any differences. The bad code in start_full_node is still there, and it still chokes a single core.
Yup, and think fast as once you are screwed, you are screwed.
Although db upgrade has an option to do upgrade across two drives (e.g., provide src and dst locations). So, if you have around 50GB free on any of your plotting drives, you can push v2 there, and once done remove v1 and bring back v2 to the original location. A config.yaml needs to be adjusted, though.
1 Like
Where do I run this command from? chia db upgrade
Here is my issue:
My current OS hard drive (SSD) was originally 500GB, however, I tried to install ubuntu and windows on the same drive so it shrunk the drive to 250 GB for each OS. However, I have not used Ubuntu since and I would like to reclaim this hard drive space. How can I do this?
However, that is not a critical issue. The critical issue, my chia DB is growing very rapidly. 3 days ago I had 30 GB of space available and today I have only 17 GB of space available. I have an additional SSD on the computer outside of the chai farmed drives. Is there a way for me to run the above command on the other SSD and then stop Chia and update the DB back on C: drive? If not what are my other options?
Thank you again for your help.
How big is that other SSD?
You can run that command either from cmd or PowerShell prompt. You can also specify src / dst, so you can point the v2 db to be on that additional drive, and later decide what to do with the v1 on the c: drive. Also, once you run v1.3.1, your wallet will shrink (from ~10 GB down to ~15MB), but you will need to manually delete the old v1 wallet, and make config.yaml modification (change v1 to v2 on wallet db).
As far as reclaiming that Linux space, the simples way would be just to reformat that partition, but you will end up with two 250GB partitions on that SSD… Or you can drop that partition, and try to resize the Win one (kind of scary, but should work fine).
My other SSD is 500GB and another one is 2TB. How do I specify the src and dst parameters? Also do I need to shutdown Chia GUI and am I running the command from the Chia that in the Appdata directory?
I ran the below command but that didn’t work I think. it opened Chia GUI back up again. Chia db upgrade.
I am on the safe/paranoid side, so I would stop chia, so blockchain db will not be moving. People report no problems while running chia but having it down for a couple of hours is not that big deal for me, and at least for me gives me a bit of peace of mind.
This is your friend - AWS Marketplace: Rocky Linux 8 (Official)
open cmd prompt, and run:
%userprofile%\AppData\Local\chia-blockchain\app-1.3.1\resources\app.asar.unpacked\daemon\chia db upgrade --input %userprofile%\.chia\mainnet\db\blockchain_v1_CHALLENGE.sqlite --output d:\chia\blockchain\blockchain_v2_CHALLENGE.sqlite
This will create v2 db on your d: drive in \chia\blockchain folder. Although, you need to create that folder first.
After it is done, check your config.yaml to make sure that blockchain db has v2 there and points to that new location.
Yeah, the current size for v2 is about 50GB, so assuming that it will grow ~50GB/year, you have some time before you need to move it from that 500GB SSD. Plus, I think that in potentially few months, Chia will switch from sqlite to some better performing db, so maybe at that time they will further remove junk from it.
First, your v1 db is just 26GB, so either it is not fully synced, or corrupted. The second line from the bottom in that cmd windows is sqlite complaining that the db is most likely corrupted (but we don’t know whether it complains about v1 or v2). Although, you didn’t show how much space left you have on that drive, so maybe it just means that you are out of disk space.
Assuming that it is whatever it is (26GB whether corrupted or not, but far away from 90GB), I would download a fully synced one to speed it up. Of course, another option is to kill that 26GB, and start from scratch (two days waiting).
If you have no problem with downloading it (I don’t, some people do), you can either download v1 db from https://www.chia-database.com/ and run upgrade on it (looks like in a couple of hours, they will refresh it), or v2 from Chia Blockchain Bootstrap Download | Elysium Pool (torrent download, looks like a couple of days old or so). Both of those places are official pools, so that makes it a bit more comfortable for me, plus both sides had discussions on this forum. Although, the main reason for me to use it is that Chia is doing virtually nothing to help people like you and me that are struggling like you are right now. As long as they don’t have their act together, I really don’t care about what they have to say about those options. It was also already discussed that possibility of those dbs being poisoned is basically zilch.
If you do want to start from scratch, kill that v1 db, start chia, let it run for a couple of minutes (just to get started with v1 db), stop chia and run that command again (will take a minute or two to convert). At this point, you will be syncing from scratch, but with v2 db (so no need for that longer update anymore). If your CPU is decent, you will be looking for 30-50 hours or so (on i9-10900, it took me ~30 hours).
1 Like
Thanks Jacek. I was trying out first on my another machine. But I selected the wrong machine. This one only had the light wallet and not the harvester. I will try on the machine that has the harvester. For now o was able to move the db to other SSD with 500GB of space so it will last for a few days or weeks. I have to go out of state and wanted to make sure my main harvester is not going to crash while I am out of state.
If using Windows delete the crap in the temp folder and prefetch, this should get you some space. Also uninstall unused software that you don’t need on that machine.