Chia blocks misplaced/pool activity - went solo, yet block rewards (1.75) transferred to pool

Hi, lately I am experiencing a very weird behaviour of my chia farm. I want to try my luck here, because no one helped me or told me what is the root cause of this bug on chia reddit and also official github where I posted 9 days ago.

Here is the issue description and what I already did (copy paste from github): 3-4 weeks ago I decided to go full solo and stop pooling with Flexpool for a while just to try my luck. I actually won a 2 blocks right after (05/17/2022 and 05/23/2022), everything went as expected (2 x 0.25xch directly to my wallet, 2 x 1.75 showed in GUI under my solo pooling session – easily claimable without any issues, done that for both rewards on 05/23/2022).

However, since then I won another two blocks, but somehow I only received 0.25xch (0.50xch) directly to my wallet and 1.75 (3.5xch in total) nowhere. So I started digging and found out that the 1.75 rewards are reported under Blocks menu within my original/former Flexpool gui - webpage. || note: I did not perform any actions with the farmer between first solo claimed rewards and these two, which went to flexpool ||

Please note that I did not send any partials anymore, also my Flexpool homepage was blank (0 workers online with 0 B of total storage since the day I stopped pooling with them) and I do not receive any small rewards for pool farming as it was before. In my chia client I saw that all plots were allocated to solo farming, 0 to pool farming. Is there any way how to solve this issue and get my rewards back?

Please let me know and I will provide my Flexpool pooling address and block height for both rewards directly - I don’t know if it is secure to share it right here to the post as a public info.

Thank you very much!

What I already did based on help and ideas provided by fellow redditors: reboot of my farm machine, tried to go solo once again, also joined SpacePool on 06/04/22 for like 12 hours (approx last peak size approx. the same as the total amount of my plots) and then went back solo (also SpacePool overview still says “last seen self-pooling”). Double or tripple checked via CLI command chia plotnft show, which states that I am SELF-POOLING with all my plots! Also both farmer reward address and pool reward address changed via gui to my newly generated wallet address in my farmer client and also confirmed in config.yaml that it changed properly.

EDIT on 06/10/2022: yesterday 21 hours ago I won another block (wow, lucky me!), but it is the same story again? 0.25xch directly to me, the rest of the reward to Flexpool, which is crazy considering the fact that the last pool I was connected to is SpacePool… I have made printscreen of everyting which I think might be useful, can provide on request to share more details with you.


Thanks in advance for your help or identifying the reason!

Version: 1.2.11, OS: Windows, Mode: GUI

Have you asked flexpool about this?

Their costumers service is quite good in my experience.
They can verify if they received those pool rewards and might be able to help figure out what happend.

3 Likes

I know it is kind of putting a new problem on the original one, but I would start with moving to v1.3.5. You will not get any support from Chia devs, as for them (and quite a lot of other people), that version is already dead (I like v1.2.11, but am also not really interested in looking into that version anymore, because for once I even cannot related my UI to yours).

Flex pool is a solid pool, so you should be reaching out to @Chris22 that works at Flex, and he should be able to help you with those XCHs that still went to Flex. In the meantime, as mentioned, I would be working to upgrade your farm.

Also, since you have also tried Space, I would go back to Space, wait for another win, and ask them for help, if it will still be going to Flex.

As long as you are farming solo, you are basically SOL, as potentially there is just not enough info in debug logs to try to bite this problem, and there is no way to duplicate it (as it seems to be that this is not happening too often).

Not to harp on the same issue again, but if you upgrade to v1.3.5, join Space, will get that 1.75 XCH sent to Flex, then you can open a new case on Chia’s github, where potentially both Space and Flex can back you up.

Lastly, by any chance, I hope that you don’t have FlexFarmer lingering anywhere around your farm. Although, it should act properly (refuse to farm), but again you have something odd with your setup, so would be nice to exclude this possibility / cleanup your farm.

By the way, when you win a block, and you are pooling, there are (at least) two entries in the debug.log. First is that your wallet / singleton received 1.75 XCH, and a minute or so later that your wallet released that 1.75 (so it went to pool). Maybe you could try to chase those log entries, as maybe there is more info there that could be helpful.

2 Likes

Hi Voodoo, yes I did and they told me that the mistake is on my side, not on theirs :slight_smile: There was also quite a heated debate in my reddit thread, but the result was they will not give me back anything and thats it, my bad luck (fully understand that this is a crypto world and it has its specifics, but such unethical behavior is just bad, and that is not only my opinion for sure…). Here is the link if you are interested: https://www.reddit.com/r/chia/comments/v21qiq/switched_to_solo_farming_yet_still_earning_only/

Hi Jacek,

yes, I like(d) v1.2.11 as well and that is basically the reason I am still running it, but it seems like it has a sort of serious bug which shook my sympathies with this version.

I have stayed on this version in recent days primarily to leave the farm in its original condition and, if necessary, to find out what the problem is. Anyway, I can perform the update and then apply the solution you are proposing - it sounds good. It just doesn’t solve the fact that I lost 5.25 xch, which is not a small amount of money even at the current prices.

I can try to reach Chris, but I already contacted them via support line (I communicated with Alex) and he said he can NOT do anything for me (it is described here as a reply to Voodoo’s post). He basically told me that the mistake is somewhere else and that I should reach devs and solve it with them…

Negative, since day 1 I am running solely on official sofware provided by chia devs via official site. Also it is the only thing installed and running on my farmer PC in separated VLAN.

OK, I can check the logs, but I am not sure how “long” is the history.

Actually resyncing the wallet db might be a good idea. That may fix why you are seeing an incorrect nft state.

How has Chia responded?
If you have partials going to spacepool, but then the reward goes to flex…that’s something they have to look at imo

1 Like

OK, I can try the wallet resync as well.

Chia devs did not respond yet, no one is assigned to the bug report on their github.

No, I joined spacepool only for like 12 hours and after that I went back solo, so the last win and 1.75xch reward for flex was not transfered to them when farming with spacepool, but when going solo again.

That was a good approach, as long as there was a hope that the issue can be easily identified, and somehow will need v1.2.11 to fix it.

However, I would say that right now is kind of too late, as all the parties had their chance to put some kind of solution, so having that version is not really helping. Plus, it may be that v1.3 will fix it.

As @Voodoo pointed out, quite a bit of those sort of bugs are also around wallet, and v1.3 is using a new format, so that may help.

Again, you have two issues on hand. To recover those XCHs from Flex, and to get your farm cleanup. So, the upgrade may help with the second one.

@Chris22 has been already pinged. He is checking this forum from time to time, so you should expect that he will chime in. Again, I would not expect him to help with how to fix the problem, but in my opinion, returning those XCH back to you is what he has to do. So far Flex has rather good reputation, and if they don’t return it, that reputation will be tarnished. Yeah, I also had some questions on Flex discord channel, but gave up on that route, and soon after quit Flex pool (that was one of the reason - worthless support on that discord channel).

The reason to be on Space while the next time that happens is not really to have Space or Flex help with fixing Chia’s bug, but rather to have two reputable pools on both sides confirming what is happening. This may be a bug with NFTs, so unless it will get fixed by upgrading to v1.3, it looks like Chia devs may be the only option to help.

Yup, the problem with full node / farmer logs is that those are rather extensive, so those rotate fast. Just grep for 175000, and that should give you date/time. Don’t forget that you also have few debug.log.XYZ files. On Linux “grep 17500 debug.log*’ will do, on Win 'findstr 17500 debug.log*” or something like that should work.

Again, you have two different issues. One is to get back your XCHs, and I don’t think that resyncing your wallet will help here, as it looks like all that is already sitting at Flex’s wallet.

The other is to get your farm back in a clean state, so trying to resync while still on v1.2 may just prolong the agony.

It is a good advice, but maybe v1.3 is a better option. Hard to say.

I fully agree with @Voodoo that this is your XCH, and Flex needs to return it to you. I also agree that maybe Chia devs are the best or only option, as this clearly is their bug. However, as I said, the issue might be ignored previously, as they don’t want to touch v1.2 anymore.

So, my take would be:

  1. Stop chia, reboot
  2. backup your .chia folder (so you have a backup of both dbs, config.yaml, etc)
  3. upgrade to v1.3.5
  4. stop chia / reboot
  5. run db upgrade
  6. make sure that both bc and wallet dbs are v2 in config.yaml
  7. start chia
  8. let it sync for an hour or two
    8.1. do not pay any attention to what your wallet shows, as the assumption is that it is screwed up in this phase
  9. stop chia / reboot
  10. kill wallet db, let it sync again (a couple of hours)
    10.1. initially, the wallet may show synced, but still have partial XCH, so let it really sync for an hour or two (or more if needed). It just takes some extra time and doesn’t provide any feedback when it is done (yeah, you can check on blocks processed). So, don’t freak out, just be patient
  11. join Space pool, if already on Space, switch to Flex, once that will be active, go back to Space
  12. wait for the next win, and we will proceed from there, but hopefully, the issue will be gone

If you get stuck at any point, just ask.

Those reboots are for nothing more but making sure that chia fully exits / shuts down. If you can check on that with either ps or TaskManager, there is no need to reboot, but those reboots are no-brainers, and for sure clear things up. So, when in doubt …

2 Likes

This was hashed out on reddit and our discord I believe.
Its a very long conversation between multiple members of our staff and the reporter.
I wasn’t too involved so I may be wrong but here were the main issues from what I’ve heard:

  1. There was no evidence that this person wasn’t with us at the time. He claims he was soloing but how do we know this is true? People claim daily on ETH to own certain miners wallets and ask us for balance transfers and we have a very intense verification process for this which many fail as they are scammers. We don’t trust we verify. When asked several questions the person began threatening our support staff rather than answering.
  2. If there was a bug/user error and XCH was sent to us, it was already distributed. Should we reduce future farmers rewards because past farmers on our pool benefited from this users mistake? Some farmers would have joined and some would have left so it wouldn’t be the same people paying it back.
  3. If there is a bug in the Chia software or an User error that resulted when that person switched from pooling to solo, who is responsible? The software dev? The user? Or us the people that got a 0.7% fee? What evidence is there that there was a bug? We created Flexfarmer and if there was a bug with it then we would take responsibility but to say we’re responsible for software we didn’t create seems like a stretch. Especially as we have stated that a main reason behind creating and recommending Flexfarmer was because the Chia software has bugs.

A reminder that we’re making around $1000/month in XCH from our pool versus several thousand a month in costs. We’re operating at a loss. At the time this was around a $50 loss on the user that was not in any shape or form the result of our software or actions. We did forward this on to the Chia Network for them to solve.

I’ve heard it said that Space would do different. We’re not Spacepool. If they want to take from their farmers to correct Chia client bugs/user errors without asking them they are welcome to. To doublecheck I ran a small poll on our discord to see if our farmers agreed with the reddit poster and there was unanimous disagreement.

1 Like

As mentioned, I also have tried Flex’s discord channel, and I have to say that my experience is similar to what @moranden reported. So, sure you can or rather should consider me as biased here, but based on what I have experienced, I would not recommend anyone to touch your discord channel. Although, this is not the point, and I will try to be objective, and hope that other folks will make sure that I stay like that.

Whether it was extensively hashed out on reddit or on your discord really doesn’t matter to me at all. He asked for support, and is providing info when asked, and it is also great that you have chimed in. So, hopefully we can get on the same page at some point.

As I stated, he has two issues:

  1. his setup may be out of whack potentially due to chia’s bugs
  2. assuming that #1 is correct, I understand what you are saying about having his XCH already distributed, but I really don’t understand why he should be penalized, and your pool status tarnished, if the culprit is Chia.

So, I hope that when he goes through the motions with upgrading to v1.3.5, his setup will get cleaned, and the first goal will be accomplished. To me, this is the potentially an interesting part for me to be engaged, and I can back off once we accomplish this task

Saying that, if his setup will start working, that is absolutely no evidence that as you stated he is not missing something in his assumptions and the problem was there in the first place. This is also the reason that I asked him to try to find those logs about those 1.75 XCH exchanges, hoping that more useful info is there. So unless he will produce some compelling proofs, the second issue will be moot.

Funny that you call that “doublecheck.” Not sure whether you realize that that discord channel is just your echo chamber. As a matter of fact, I left Flex, partially based on what happened there, so you can count people like me that would rather strongly disagree with whatever nonsense your run through that channel. You may try to run similar poll on this forum, and I can promise that I will vote.

UPDATE
Yes, reading what I wrote before, and what you quoted, I should have stated there that Flex has to return that XCHs, if there are good enough proofs (I guess, “good enough proofs” is where we may not have an agreement at the end, though). Assuming that this is the case and this is chia code bug, I would argue that Flex as a pool should go back to Chia, and ask them for compensations, rather than penalizing (former) pool members. To make the ecosystem whole, I would assume that Chia should be making efforts to help all of us move forward, and be assured that software that they put out is backed up by them. That was more or less the gist of my take to get both Space and Flex involved when dealing with this issue on Chia’s github page.

One more thought. Assuming that the problem is still there, and all his 1.75 XCH go to Flex, maybe you could suggest an experiment that he could try to reproduce this problem? Maybe something like:

  1. Joining Flex to and get a proof that Flex was joined
  2. Quitting Flex (to go to Space, …) to get the proof that he quit Flex
  3. Wait for his next win, and scrutinize what has happened

Maybe you can run it through your team to figure out whether such test would be possible. If it would, to me that would be really strong argument to join forces, and push Chia to provide a solution (transfer some XCH to either Flex or @moranden wallet). They are sitting on the pre-farm, and it would really be f-ed up, if they would not get their act together to make those things go away. This is their software that potentially is the culprit.

1 Like


Hardly an echo chamber there are some pretty smart people farming Chia. Theta was the one who dealt with the original ticket so he knows more about the incident than I do.

As for your suggestion to debug the Chia farmer, we’ve already done so when we made our own. As Theta pointed out this is a very rare issue that has only happened during a dust storm (and there wasn’t one going when this was reported). Its more likely to be user error. But even if not then its a bug with a farmer that isn’t ours. And obviously we were quite annoyed once this person began threatening our staff.

In the end Chia’s software is just the reference, we’ve referred to it and made something better. Just like Ethereum and Bitcoin released with their own reference mining programs that didn’t last. Use non-recommended software at your own risk and if there’s a bug talk to their support. If there was a Flexfarmer bug I’m sure the Chia Network would tell the complainant to come to us.

PS: Chia to the moon!

1 Like

@Chris22
Would you really like me to reply to the garbage you wrote? I can try, but we will go the ugly path.

1 Like

@moranden Before you get to #11 on that list, let’s review your status.

1 Like

Hi Jacek,

yes, fully understand that and I also see it as two separate issues.

Yesterday I moved from 1.2.11 to 1.3.5 basically without any issues, I also did the full backup of the old 1.2.11 client and .chia + .chia_keys folder files (somewhere around 160gb - kinda huge). Just to be sure, I wiped the wallet/db folder and it now contains only 3 blockchain_wallet_v2_mainnet files (sum is somewhere around 5.53MB which is a huge difference regarding the size comparing it to v1 wallet which is somewhere around 18.5GB at least in my farm).

Steps progress:
1 - done
2 - done
3 - done
4 - done
5 - done
6 - not done - I would appreciate help with this one, what am I looking for in config.yaml file?
7 - done
8 - done - this was a bit weird, because it synced in a matter of minutes from scratch and before it finished I saw quite intense movements and automatically performed changes in my pool view in GUI (switch between SOLO POOLING and flexpool pooling - this one with status Unconfirmed in red letters) | anyway all my wallet movements in overview and current chia height is correct. I am also aware that the wallet resync will not bring my rewards back, sadly.
9 - done
10 - did not done this one, if it is necessary, I can do that today
11 - done, joined Spacepool right after the update, GUI shows that I am farming with them, also my webpage. Partials arrived already as well. Also no sign of activity with flexpool.
12 - in progress :slight_smile:

Thanks for your help!

Hi Chris,

  1. No worries, it is quite understandable that, based on a few forum messages, you will not immediately start sending rewards back to the miners, but here I dare to oppose. First of all, nobody from your team provided me with any answer, how it is possible that my webspace on flexpool showed 0 activity, 0B of data and 0 workers online for past 24 hours windows (which is your webpage overview time interval limit and to be honest, it is quite short), yet in the “Blocks” menu there was a new record no older than 24hs, which means one of my plots won, 1.75xch went to flexpool, yet my session with you said I was not farming with you and I also did not recieve any partials. Just btw regarding the size of my farm I did recieve partials approx. every 15-20hs which is also stated in the payments history). Secondly, none of your team members explicitly required any additional records or evidence, either in direct mail communication or via reddit thread. The only thing they kept repeating was the fact that the mistake was not on your side.

  2. Sorry, not the same thing, because I was no longer farming with you, so I did not recieve any partials and yet you still consumed my rewards - sort of unfair, don’t you think?

  3. Hehe OK, I know that this is a sort of different world than the “real and old one” - I mean banks - but here I am gonna use the argument from reddit. When you accidentaly recieve 1m dollars in your bank account, will you keep it? Nah, I don’t think so. But you are right that those who have to solve this and oversee the right procedures and solutions are the developers and creators of blockchain.

I don’t understand your next argument within the paragraph of text - why should i care about the management of your team in connection with the operation of the pool? This is how it sounds to me that I contributed a gift to you and helped cover the loss, great. BTW 5.25xch is not 50 bucks…

And the last one - asking pool members for their opinion on voluntarily giving up money they have earned basically without any effort, thus for free, is extremely misguided - there is no point in commenting on that.

Btw mentioning the threatening in point n. 1 - I sincerely hope you don’t mean me because that’s definitely not true. I see no reason to threaten anyone, but at the same time you have to understand that this situation annoys me.

Wow, you mention threats again, so you really thought of me in your earlier post. I took the time to go through the e-mail communication with your team and at the same time reddit, where exactly should I see the threats on my part towards your people?

OK, if you consider as a threat the mention of providing the community with information on how you are dealing with this problem and, in fact, still pushing me out, I have to smile. In my country, this is called a review, if you consider this a threat to you, then I’m sorry.

I consider something in the style of physical violence and the like to be a threat, which is completely misleading. We are humans, not animals, I would just like us to be fair, at least within this community, and not to start stealing food from each other under our noses.

Anyway, if you feel that I was somewhere across the line in the past communication, feel free to share it here, I’ll be happy to take a look.

I will try to go thru the logs today and I will also post here the printscreen I took right after I hit the last (and the third) block - I got GUI prinstcreen, flexpool site, CLI exports and printscreen as well - so I hope it will be enough, but as you mentioned, there is everytime a discussion about what is “proof good enough”. The funny thing is that nowhere is it officially written what everything is needed to have as a bulletproof evidence/record… Actually I am discussing this all the time with my clients, quite exhausting.

I would refrain for now from engaging with Flex, as it is clear (at least to me) that it is on its way to become one of those scum pools. Further, I would rather want to see your primary issue resolved (missing 1.75 XCH xfrs), before doing anything else. Furthermore, for now it just clutters this thread.

I would also put aside those copied folders, as those may be handy later, if there would be a need to further scrutinize v1.2.11). Also, this way, your saved logs will have potentially some a bit more relevant events.

As far as #6, look for those entries in your config.yaml:

full_node:
  database_path: D:/db/blockchain/blockchain_v2_CHALLENGE.sqlite
...
wallet:
  database_path: D:/db/wallet/blockchain_wallet_v2_CHALLENGE_KEY.sqlite

The full node most likely was updated during db update, but wallet may not be.

By the way, there were changes in farmer / harvester protocol in v1.3, so you will need to also update your harvesters, and re-init them (with your SSL files).

For #10, if you see the latest transactions in your wallet, there is no need to kill / resync the wallet db. However, the wallet resyncing process is right now really fast (an hour/two) and it doesn’t affect farming, so if there is a need in the future to do it, it can be easily done.

If you search your debug logs for “Submitting” keyword, you should see plenty of entries there (when pooling). I would think that those lines can potentially be used when checking with the pool about what it has logged in. On those lines you have both, your launcher ID, as well as the destination (e.g., which pool those proofs went to). So, the minute you win a block, make a copy of your debug.logs, as those will show where your proofs were going, as well as those 1.75 temporary xfrs, maybe some other relevant entries.

Also, you may want to search your logs for that launcher ID, as it may show your pool switches. Although, I am just guessing it, as my logs don’t cover my last pool switch.

What is your ETA for the next win?

Maybe you can also open a new thread, and ask whether anyone knows how to scan bc for pool changing events. I don’t really know how to search for those, but if this is doable, it may help in the future. Maybe you should post it also on keybase, to get Chia devs input. I would get it going in parallel to this thread, and as soon as you can. It may be that such search is impossible on the bc, but devs can provide some feedback how to scan for it on a relevant wallet db. This may even be more feasible on v1 wallet, as it stored plenty of junk (and you already have a copy of that wallet).

1 Like

Hi Jacek,

sorry for the delay. Well mentioning #6, in my current config file both full_node and wallet still point to v1, which is weird. Even after update to 1.3.5, full_node v2 file did not occur, so I am still running on old v1 150gb file and v2 is not present at all (should I kill the client and processes, delete V1 files and force it to create v2 this way?).

Weird is that my wallet is in .chia folder showing as v2 but config still points to v1… V1 wallet is moved out of the prod folder and is sitting in a backup folder on my desktop, also date of last change on my v2 is current, so it is updating its content continuously, v1 on desktop for sure with the last change date when moved out.

Yes I did the update also on my harvester, so It is running as expected and showing all my plots.

OK, hopefully in following days I will find some time to dig deeper in the logs and I will also open another threads as mentioned and suggested by you. But first of all I will get the printscreen together, I just need to blacken the sensitive data.

My current ETW is 20 days btw.

Btw still no reaction on chia github and none of admins assigned to my task… Chia blocks misplaced/pool activity - went solo, yet block rewards (1.75) transferred to pool · Issue #11847 · Chia-Network/chia-blockchain · GitHub