Protip: check your debug.log regularly

So protip you DEFINITELY want to be looking at your debug log file regularly:

Windows
C:\Users\user\.chia\mainnet\log\debug.log

Linux
~/.chia/mainnet/log/debug.log

You want to search for WARN and ERROR in particular… based on that I cleaned up quite a few things:

Partially copied plots

Size is {x} GiB but expected at least 99.06 GiB. We assume the file is being copied.

2021-04-23T18:53:42.229 harvester chia.plotting.plot_tools: WARNING  Not farming plot \\192.168.1.222\public\chia-plots-1\plot-k32-2021-04-02-05-36-320c41a5e8f740f97710bed9459e51c6bb5a50bb12c8712d89d0d6f2b9cff408.plot. Size is 43.3938511563465 GiB, but expected at least: 99.06 GiB. We assume the file is being copied.

Duplicate plots

WARNING Have multiple copies of the plot {filename}, not adding it.

2021-04-23T18:53:41.807 harvester chia.plotting.plot_tools: WARNING  Have multiple copies of the plot \\192.168.1.222\public\chia-plots-1\plot-k32-2021-04-06-19-40-aad47674376adb019e8bc03ad534f64bffc19fcc0846febb111f82ab82dbe7ca.plot, not adding it.

Invalid plots

Failed to open file {filename}. Invalid plot header magic

2021-04-23T18:53:39.783 harvester chia.plotting.plot_tools: ERROR    Failed to open file \\192.168.1.97\public\chia-plots-1\plot-k32-2021-04-20-06-03-8b464aab21ec90f325deda349ce516e017cce021be1b1ad58033c37d2d2aa69f.plot. Invalid plot header magic Traceback (most recent call last):
  File "chia\plotting\plot_tools.py", line 183, in process_file
ValueError: Invalid plot header magic

Error using prover object Src size is incorrect

2021-04-23T22:42:04.301 harvester chia.harvester.harvester: ERROR    Error using prover object Src size is incorrect
2021-04-23T22:42:04.302 harvester chia.harvester.harvester: ERROR    File: \\192.168.1.223\public\chia-plots-1\plot-k32-2021-03-30-00-13-67db759c36afedb74e92e37b197a3f7b1678936a7935d10aab9095e53c40a743.plot Plot ID: b'g\xdbu\x9c6\xaf\xed\xb7N\x92\xe3{\x19z?{\x16x\x93jy5\xd1\n\xab\x90\x95\xe5<@\xa7C', challenge: ee8baaa2359286f497daa01c7e65b14bbd028c794341684611cc1e0012f6a0b8, plot_info: PlotInfo(prover=<chiapos.DiskProver object at 0x00000158DCC2F7F0>, pool_public_key=<G1Element b59e5c3f507029ac4bbff11e00aac05588effc49d210273080ee17f0fcbdf09c70501ce9aa2686411d00ed3bc8d77814>, pool_contract_puzzle_hash=None, plot_public_key=<G1Element a2246279ae675c8e05510e0eaf928f37628225304db9daa96a7c67e0851fbe243c465779d4221018a36d92f37379cd53>, file_size=108824625965, time_modified=1617131313.49253)
8 Likes

The log level is specified in your config.yaml like so:

  logging: &id001
    log_filename: log/debug.log
    log_level: WARNING
    log_stdout: false
  • The default log level is WARNING.

  • A log level of INFO makes it very verbose for troubleshooting.

  • I am assuming there is also a stricter level of ERROR which only shows errors?

2 Likes

TIL you can tail | grep in Powershell

For monitoring your farming times.

Get-Content -Path "~\.chia\mainnet\log\debug.log" -Wait | Select-String -Pattern "found"

If you’re only looking for errors

Get-Content -Path "~\.chia\mainnet\log\debug.log" | Select-String -Pattern "error"
5 Likes

For Linux users (Ubuntu, etc), you can find your config.yaml file and logs in a similar path:

config.yaml~/.chia/mainnet/config/config.yaml

logs → ~/.chia/mainnet/log

1 Like

Cool I will edit that into first post

1 Like

There is now a convenient Grafana dashboard, but one has to build it oneself.

6 Likes

@codinghorror Good tip! Just found a bunch of “failed to fetch block” error messages in there. Would you suggest it’s best to leave the error setting on WARNING or increase it to INFO? Or is INFO too verbose unless you’re investigating a specific problem?

1 Like

Info is quite verbose but since failing to meet the harvester proof challenge in 30 seconds is a hard fail and is only logged as INFO at the moment, I recommend switching it to INFO. If you are confident your harvester proof checks are taking less than 30 seconds I would not worry about it.

You definitely want to scan the logs for ERROR without a doubt though. Anything showing up as an ERROR is indeed a real error!

4 Likes

When I check debug.log and found proof challenge I have 31 - 33 s. This is not correct time right ?

That is indeed not correct. It should always be under 30s.

1 Like

I’m wondering whats goin on, because I don’t have any errors or warnings ;/ in debug.log

It entirely depends on your setup, but you might want to read this topic

1 Like

I have a small ~2.5TB farm and no issues at all. Highest time is 3s and it only happened 2 times according to logs.
PC is plotting to an external SSD and farming from external HDD, both connected via USB 3.1.

3 Likes

Do you need to restart chia to apply the change to the config.yaml (switched to INFO to check if everything is working fine)

1 Like

hi. i have been farming 66 plots for like a week and i still have No blocks farmed in my client so i starting to think that something is wrong, i know 66 plots is smal but atleast i shuld have farmed or ?
my debug log have this entrys in it that i dont understand, the ip numbers is not mine and i have port 8444 open, im quessing its others that it tryes to connect to that have problem but i want to check if someone here knows more abute this errors, im running the full node under windows server 2019

2021-04-28T21:59:50.992 full_node full_node_server        : ERROR    Exception Was not able to get peername from <function WSChiaConnection.__getattr__.<locals>.invoke at 0x000001BD778C7DC8> at 81.170.53.173, exception Stack: Traceback (most recent call last):
  File "chia\server\server.py", line 243, in incoming_connection
  File "chia\server\ws_connection.py", line 59, in __init__
ValueError: Was not able to get peername from <function WSChiaConnection.__getattr__.<locals>.invoke at 0x000001BD778C7DC8> at 81.170.53.173


2021-04-28T22:05:18.474 full_node full_node_server        : ERROR    Exception:  <class 'concurrent.futures._base.CancelledError'>, closing connection {'host': '109.233.88.85', 'port': 8444}. Traceback (most recent call last):
  File "chia\server\server.py", line 521, in api_call
  File "asyncio\tasks.py", line 435, in wait_for
concurrent.futures._base.CancelledError


2021-04-29T14:37:40.210 full_node full_node_server        : ERROR    Exception: Failed to fetch block 204103 from {'host': '2.45.173.92', 'port': 8444}, timed out, {'host': '2.45.173.92', 'port': 8444}. Traceback (most recent call last):
  File "chia\server\server.py", line 511, in wrapped_coroutine
  File "chia\full_node\full_node_api.py", line 100, in new_peak
  File "chia\full_node\full_node.py", line 358, in new_peak
  File "chia\full_node\full_node.py", line 307, in short_sync_backtrack
  File "chia\full_node\full_node.py", line 292, in short_sync_backtrack
ValueError: Failed to fetch block 204103 from {'host': '2.45.173.92', 'port': 8444}, timed out

On this forum, there are people with 500+ plots and no win, and there are people with < 10 plots with a win.

Looks like that’s the case. It’s a peer-to-peer network, a lot is happening behind the scene and peers get disconnected all the time for different reasons.

nice, yes i know that its a gamble to win, i was just thinking abute that i dident had any chans to even farm but i think you are right, it working and the errors i see is generated from outside,

well i can plot 22 plots / 36h so it just plotting on

1 Like

1 plot were eligible for farming 3… Found 0 proof in 0.297 s

  1. I guess this is basically one plot passes the filter but I don’t find the best hash, so no reward?
  2. as long as time < 30s I’m good to go? or I should optimize to use less time?

Exactly :+1:

Less than 0.3s is a very good time! I wouldn’t bother doing anything unless those timings get above 5s constantly.

1 Like

Where did you pull this info from?