Health Check your Farm - Tips... e.g. Missing Signage Points, Invalid Partials, ASIC Timelords

Missing Signage Points

Missing signage points can occur for several reasons, including network issues, time synchronization problems, or issues with your Chia node. Here’s how to address them:

  1. Network Stability: Ensure your internet connection is stable and your DNS settings.

Tip: A wired connection is more reliable than Wi-Fi. Also set Port forwarding to 8444 on the router assigned to your computer IP and turn off upnp on your router. Also Benchmark you DNS from your location e.g. GRC’s DNS Nameserver Performance Benchmark

  1. Time Synchronization: Ensure your computer’s clock is accurately synchronized with an internet time server. Chia relies on precise timing, and even slight discrepancies can cause problems.

Tip: Use Chrony NTP Time Sync and configure for a high-performance NTP server

  1. Node Health: Make sure your Chia node is fully synced with the blockchain. A node that is lagging or frequently disconnecting can miss signage points.

Tip: On a Mac use Amphetamine on the Max Store to keep SSD/HDD awake

ASIC Timelords

ASIC Timelords are specialized hardware designed to speed up the VDF (Verifiable Delay Function) computations in the Chia blockchain. As an individual farmer, you cannot directly connect to a specific Timelord. However, improving your overall connectivity to the Chia network can help ensure that your farming operations benefit from the speed and efficiency of ASIC Timelords:

  1. Port Forwarding: Ensure that port 8444 (or the port your Chia node operates on) is forwarded correctly in your router settings. This improves your node’s connectivity to the Chia network. A suggestion could be run a port checker such like https://portchecker.co. If you want to check all ports on your network then GRC’s ShieldsUP! will do the trick.

  2. Participate in the Network: By keeping your node healthy and online, you contribute to the network’s robustness, indirectly benefiting from ASIC Timelords.

Invalid Partials

A 1% rate of invalid partials over say 9 days might not be alarming, but it’s good to minimize this. Invalid partials can occur due to network latency, clock skew, or configuration errors in your farming setup:

  1. Check Plot Integrity and Status: Use Chia’s plot check tool to verify the integrity of your plots. Corrupted or improperly created plots can lead to invalid partials. Other 3rd party tools such as Check Plot Status

  2. Optimize Farming Setup: Ensure your plots are being farmed efficiently. This includes having a stable and fast connection between your harvester(s) and your farmer.

  3. Network Latency: High network latency can cause partials to be submitted late. Optimize your network for lower latency.

Additional Tips

  1. Update Chia Software: Always run the latest version of the Chia software. Updates often contain fixes and improvements for connectivity and farming efficiency.

  2. Community and Support: The Chia community, including forums and official Discord channels, can be a valuable resource. Other farmers may have encountered and solved similar issues.

Anyone else have any optimisation suggestions?

2 Likes

These are some good tips.

It looks like the Check Plot Status tool is really only for monitoring the status of your plot creation, not the status or health of your plots being farmed. Or am I missing something?

Another tool that I sometimes use that will report on details of your farmer: https://github.com/kubec/chia-log-analyzer It does not seem to include remote harvesters in the analysis from what I can tell, however.

1 Like

Is there any way of monitoring if its staying in sync ? When i check the gui it is almost always showing it is, but sometimes a short delay in loading the information. So It could be possible it’s loosing sync occasionally and catching back up.

Always displaying some missing signage points. All cable Ethernet and router port open, db file using ssd.
Never thought about changing the the time server though.

Just put it down to internet delays and isp issues, therefore nothing could be done, and some say missing signage points are normal/to be expected

Missing signage points (SPs) are critical to understand as they can directly impact your farming efficiency and potential rewards. An increase in missing SPs can indicate a disconnection (such as a Firewall in the way on your router, an ISP rotating your IP, NAT switched on or otherwise) or significant distance from a Timelord within your Full Node network (bad nodes connected to you). Essentially, the total number of missing SPs acts as a barometer for the rewards you might be missing out on. Here’s a breakdown of the key points and some strategies for monitoring and mitigating the issue.

Common Reasons for Missing SPs

  • Network Issues: If your Full Node or Farmer cannot receive SPs, it may be due to network connectivity problems.
  • Sync Issues: A Full Node that is out of sync with the blockchain can miss SPs. This might happen because the node is having trouble maintaining a real-time update with the blockchain or because of temporary disconnections - this can also sometimes be due to a slow network card, network port setting such as DNS, or as simple as an SSD being a bit slow being accessed due to CPU or i/o load - consider a fast SSD as an example for your blockchain db.

Monitoring Sync Status

To ensure your node stays in sync, it’s crucial to monitor its status beyond the graphical user interface (GUI), which might sometimes lag or delay in displaying the current state. Here are a few methods:

  • CLI Tools: Use command-line interface (CLI) tools provided by the Chia client to check the sync status more directly and possibly in real-time. Commands like chia show -s and chia farm summary can give you immediate feedback on your node’s and farmer’s status.
  • Log Files: Regularly check the log files generated by your Chia client. These logs can provide insights into any disconnections or sync issues that aren’t immediately apparent through the GUI.
  • Third-party Monitoring Tools: Consider using third-party tools or scripts that can monitor your node’s health and alert you to any issues. These can be particularly helpful for catching and addressing issues quickly before they lead to significant numbers of missing SPs. My preference here is something like Chia-Dashboard | (chiadashboard.com) which uses GitHub - felixbrucker/chia-dashboard-satellite.

Recommendations

  • Network Infrastructure: Ensure your network infrastructure is as reliable as possible. Using Ethernet cables over Wi-Fi can improve stability, but it’s also essential to ensure that your router and internet connection are stable and fast enough to handle the data from the Chia network.
  • Time Synchronization: While not mentioned as a common issue, ensuring your system’s clock is accurately synchronized with a reliable time server can help (ChronyControl - here), as blockchain protocols often rely on precise timing for operations - consider the config/install very carefully and tweak for your system needs - most issues come from time drift.
  • ISP and Internet Delays: While sometimes these factors are beyond your control, it might be worth discussing with your ISP if there are known issues or if there are service options better suited for high-uptime requirements or a static IP address for your router.

Final Thoughts

It’s important to note that some level of missing SPs can be normal, especially considering internet and network variability. However, consistently high levels of missing SPs warrant investigation and action - a sign that something may be wrong with your setup or connectivity. By actively monitoring your node’s sync status through CLI tools, logs, and possibly third-party monitoring solutions, and by ensuring your network infrastructure is as robust as possible, you can minimize the number of missing SPs and potentially increase your farming rewards.

I hope this helps.

How many missing signage points is normal in a 24 hour period?

1 Like

It’s considered normal to miss a few signage points per day due to temporary issues like local network outages or as I indicate above. However, missing fewer than 100 signage points per day is not seen as a serious concern. Given that there are 9216 signage points per day (one every 9.375 seconds), missing up to 9 signage points still means your farm was operational 99.9% of the time, which is considered normal, even for farms with stable internet connections. Consistently missing more than 1% of the signage points, or about 100 per day, should prompt an investigation into the root cause.

I hope that helps.

3 Likes

Analyzing and optimizing a Chia client mainnet\config\config.yaml file ensures that the settings are fine tuned for the best possible performance against Missing Signage Points or Stale Partials and resource usage, without compromising on the security or functionality of the node. Here’s a detailed optimization:

General Optimization:

  1. Network Configuration:

    • Ensure daemon_allow_tls_1_2: false is set to true if compatibility with older versions is not required. This will enhance security by using more up-to-date TLS versions for encrypted communications.
  2. Logging:

    • For log_level: WARNING and similar settings in various sections, consider if you need detailed logs. If debugging is not required, this level is fine. Otherwise, for troubleshooting, you might temporarily set it to INFO or DEBUG.
    • Review log_maxbytesrotation and log_maxfilesrotation settings to ensure they match your storage capabilities and log retention needs. Keeping too many or too large files might consume unnecessary disk space.
  3. Daemon Settings:

    • daemon_heartbeat: 300 seems reasonable. However, if you’re monitoring your system closely and want more immediate feedback, you could lower it, keeping in mind it might increase network traffic and system load.
    • daemon_max_message_size: 50000000 (50 MB) is usually enough for most operations, but ensure it matches your operational requirements, especially if you’re sending large transactions or messages.
  4. Data Layer:

    • client_timeout: 15 is a standard setting. Adjust based on your network stability; increase if you face timeouts due to network issues.
    • database_path and server_files_location should be located on fast, reliable storage, especially if you’re operating on a large scale.
  5. Farmer and Harvester Configuration:

    • For full_node_peers, ensure you’re connected to reliable and fast nodes. This is crucial for timely updates and staying in sync with the network.
    • Consider the number of plots and adjust num_threads in the harvester section accordingly to optimize plot scanning. Too high might lead to unnecessary CPU usage, too low could slow down plot scanning.
  6. Full Node:

    • enable_upnp: true is good for most users; however, if you’re experiencing network issues or have a complex network setup, manual port forwarding might be more reliable.
    • db_sync: auto - Ensure this setting matches your system’s capabilities. On systems with SSDs, setting this to true might improve performance.
    • sync_blocks_behind_threshold: 300 and short_sync_blocks_behind_threshold: 20 are crucial for staying in sync with the network. Adjust based on your network speed and reliability.

High-Speed, Reliable Networks:

  • If you’re on a high-speed and reliable network, the default settings might already be optimal. However, you could slightly increase the sync_blocks_behind_threshold to 500 to give your node more leeway before considering it out of sync, which can be helpful during temporary network or peer issues. Since the default setting for short_sync_blocks_behind_threshold is 20, and considering you’re providing more leeway with the main sync threshold, a proportional adjustment would be beneficial to Increase between 30-40. This adjustment maintains the relative difference between the short sync threshold and the main sync threshold, allowing for minor network fluctuations or short periods of downtime without triggering an immediate sync attempt, which is particularly useful in maintaining the node’s operation without overreacting to brief disruptions.

Moderate-Speed, Occasionally Unreliable Networks:

  • For networks that generally perform well but occasionally have stability issues, consider adjusting the sync_blocks_behind_threshold to around 400 and the short_sync_blocks_behind_threshold to around 30. This adjustment provides a balance between staying in sync and not overreacting to minor network interruptions.

Low-Speed, Unreliable Networks:

  • On networks known for their low speed or frequent disconnections, you may want to be more conservative to avoid your node frequently flipping between synced and out-of-sync states. Setting sync_blocks_behind_threshold to 600-800 and short_sync_blocks_behind_threshold to 40-50 might help mitigate these issues by giving your node more buffer room before initiating sync processes.
  1. SSL Configuration:

    • Regularly review and renew SSL certificates as necessary to maintain security.
    • Ensure private keys are securely stored and access-controlled.
  2. Resource Management:

    • inbound_rate_limit_percent and outbound_rate_limit_percent can be adjusted based on your network’s bandwidth capacity to ensure Chia doesn’t monopolize your internet connection.
    • Adjust reserved_cores under the full node settings if you want to reserve CPU cores for other processes.
  3. Plot Directories:

    • Ensure the directories listed under plot_directories are mounted correctly and have sufficient space. Consider spreading plots over multiple drives to balance I/O.
  4. Wallet:

    • db_readers: 2 might be increased if you’re frequently accessing the wallet, especially with multiple transactions or operations happening concurrently.

Additional Considerations:

  • Security: Regularly review your network and system security. Ensure your firewall rules are up-to-date and only necessary ports are open to the internet.
  • Updates: Regularly update your Chia client to the latest version to benefit from performance improvements, bug fixes, and security patches.
  • Monitoring: Implement monitoring for your node’s performance and network connectivity to proactively identify and resolve issues.
1 Like