If you’d like to restart your farmer without interrupting your plotting, you can use the powershell alias listed here to make it a lot easier, just open powershell, paste the alias, then run chia start -r farmer: Adding chia.exe to your PATH on Windows for easy use
As to what’s causing this (if your logging level is set to INFO) check your logs for large blocks of Don't have challenge hash <hash here>, caching EOS. I noticed I was getting roughly 2 minute blocks of these and losing sync for those 2 minutes. After changing out my database for a backup that I made using sqlite vacuum, that stopped happening. The vacuumed database seems to have improved the performance of the node.