Those write speeds below 100MBps are quite common on the last ~500GB of free space.
I remember that early plotting was done with SSD buffers between the plotting nvmes and the farming hdds, so that the plotters could resume their work. Ultimately the optimum was to have multiple transfers going out of the plotting servers to different jbods/nas/hdds, from those temp SSDs.
It doesn’t look like GPUs will speed things up too much, so this strategy could be copied.
3 Likes
85 MB/s is about what I get writing to my drives in a DS4246 as well. I imagine you’re doing daisy chaining too?
Anyway your question is a very good one that not many people are thinking about yet. GPU plotting will be closer to 2-5 minutes on pretty reasonable hardware. Even at full HDD write speeds you’ll need to be writing to multiple HDDs in parallel or be okay with your plotter sitting idle part of the time (which isn’t a big deal tbh, you’re still way faster than CPU plotting). A fast SSD buffer drive would help as well (like a few SSDs in RAID0).
2 Likes
I solved my slow write speeds, I was able to achieve 165MB/s to about 150MB/s as the drive filled. I had just converted my farm over to Linux from Windows and my drives were still NTFS, once I formatted my test disk with EXT4 I was able achieve the higher write speeds. So that helps. So when replot 2.0 start, I will convert the rest of my disks to ext4. I should be able to keep up with my two plotters at that speed, until I start playing with the GPU plotter then I will probably have to setup a SSD buffer. Let the fun begin.
2 Likes
Be careful, as Linux will use whatever available RAM, and move those plots first there, thus your mv, etc. command may/will show higher rates than the actual ones (rates of removing file from the src, not transferring it to the dst). I use bpytop, as it shows direct disk io. This is more pronounced toward the last 500 GB or so of free disk space.
The same thing happened to portable plots. I will need to write a script that plots one thing, delete one thing, copy over. Since the size is smaller this time, I will need to check if the deletion is needed. If not, then copy straight.
if you plot 5min/per plot, you need about 8 drives to keep up. Each drive is about 10min to store plot. You need multi-destination or some kind of buffer. I still see no economical point - most kids have no idea what plotting costs.
they plot for 0.5$/plot, which is 20 months ROI…then they add 10gb net infrastructure…SSD cache for 500+ $. 1$/plot…40 months ROI just plot. Meanwhile this plot will be obsolete in about 5 years.
just HDD ROI is 6 years, if you can get one for 15$/TB. It is amusing to watch. How the hell do you plan to repay something in 10+ years if the warranty is 5 years, and possible forced minimum k33 around the corner.
The usual argument is “I do not care about plotting costs…it is nothing compared to farming.” Always amusing to watch. If you waste 50+k and your ROIC is <10%, you do something wrong.
I do it for the fun and entertainment. You don’t buy a car expecting it to die at the end of warranty, same way with a hard drive. Generally if they fail early its due to a defect and it will be under warranty. Otherwise I’ve had them last way past warranty, so it is possible to ROI on them. I would rather have buy re-certified NAS drive then a new shucked usb drive. There is a reason they sell the external usb drives cheap.
With 3 minute plot I see 3 HDs approx balanced w/USB 3.0 or SATA 6Gb/s connection & w/4 HDs all drives r not busy. Are u using USB 2.0? How ur HDs so slow w/5 min plots? Compressed or regular…what K & C?
How do u come up w/these numbers/costs?
Yes write speed on NTFS is terrible under Linux, I found that out as well recently.
Since I am plotting on Ubuntu but farming in Windows, I switched to ExFat and then copy was like 5minutes or so per plot (until the harddisk starts running full).
ExFat is not without problems either (on both windows and Linux) but it worked in the end.
***for those who run into a similar problem:
I had to first create the partition tables in windows, but not format the drive yet (this caused my system to hang on post, not even able to go into bios)
Then format the drives to exfat in Linux.
If I create them directly in Linux, Windows cannot read the partition table correctly.
SATA III HDD 150-280MBs, ~10min to store plot. With 3 drives, you may struggle while disk is full. I have seen speeds as low as 130MBs by some HDDs.
Not to mention I rarely use single drive. Always 4-8 HDDs jbod.
Plotting price is simple to calculate - in same maner as farming. You need to know costs. How else do you earn money?
If you have full RAM plotting rig…costs is your electricity bill + all crap around it. Including HW, unless you can resell it in future.
In hybrid mode…SSD wear, and so on. If you buy 3000$ rig, are you going to ignore this costs…while claiming your plotting is free?
It is same as some genius with solar panels that believes solar is for free…he repaid all, he is finally free so he expands for another 20k $…and call it free. Math genius with 1+1=3 
why not make dedicated Linux farmer? Then you format XFS, and live happily ever after
Windows is like modern princesses…cute, unstable, rarely works when you need it most.
because managing drives in Linux is an absolute nightmare to me. And troubleshooting issues costs too much trouble for a Linux newbie like me (and Linux has plenty issue to troubleshoot, as evidenced by the wide array of Linux forums with people seeking solutions to their problems). Bit of a rant, but I just think the “trouble-free” nature of Linux is overrated by those who use it.
My win 10 farm has been running non-stop without any major issue for more than a year now. And if something does go wrong, at least I have a better chance of fixing it 
Plotting tends to be faster in Linux though, but that might also just be a case of the software being optimized for that OS.
2 Likes
OK glad it is simple for you…so, again, “How do u come up w/these (specific) numbers/costs you quote?” IOW, can you give those calculations? Thanks.
In my 35 years in computer world…99% of problems in Unix/Linux/BSD systems are between keyboard/ground/chair
Whereas Windows has self-screw-upTM function since Win 1.0 where most problems come from within
why do you think it is a nightmare? preparing disc in GUI based distros such as Ubuntu is less painful than Win. Doing it in prompt, few seconds
Just try live CD, you will be surprised…once you get a hang of it, you never go back.
Plotting in Linux is faster because Linux/BSD is built for IO, fast response, max performance. Microsoft just realised we have SSD disks, and they offer now StorageAPI…after almost 15 years ROFL
you know electricity costs no?
Any sane person with sensitive electronics use UPS (it costs 100$ nowadays). You measure consumptions…simple no?
now, you know your SSD has got theoretical maximum TBW…my Corsair MP600 2TB has 3600 TBW…I am at >3700 TBW, and it is still alive 
You know how much plotting uses per plot. You have price per plot for consumables.
now, there is hot discussion about CoO (Costs of Ownership). I bought years ago my workstation for 3000CHF…bought new CPU 500CHF…wasted 600CHF on 128GB RAM. It is my work/fun/plotting rig.
In modern you cannot live without computer anyway…I do CAD/CAM/DTP…for me I always have PC around for “free”. For me CoW is only RAM…it may be sold for at least 300CHF anyway. Your HW did cost you 1000$…still can be resold. Let’s say your CoO is just 300$…10000 plots/300$…and so on, and so on.
My 2nd gen plotting rig plotted ~18min with <50W/plot, 0.03$/SSD wear…final plot <0.05$/plot
If you are full RAM plotting…your costs are HW…I just bought dual CPU server with 512GB for 600CHF + 350 CHF for 3060Ti…let’s say 1000$…both can be still resold for at least 50%…your costs are 500$ per unlimited plots…if you do 10’000 plots, it will cost you 0.05$
Latest experiment with GPU hybrid plotting is <6min.
It is also known as ROIC (Return On Invested Capital)…most talk about ROI (Return On Investment) but they usually have no idea about costs, and just hope XCH skyrockets, and they make killing.
I offload 2 minute plots to an enterprise 4TB SSD drive. From there, plow.py script moves those plots to the HDDs in the JBOD. Transfer speed via SAS is incredibly fast compared to SATA/USB. My buffer drive is unnecessarily large. I probably only need 1TB cache/buffer.
2 Likes
I offload the 1.5min plots to a ~246 GB RAMdrive and from there to two raid0 NVMe(4xNVMe) on the plotter. From there I have enough time to plow.py them to the harvester at 3-5 plots at a time to different HDDs. Not the best solution but works with what I have.
What do you think about a ramdrive ? Would cost about the same and never degrade, and also be faster than an SSD drive.
How big of a ramdrive would you need to keep every new plot there and do without the SSDs ?