You can find used Aruba switches on ebay for $70-80 all day long with 4x 10gb ports + 24/48 1gb ports.
Also for the PCI-E slots, on my rigs I only use a PCI-E 1x card to set them up but run them without a gpu and just remote into them, works great for me.
For reference I used these SPF modules at $43 each and they worked fine:
This converts the SPF port to a regular RJ-45 (ethernet) connector. I have zero experience using any other kind of networking connectors, so to me, networking IS these cat5, cat6, cat7 cables with RJ-45 connectors on the end!
I use Mellanox cards. I can recommend them highly. They got aquired by Nvidia recently. The connect -x 3 and onwards are also compatible with windows 10 with drivers still supported for OS (i think -x 2 is compatible but no officially supported drivers still) and if you get an enterprise version of windows 10 you can do stuff like direct write to disk without involving the cpu. Used Mellanox cards are quite cheap on Ebay and so are used sfp cables (if you want new compatible stuff fs.com has worked fine in my case) and since many datacenters have upgraded to newer versions you can find quite many for sale. Personally I bought two new connect - x 4 for my main servers and then I have 10 used connect-x 3 and they are a mix of sfp 10gb and a 2nd 40gb network. I then have 5GB to my 4 NAS and that is more than enough to keep up with write speeds. Oh and another small tip the sfp to base-T converters can get quite hot so if you use those make sure to space them out. I use a mix of DAC SFP+ and Infiniband for close to switch computers and active SFP+ to further away computers (7m+).
Double-thumbs-up for used Mellanox ConnectX-3’s from Ebay, and generic transceivers from fs.com!
There are a lot of used SFP+ cards that work, and maybe you can save a couple bucks with other models… but ConnectX-3’s work in any OS and only need a PCIe 3 4x slot. Worth it!
And
The SFP copper (inc transducer) interconnects can be found around $10 and the NIC for about $70 used off ebay
All you need to get your first taste of 10g!
Using an Intel X540-T2 dual port from workstation to Synology NAS that also has a dual 10Gbit NIC to send plots off. Then the farm pulls them through another Intel X540-T2 single port down to a local SSD where it caches them ready to push them to the HDDs. That’s all run through a UniFi 10gig switch. Before I switched over to RJ-45 I was using SFP+ (technically QSFP+ with fan out) through Mellanox cards like earlier posters in this thread with DAC to the NAS and a secondary box but a couple of cheap X540’s came up, and so did a UniFi switch. Highly recommend the Mellanox cards and yeah, those converters get a little toasty at times.
I’m using 10Gbps over good quality Cat5e cable - longest cable is 17 meters. It would have been a major job to replace, so I just tried it. Had the 10Gbe network in place prior to Chia, but it certainly makes for quicker transfers (about half the time of 1Gbps), but the bottle neck then becomes how fast the drive can write to the disk.
