Linux 2.5Gbe Ethernet Networking issues for Chia

Moonub · July 14, 2021, 7:28pm

I actually have a Titan Ridge card. But what is the suggestion?

Add Thunderbolt and use Thunderbolt networking ? My second machine does not have TB.

Or use thunderbolt to connect to an external drive and carry that across…? Possible but not very scalable. (Lots of work)

Or use Thunderbolt to connect to a Ethernet adapter? Possible. But I’m then using $90 thunderbolt card and $40 adapter. If it all works it’s a bit of a overkill solution. If I’m going that route I might as well return the TB card and try to find a 10Gbe card that is supported on Linux.

Really still hoping someone with the same problem on Linux can offer advice on how to successfully patch the drivers or Kernel to make this work, ideally if they have Aorus Master x570 in the mix.

At the end of the day this cuts my plotting time in half, but if I have to spend an extra 7 or 10 days plotting, that’s cheaper than spending $100 on more hardware to overcome this.

Chia.Switzerland · July 14, 2021, 7:31pm

Ok but you would be faster if you would just use an external SSD with USB 3.2 Gen2 which allows data transfer of up to 2000 mb/s…

It’s an ethernet network card / adapter and will allow you to connect your pc to your network, other PCs (via their ethernet card etc.), your router and eventually the internet. There might be options below 10G as well. It is basically an external networking card, which you do not put into your PCIe slot but connect via a cable which looks like a USB-C cable. It replaces the regular ethernet option on your mainboard. You could also use both though. For instance, one NIC to connect to the internet and the other one to transfer files in your home network. Since your major problem is the lack of drivers of the network adapter on the mainboard, this would allow you to access your network on linux. Just check if there are linux drivers on the website of the manufacturer before you buy a model.

Theoretically, if the other board has thunderbolt as well, you might even be able to connect them directly just via Thunderbolt. But I know too little about Thunderbolt to verify this. Also cable length should be more crucial here. Long USB / Thunderbolt cables are EXPENSIVE. I mean really fucking EXPENSIVE.

Chia.Switzerland · July 14, 2021, 7:36pm

Did you actually try contacting Realtek and did you check their website for linux drivers…?

Chia.Switzerland · July 14, 2021, 7:46pm

Also, I just found 1G Ethernet Network Adapters for USB at like 30 bucks on the net… Just in case the 10G ThunderBolt solution is too expensive.

2.5 and 5G via USB also exist. Your board has Wi-Fi? Is that recognized under Linus? Depending on your Wi-Fi AP tech you could even hit 2.4 Gbit/s via WiFi. Linux drivers should be available for the Intel WiFi adapters found in most mainboards these days. If you want to upgrade your WiFi adapter

If you want to buy SFP+ transceivers, get them from FS.com. There is prolly one factory in the world which produces these and all brands just put their name and their price tag on it. LOL.

khominhvi · July 14, 2021, 7:50pm

I have the mobo, and am on Linux, so I’ll have a play with it later. But… I’ll need to find another device with 2.5Gbe to try to transfer files to. Maybe one of those SFP+ transceivers that @Chia.Switzerland might be useful but I’ll have a look later. I’ll get back to you when I have more info.

khominhvi · July 14, 2021, 8:08pm

Just remembered i have some 10Gbe RJ45 nics somewhere… . Will see if they can be configured for 2.5Gbe connections first. But thanks for the link.

Chia.Switzerland · July 14, 2021, 8:17pm

If both mainboards have Wi-Fi you could even Wi-Fi Direct them together. This way, slow WiFi AP would not bottleneck and you would not slow down your home network with the traffic at all.

Moonub · July 15, 2021, 2:03pm

I appreciate the sentiment of making these suggestions but trust me , I have looked at them.

Porting data manually by using USB drives and other externals doesn’t scale.

1Gbe cards are cheap, yes, but as I mentioned above, I already have them and they are too slow.

Thunderbolt is an option for peer to peer networking if both machines have it. At best only one does.

SFP+ connections are possible using cards that cost around $40 on eBay and cables that cost around $40. But that again a $120 “more hardware” solution, so not really different to goin any buying a 10Gbe card for the machine currently suffering the lack of a Linux support.

The goal here is a software solution not more hardware used for just a few days.

On the software side:
Yes I have checked that Linux doesn’t support R8125 and yes of course I have gone to get updated drivers from Realtek.

As I have mentioned above the issue I have is not being able to get any of the updated drivers to work. What I am looking for here is someone with actual firsthand experience of solving the issue on Linux with this board

I appreciate the attempts at creativity but I don’t want more ideas about alternatives using more hardware or manual methods. So genuinely thank you for your interest , and let’s stop the brainstorming now.

If someone has first hand experience solving this issue by getting drivers to update and be recognized on a board with the r8125 chipset and ideally on an x570 Aorus Master please shout !

khominhvi · July 15, 2021, 2:05pm

Soooo, as far as I can tell… mine just works?

I spent a couple of hours playing with it last night and this is what I learned.

Firstly, I am running Ubuntu 20.04 LTS and kernel 5.8, which can be found by running uname -r.

Running lspci -k I found my card was listed and loaded properly.

Now, I can’t remember if I already loaded driver r8125 or if it was originally r8169, I didn’t do a good job of keeping track of my changes. The driver you need is r8125.

Apparently the RTL8125 2.5GbE controller isn’t natively supported until kernel >5.9. For anyone with less the kernel 5.9, you can install the driver provided by Realtek , read on.

If you don’t have the kernel driver r8125 you can download it from here: Realtek Drivers
And download: 2.5G Ethernet LINUX driver r8125 for kernel up to 5.6

Then, follow the instructions in the README to install the driver, which is just a script.
sudo ./autorun.sh

I think I rebooted here. Can’t remember.

Check the driver is loaded (lspci -k), then continue the setup in your network manager.

After setting up my connection in the Network Manager, running ifconfig I found my interface:

And running ethtool <interface> I could see my ethernet controller’s capabilities.

As you can see it shows it supports 2500baseX/Full capabilities. But unfortunately, the 10GbE RJ45 card I have doesn’t support 2.5GbE, so I couldn’t test it. It just auto-negotiated the fastest available speed it could do (1000Mb/s).

I did however connect it to my gigabit ethernet switch, and I have internet and network access. No problems there. And running an iperf3 via the network to my unraid server shows decent gigabit transfer rates.

I did run a file transfer that did it at these speeds, so no problem there either.

Thats all I can do for you for now, I have ordered anotheter 2.5GbE card on amazon, but it doesn’t arrive until tmr. At which point I can run more tests and see if it actually runs at 2.5G speeds.

References:

Voodoo · July 15, 2021, 2:30pm

(you might have already done this)
I think if you are looking for Linux software/hardware support, you’d better also ask in a Ububtu/Linux forum. A lot of people here on Ubuntu, but many like myself are quite green with it

aschen · July 15, 2021, 3:47pm

You have 2 NICs on this board, do you have 2 NICs on the destination machine? You could use both in parallel if so - easiest way to do this is to direct connect the machines with 1 cable per pair of ports, set up each connected pair of adaptors to use a a different subnet and give the destination multiple IPs/hostnames and then run rsync jobs in parallel from the source machine pointing at the different (virtual) destinations.

Parker1c · July 16, 2021, 1:14am

Thats slow for 1gbe. I have my madmax plotter connected to 1gbe and do under 1000s transfers. Windows plotter to switch to Windows full node/farmer.

Never any issues with my onboard 2.5gbe realtek(in Windows) (Msi Meg Ace x570) I believe you need newer kernel in ubuntu for 2.5gbe.

And 2.5gbe is odd duck. It is 60-80% the price of 10gbe if buying new nics(but 1/4 the speed if 10gbe). If buying used or new old stock/or China clones. 10gbe is often cheaper. And not many 2.5/5/10gbe switches… But lots of 1/10gbe switches.

khominhvi · July 16, 2021, 2:50pm

Ok, so.

I am using the Aorus X570 Master with AMD Ryzen 3900x. (Bios Version: F31e)
My RTL8125 2.5GbE Controller loads correctly in Ubuntu 20.04
Driver from Realtek website: r8125
I now have a PCIe RTL8125 2.5GbE Controller in my server which also works.
I have connected the two with a regular Cat6 ethernet cable and the controllers can communicate.
The client (Aorus X570 Master) has been given a static IP of 10.0.0.1 (netmask 255.255.255.0)
The server’s 2.5GbE port has been configured with a static IP 10.0.0.2 (netmask 255.255.255.0)

Here are the results:

Network manager shows that it connected, 2500 Mb/s.

ethtool confirms 2500Mb/s connection and controller capabilities.

iperf3 show 2.3Gbits/sec bitrate. Which is pretty good.

A 5GB file transfer confirms the connection works.

As you can see, it works without issue and getting more than twice the speed than my previous post (at 1000mb/s).

So, I can confirm, that the RTL8125 2.5GbE Ethernet Controller on the Aorus Master X570 works with Ubuntu 20.04, on kernel 5.8.

As a sanity check, I also loaded stock 20.04 and 21.04 to USB boot drives and ran the same tests.

As you can see, even with the r8169 drivers installed the RTL8125 controller works flawlessly.

My guess is you haven’t set up your connection properly (with static IP addresses), or your server/switch doesn’t support 2.5GbE, or your bios might need an update. I remember in the early days this motherboard needed a bios update to get the RTL8125 working properly. Maybe that is your issue. Either way, I think we can rule out the OS or driver problems.

Moonub · July 17, 2021, 3:40pm

Hi Aschen

This is interesting. I haven’t tried that. I looked at solutions before for “teaming” which I think means the two connections merge and act as one, but that wasn’t supported. I haven’t attempted setting up two completely separate connections. I wonder how that would work for file transfers. Hmm.

Moonub · July 17, 2021, 3:42pm

Brother, thank you. Let me try to recreate everything you have done and get back to you.

Confident this isn’t a bios issue as it runs fine in windows.

Thanks for your help here. Will reply soon.

Chia.Switzerland · July 17, 2021, 4:01pm

Do not use GUI on servers unless you want to get hacked easily.

khominhvi · July 17, 2021, 4:23pm

Errr ok… Everything you see in my post is from client side.

Chia.Switzerland · July 17, 2021, 4:25pm

Ok. Just wanted to point out that GUI is a huge security risk.

kreaninw · July 17, 2021, 5:22pm

@Chia.Switzerland

Can you explain this in detail?

As I fas as I know, GUI sits atop CLI. Therefore they should be equal security-wise.

aschen · July 17, 2021, 5:54pm

Yep, you could set up bonding/link aggregation - you’d do that typically for convenience (so that there only appears to be one connection), but it’s more complicated to set that up than it is to set up 2 independent connections, however with 2 connections you’d have to partition your files into 2 sets manually and send half down one connection and half down the other, e.g. run 2 rsync jobs at a time.