30th of May

Farm status

CPU only

Running Rosetta work.

Intel GPUs

All running Rosetta work.

Nvidia GPUs

Two running Rosetta overnight.

Raspberry Pis

Two running Rosetta. Six running Einstein.

For news on the Raspberry Pis see Marks Rpi Cluster

Other news

I made it into the top 100 participants for Rosetta. As of this morning I was in 86th place.

The extra network cables arrived so I have used some on the machines that have multiple network cards. They now have two connections into different switches. I also had a 1M cable running between two switches and it was pulling one end so that got replaced by a longer cable.

I had a look around for some simple 2.5GbE switches and managed to only find one from D-Link. Most others have what they call multi-gig ports where it supports 1, 2.5, 5 or 10GbE speeds. The switches I am using at the moment have 8 x 1GbE ports and 2 multi-gig ports, one of which gets used as the up-link. You think there would be something like a 4 x 2.5GbE ports with a high speed up-link, or even an 8 port version.

Farm makeup

At the moment the compute portion of the farm consists of:

Type	Number	CPU	GPU	Memory
CPU only	1	AMD Ryzen 3900X (12C/24T)	GT710	64GB
Intel GPU	6	Intel i7-8700 (6C/12T)	Built-in	32GB
Nvidia GPU	4	AMD Ryzen 3600 (6C/12T)	GTX 1660 Ti	32GB

That gives a total of 72 cores/144 threads and 384GB of memory. There is another Ryzen 3900X on order.

23rd of May

Farm status
CPU only
Running Rosetta work.

Intel GPUs
All running Rosetta work

Nvidia GPUs
Two running Rosetta work

Raspberry Pis
Two running Rosetta. Six running Einstein work.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
The 64GB memory kits arrived. I've installed one kit in the CPU only machine and removed the 4 sticks of 16GB that it had. They've been reinstalled into the remaining Nvidia GPU machines. That brings all the 6 core/12 thread machines up to 32GB of memory. The 64GB kits have RGB lighting which I find rather annoying.

The other CPU only machine that I ordered is delayed. The motherboard seems to have been back ordered and keeps changing dates as to when it will arrive.


2.5GbE networking
The 2.5GbE network cards arrived. They only use a PCIe gen3 x1 slot but when I plugged it into an x1 slot the kernel started spamming the console with messages about PCIe bus errors. After swapping it to a free x16 slot it worked fine, so probably something to do with how the PCIe slots are wired up.

The first card went into the proxy server and the second card is in the CPU only machine. I still need to try some speed tests between them and the 10GbE machines (the storage servers).

I used to have the gigabit ports bonded so they share the network traffic but have now switched to running the 2.5 gigabit card as primary and the motherboard (gigabit) port as an active backup.

Debian don't have a driver for the Realtek 8215. Support for the chip was added in August 2019 but the Debian firmware-realtek package is from July 2019. I ended up getting the driver directly from the Linux kernel web site.

I've ordered some more of these EDUP cards off eBay. The original seller increased the price to 69 AUD but I found another seller with them listed in GBP which worked out to 60 AUD each. I only need one more for the next CPU only machine.

Running two network ports per machine now means I need more network cables. Something else to buy.

Update: 26 May 2020
I managed to get one of them to do around 2.5Gbit. According to iperf3 they were doing 2.27Gbit. That was between a 10Gbit and through a few switches into a machine with a 2.5Gbit card. I tested in both directions.

16th of May

Farm status
CPU only
Running Rosetta work

Intel GPUs
All running Rosetta work

Nvidia GPUs
One running Rosetta work

Raspberry Pis
Two running Rosetta work. Six running Einstein work.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
The farm continues to concentrate on Rosetta work on as many cores as possible (within power limitations).


Another Ryzen 3900X build
I ordered parts for another Ryzen 3900X system as they are on sale. Its the same specs as before. The power supply has already arrived. It came from a different supplier. The remaining parts should arrive next week.

The 64GB memory kits (2 x 32GB sticks) came back into stock so I ordered two kits, one for each of the Ryzen 3900X builds. These are from yet another supplier.

The PCH fan on the current Ryzen 3900X system is not spinning. It doesn't seem to be causing any problem at the moment. Its part of the motherboard so I might have to drop the machine off for repair. I would expect the shop would have to replace the motherboard as its soldered on. I will wait until I have built the 2nd Ryzen 3900X before doing this.


2.5GbE networking
I found some EDUP network cards on eBay for 55 AUD each. They have a Realtek chip. I bought two. They are coming from China so I wouldn't be surprised if they fail after a few months or they are really 1GbE network cards. I should be able to get my money back if either happens.

I was thinking of putting one in the proxy server and another in the Ryzen 3900X build. Now that I have a second Ryzen 3900X on order I will probably put one in each of them. I will wait and see how they work before buying any more.

10th of May

Farm status
CPU only
The Ryzen 3900X is running Rosetta work

Intel GPUs
All running Rosetta work

Nvidia GPUs
Off

Raspberry Pis
Two running Rosetta work. Six running Einstein BRP4 work.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
Most of last fortnight was taken up with the Ryzen 3900X build, getting it going and doing a burn-in on different projects.

I spent a little time on the proxy server. I put a spare 10GbE network adapter into it which didn't make any noticeable difference. The hardware side was simple, however changing the bonding to adaptive failover took most of my time. There are examples on the internet and most of them don't work any more. Rosetta changed their project comms to HTTPS so that means we cannot cache downloads for them. In the end I went back to the dual 1GbE network adapter that I have used for the last year and bonded the ports.


2.5GbE networking
I did a bit of research into 2.5GbE network adapters. There are a few (mostly based upon Realtek chips). Intel has an i225 chip but it has a bug where the inter-packet gap is incorrect and it requires a respin of the silicon to fix. Aquanta also had one but seem to sell it for more than the 10GbE network adapters.

None of the computer stores are selling them in Australia. There are USB 3 to 2.5GbE external adapters available. I found one PCIe card on Amazon coming from the US. It was 87 AUD. There was confusion over its warranty with Amazon stating it had an "International warranty". Of the 10 reviews two people complained their card failed after 3 months.

Ryzen 3900X build

In my previous post I mentioned the Ryzen 3900X build that I was going to do. All the parts arrived and I put it all together. I had a couple of issue.

The first issue was with the thermal paste squirting over the side of the CPU which I cleaned up as best I could.

The second turned out to be the additional power connectors on the motherboard. I had plugged both of the aux 12 volt power connectors and it refused to POST. The fans wouldn't even spin up. I thought that maybe the thermal paste had shorted something out. Apparently Noctua thermal paste is non-electrically conductive. After checking one of my other Ryzen builds I unplugged the 2nd aux 12 volt power connector and that got it going.

Rather than taking pictures of the computer I thought I would share the BOINCtasks view of it running 23 Rosetta tasks. I've left one thread free for the operating system/other programs.

And how top sees the same workload.

I wanted to get a 64GB memory kit (2 x 32GB) for it and then that will allow me to put the existing memory into the Ryzen 3600's which is what it was intended for. Unfortunately the supplier selling the 64GB memory kit doesn't have any stock. Its G.Skill Trident Z DDR4-3200 memory which I haven't used before.

Another upgrade I would have liked to do was a 2.5GbE network adapter but none of the computer places seem to sell them in Australia. I could get 10GbE but they cost 180 AUD. In the mean time I will just put a spare 1GbE adapter in it and bond the ports together.

I might build another one of these because the price on the Ryzen 3900X has dropped again, in fact all of the Ryzen 3000 series seem to be on sale at the moment. They might be clearing stocks before the Ryzen 4000 desktop CPUs come out.