27th of December

Farm status
CPU only
Off

Nvidia GPUs
Off

Raspberry Pis
Pi3's running Einstein work. Pi4's running Einstein and Rosetta work.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
I finally got around to updating the storage server to Debian bullseye. It was the last machine needing to be reinstalled.

I still have to update the GPU's in two of the Nvidia machines. They are sitting in boxes on the floor waiting for me to find the time. I was planning on doing that tomorrow but now I have been asked to do something else.

Christmas was hot but things finally cooled down a bit for boxing day so everything was off, even the Raspberry Pis were off! I haven't started up the x64 machines but the Pi's have all resumed crunching duties.

12th of December

Farm status
CPU only
Have been doing Rosetta overnight.

Nvidia GPU's
Two running Einstein GPU work.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta work.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
Rosetta seems to have some work again so I have been running the Ryzen 5900X's machines for it overnight.

I swapped disks in the Chia storage server and have resumed plotting, at least until I run out of disk space again. For news on my Chia farming see Marks Adventures in Chia

Next on my list is to upgrade the two remaining Nvidia GPU machines. I need to swap out the power supply and the GPU's. I have the parts.

5th of December

Farm status
CPU only
Off

Nvidia GPUs
Two running Einstein work.

Raspberry Pis
All running Einstein work.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
The weather has been all over the place. Its been hot, its been wet and its been cool. At the moment its cool so I have a couple of the GPU running. Rosetta doesn't have any work (apart from python tasks) so there haven't been a need to use the CPU only machines. While they could run Einstein tasks on CPU its much quicker to use the GPU equipped machines.


Rumour has it
There is supposedly a 12GB RTX 3060 Ti on the way. There is also talk about a RTX 3060 Super and a re-released RTX 3090 Ti. I am not sure what, if any, changes they will offer.

The RTX 40xx series are expected to be released towards the end of 2022. See This video for a send-up of the 4090 and 4090 Ti. It might not be too far off with suggestions the 4090 may use 500 watts.

27th of November

Farm status
CPU only
Off

Nvidia GPU
Two running Einstein work

Raspberry Pis
All running Einstein work

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
Rosetta ran out of work. I've picked up a couple of resends for failed tasks but that is all they've had for the last two weeks. The farm has been running Einstein work.

One of the computer shops I purchase from got hacked last week so I had to cancel my credit card. I've been waiting for a replacement which is super annoying due to the Black Friday sales being a great time to pick up a bargain and I can't buy anything online at the moment.


Ampere Altra
I fired up the Altra for a couple of hours this weekend. Its super noisy being a 2U server so I have to wait until I have the house to myself before using it. It ran 76 Einstein BRP4 work units in an hour. Two of the work units got a download error and I have reserved two CPU cores for the machine to use which is why it didn't do 80 of them.

I found out the front drive bays are SAS by trial and error. I tried an U2 SSD which wasn't found (but the drive bay power LED came on), then a SATA 2.5" HDD which also wasn't found (no LED's) and finally a 300GB SAS 2.5" HDD which was found and I re-partitioned and re-formatted. Given the capacity I won't be using it in there, its much too small. I will have to purchase some larger capacity drives. Seagate make a 960GB SAS SSD which sells for 750 AUD or for about the same price I can get a 2.4TB (10k RPM) HDD. It has an M2 SSD on the motherboard as the boot drive but it will wear out fairly quickly if I use it for number crunching.

I still haven't tried running Rosetta@home as I would need it to run for 8 hours at least. Einstein also has a Gamma Ray search app that the Pi4's are taking 27 hours to complete. The Altra should be able to do them in around 10-12 hours but I can't leave it running that long at home.

7th of November

Farm status
CPU only
Off

Nvidia GPUs
Off

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
As you can see above most of the farm is off as its been quite warm, although not particularly hot.

The two RTX 3060 Ti's arrived so I need to swap out the remaining GTX 1660 Ti's (and power supplies) when I can find the time.

The 10GbE switch that I ordered months ago arrived. I have swapped the older one out. The old one is going to the other side of the room. The new one has dual 40mm fans and is quite loud when it starts up but reduces speed once booted, however its still annoyingly loud when its around 23-28 degrees.


Alder lake and beyond
Intel now seem to have some faster CPU's that are beginning to compete with the AMD Ryzens. They are rather expensive as they introduce DDR5 and PCIe gen 5 technologies. I don't see any reason to upgrade just yet. DDR5 memory speeds aren't compelling yet and nobody has any PCIe 5 add-in cards or even NVMe SSD's yet. This will of course improve over time.

The i9-12900K and i7-12700K use so much power for the number of cores they provided (up to 230 watts under full load) that I still think the Ryzen's are better at the moment. I generally go for the non-K versions as they aren't as power hungry but Intel haven't announced the rest of the CPU line up.

17th of October

Farm status
CPU only
Off

Nvidia GPUs
Running Einstein and Milkyway

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


RTX 3060 Ti
In the wake of CPL Online cancelling my order for an EVGA RTX 3060 Ti, I went to check their website today. It seems they now have stock and the price has gone up. I think they cancelled my previous order because it was at a lower price and they didn't want to lose money.

They didn't list any quantity limit like the other online computer stores so I ordered two. If they honour this order then I will be able to replace the remaining GTX 1660 Ti's and all the Nvidia GPU machines will be upgraded to 3060 Ti's. I prefer the machines to have the same configuration where possible.


BOINC certificate issues
The Lets Encrypt certificate that a number of BOINC projects use expired. This effects the Windows client as it has a bundled certificate store. Linux uses an OS-wide one. They are now trying to get an emergency release out with the updated certificate bundle. See the main BOINC message board for work arounds until they get a new version out.


Possible time issue
There is a bug with gpsd versions 3.20 through to 3.22. Gpsd translates the messages from GPS's that many NTP hosts use for an accurate time source. The bug is gpsd is expected to jump back 1024 weeks from the 23rd of October 2021.

I raised a bug report with Debian and they have a patched 3.22-4 in buster-backports and bullseye. Note the -4 in the version number. I don't know the status for other Linux distros.

See gpsd issue 144 for more details.

10th of October

Farm status
CPU only
One running Rosetta.

Nvidia GPU
Off.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


RTX 3060 Ti orders
I mentioned in my last blog post that I had another RTX 3060 Ti on order. This week the shop cancelled the order and is meant to be giving a full refund. The other couple of online shops I tried don't have stock of the particular brand/model that I am after.

I went to my usual online computer shop where I got the 3060 Ti's from and it seems they're keeping track of how many you've purchased. While they had stock it wouldn't let me add one to my shopping cart and told me I had reached my limit. I thought about using another email address to place an order but I think they'll be able to tell by the shipping address. They don't ship to post boxes either. I might need to enlist family members to place orders.


Debian point release
Debian did a point release, that is where they save up a bunch of what they consider important fixes and release them. This is their first point release since Bullseye came out. After rebooting my Ryzen 5900X machines the 2.5GbE add-in network card in both machines was no longer detected.

After much messing around with network cables and applying a couple of previous BIOS versions I am back to using the built-in 1GbE network port. Fortunately it didn't effect the Nvidia GPU machines which also have add-in 2.5GbE network cards. The Nvidia GPU machines use Realtek based cards whereas the Ryzen 5900X machines have Intel based cards. The Intel driver is built into the Linux kernel. I raised a bug report.

3rd of October

Farm status
CPU only
Off

Nvidia GPUs
Two running Einstein and Milkyway in the morning.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other bits and pieces
Its been warm and we got a hail storm a couple of days ago. No real damage apart from most of the plants losing some of their leaves, but I did have the house buttoned up.

The two RTX 3060 Ti's have been busy with Einstein and Milkyway work. I haven't managed to try GPUgrid as they don't seem to have any work available.

The computer shop I got the two RTX 3060's from has stock available. I think the prices are so high that people aren't buying (except me) but they are still limiting them to one per household.

26th of September

Farm status
CPU only
Doing Rosetta work overnight.

Nvidia GPUs
Doing Rosetta overnight and Einstein in the morning.

Raspberry Pis
Doing Einstein and Rosetta work.

For news on the Raspberry Pis see Marks Rpi Cluster


3060 Ti news
I have (had) four machines with GTX 1660 Ti's installed. I am trying to get them all upgraded now there is stock available. The RTX 3060 Ti has been working out about 30% faster. I ordered a 3rd 3060 Ti which has already been installed. They use more power, run a bit hotter and there is more fan noise from them.

I ordered my second card 3 weeks ago, from a shop I don't normally deal with. They still have them listed as pre-order. I asked for an update and they replied they don't have any stock and don't know if or when they will be getting any. Meanwhile the price has gone up $200 since I ordered my first one so I will leave it on order and they will have to deal with the price increase. I don't expect the prices will drop any time soon.

I will wait a week or so to see if my regular shop runs out of stock again before placing another order. The shops are limiting GPU sales to one per household so I have to order one at a time which adds to the shipping cost. The other couple of computer shops I deal with either don't sell the EVGA brand or if they do sell them don't have any. I could also use an ASUS Dual model but nobody seems to have them in stock, although they have other ASUS models available.

5th of September

Farm status
CPU only
Off.

Nvidia GPUs
Running Einstein and Milkyway GPU work in the mornings.

Raspberry Pis
Running Einstein.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
As mentioned in my last post I got an EVGA RTX 3060 Ti. I have managed to pre-order a second card from another supplier today. It doesn't say when they will be in stock, however most shops aren't accepting pre-orders until they know they are getting stock. I still need another two cards.

I was informed the 10GbE switch which I have on back-order is not expected until November now.

I upgraded the remaining Nvidia GPU machines to Debian Bullseye after using the 3060 Ti machine as a guinea pig. Everything worked pretty much the same as before apart from updated software packages. Its not so much as an upgrade but rather a clean install of the latest version.

In project news Rosetta ran out of work again. They had 4 million work units in the queue but it seems a particular variant of the work had a bug and crashes within 60 seconds so they have millions of failed work units.

29th of August

Farm status
CPU only
Running Rosetta overnight.

Nvidia GPUs
Running Einstein in the morning and Rosetta overnight.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster

For news on my Chia Plotting see Marks Adventures in Chia


Other news
I managed to buy one EVGA RTX 3060 Ti. Hopefully it will provide a performance improvement over my GTX 1660 Ti's. Its totally overpriced but its from one of my usual computer shops so it comes with a warranty. Now all I need is another 3 to replace the rest of my 1660 Ti's. I need to upsize the power supply in the machine that its going to get the 3060 Ti. I already have four 850 watt power supplies ready to install.

I have now updated both the Ryzen 5900X machines to Debian bullseye. I also updated one of the Nvidia GPU machines without issue so far, apart from all the projects assuming its a new host and having to merge it with the previous one. The exception is CPDN which doesn't allow deleting old hosts, nor merging any with a difference (like OS version).


Update 31 Aug 2021
The RTX 3060 Ti arrived. I swapped the old power supply (550 watt) for a new 850 watt one. I swapped the old GPU out for the 3060 Ti. The old GPU will be sold to offset the cost of the new one.

Running slow

Farm status
CPU only
Off

Nvidia GPUs
Idle. Have been doing some Einstein in the morning.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
Its been quite warm so I've had the CPU only machines off. Partly due to the weather and partly due to Rosetta not having any work. Rosetta now have work.

As mentioned in my last post I have a 10GbE switch on back-order. It is supposedly arriving next week but given all the chip shortages I expect it won't. All of the compute nodes should have at least a 2.5GbE connection but now some machines are back on 1GbE. The new switch should have an SFP+ port so I may be able to use a fibre connection between the old and new 10GbE switches. While the speeds are the same SFP and SFP+ have much better latency than cat6 cabling.


OS upgrades
Debian released bullseye on the 14th of August. I've found a bug where the desktop has high CPU usage if you log into the console as root, at least on amd64 with the mate desktop. I've raised a bug report for it (#991994) although there is no acknowledgement they're looking at it. They might be swamped with issues.

I have upgraded one of the CPU only machines this morning, well it was a clean install really. This gives an updated BOINC client and Linux kernel as well as the various OS updates.

Pi-hole doesn't work properly under bullseye at the moment. You can install it if you disable the OS version checking and then the pihole-FTL service doesn't start automatically. One of their scripts is using the "service restart" command which no longer exists. I changed it to a "systemctl restart" which fixes the restart issue at least. The Raspberry Pi Foundation also haven't updated to bullseye yet. I suspect the pi-hole developers will wait until that happens before they update their code. Pi-Hole was originally written for the Raspberry Pi but is also available for other ARM and x86_64 Linux versions.

I had some issues with squid not being able to create its cache directories, but after purging it and reinstalling managed to get it to work.

7th of August

Farm status
CPU only
Running Rosetta (and now Einstein) overnight

Nvidia GPUs
Running Rosetta (CPU) overnight and Einstein (GPU) in the morning

Raspberry Pis
Pi3's running Einstein. Pi4's running Rosetta and Einstein


Project news
This week Asteroids temporary server died so they are off the air until they can sort out new server hardware. They think it will be a few months before they can get replacements.

Rosetta also ran out of work.

Last week Einstein also ran out of BRP4 work which is what the Raspberry Pis run. They now have work available which is good considering the other couple of projects don't. I have resorted to running Einstein CPU work on the Ryzen 5900's which isn't the most efficient project for them.


Other news
I've ordered another 10GbE switch with more ports. I have an 8 port switch but I have only got one port free on it due to all the down-stream switches. I've ordered a 16 port model and will put the 8 port over on another rack which has mostly storage servers.

My ISP had me on a trial of NBN Superfast (250/15 Mbit speeds) for free. Unfortunately that trial ended at the beginning of this month so my internet speed has dropped back to 50/15. I will have to pay some more money to get higher speed. I'm also thinking of switching to another ISP but have been waiting until I am out of contract.

10th of July

Farm status
CPU only
One Ryzen 5900X running Rosetta 24/7. One running Rosetta overnight.

Nvidia GPUs
Running Einstein and Milkyway in the morning (GPU) and two running Rosetta (CPU) overnight.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
It looks like I might be able to get replacement GPU's soon. I have seem some retailers now discounting the RTX 3060 and the occasional RTX 3060 Ti's appearing for sale. I'm still waiting for the price to drop a bit more. The retailers are still limiting them to 1 per household.

I'm attributing the increase in availability to the China cryptocurrency crackdown and the light hash rate (LHR) that all the newer Nvidia cards have. It has also made the value of the cryptocurrencies drop so its no longer profitable for them.

19th of June

Farm status
CPU only
Running Rosetta overnight.

Nvidia GPUs
Running Einstein (GPU) in the morning and Rosetta (CPU) overnight.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.


Other news
Despite doing Chia plotting (on a dedicated plotter and storage server) the farm is still doing BOINC crunching as well. The CPU only machines were running 24/7 for a while but due to wind and rain (meaning I can't open a window to let the heat out) I have scaled back to overnight/morning operations again. Rosetta also ran out of work earlier this week, along with Asteroids which is my backup project for the CPU machines.

RTX 3060's seem to be readily available from computer shops now, for $1300 (AUD). That is totally over priced so I am not touching them, besides I am after the 3060 Ti which still isn't available. It might be the crack-down on crypto mining by the Chinese government that is helping the supply situation.

6th of June

Farm status
CPU only
Running Rosetta overnight.

Nvidia GPUs
Running Einstein (GPU) in the morning and Rosetta (CPU) overnight.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
Most of my effort at the moment is going into the Chia machine which I am documenting on a separate blog HERE

27th of May

Farm status
CPU only
Running Rosetta overnight

Nvidia GPUs
Running Einstein in the morning (GPU) and Rosetta (CPU) overnight

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
I've decided I might have a crack at Chia farming. I figure I have plenty of disk space so might as well use it. I've taken the Toms Hardware budget PC build as a basis along with an idea from Sloth Tech TV (on YouTube). The parts I have ordered are:

ASUS Z590M-P motherboard
Intel i5-11400 CPU
Noctua CPU cooler

I'm going to repurpose an old case/power supply. I have spare memory and M.2 SSD which will be the boot drive. I also ordered some pre-loved 300GB 10K SAS hard disks and a set of cables off eBay. The idea is the SAS drives will be used for the work space. I already have a SAS controller. The drives only cost $17 each (AUD) plus shipping so I ordered 4. I figure it can do 3 plots at a time with staggered starts so I only need 3 drives with the fourth being a spare. If the SAS drives don't work out I'll look at using a U.2 drive instead. If it turns out to be a flop then I can always use the PC for number crunching.

I've started a separate blog for the Chia farming: Marks Adventures in Chia

 

Update 28 May 2021
First problem. SAS cables arrived and they have the wrong connector. My fault. The SAS controller doesn't come with a printed manual and it doesn't say on the box what connector its got. I had to lookup the manual for the controller to find out it has an SFF-8643 connector. I have now ordered another set of cables.


Update 1 June 2021
Second problem. Replacement SAS cable arrived along with 4 x 300GB SAS hard disks. SAS cable works fine. Two hard disks report multiple errors and cause the controller to refuse to boot the machine. The 3rd drive reports errors when writing. The 4th drive sounds like its doing retries. That is 3 out of 4 drives failed. I contacted the eBay seller and he promptly refunded me for three of the drives. I suspect they got damaged during shipping as they were just wrapped in thin bubble wrap and a layer of cardboard.

Having been turned off pre-loved disks I have ordered a new SAS drive to see if it works with my controller. These days some companies are signing their firmware to make sure you only use their products. If it works I will buy another two and if not I'll be going down the SSD route.

16th of May

Farm status
CPU only
Running Rosetta overnight

Nvidia GPUs
Running Einstein (GPU) and Rosetta (CPU) overnight

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
The weather turned cooler so I have had the GPU's running in the mornings and all the CPUs going overnight.

I'm still keeping an eye out for more up to date GPU's. There seem to be RTX 3060's available from one retailer, however the pricing on them is more than the RTX 3060 Ti. The Ti has more compute units and is what I am trying to get. Maybe with the mining rate limiter on all new Nvidia GPUs availability will improve. What is not known is if it will limit non-mining compute work.


Project news - Milkyway@home
The university network appears to have been hacked and has been locked down while they investigate. The project has been unavailable since the 8th of May. Here is their Twitter post from the 9th of May:

"Information advisory: Rensselaer is investigating a trespass into our university network. We have temporarily suspended access to the network, as we work with law-enforcement and cybersecurity experts to determine the extent of the trespass."


Ampere Altra
I managed to get Debian Bullseye installed. The Debian Buster (current release) installer would lock up after prompting for the language. Debian announced a release candidate version of the installer for Bullseye (their next release) so I tried it. Debian isn't an officially supported operating system so I wasn't surprised that I had issues getting Buster installed.

I still haven't worked out how to reduce the fan noise. The fans spin up to full speed when the machine does its POST, it then reduces the speed a little once it boots. I still need to use hearing protection if I am in the same room.

I have installed BOINC on it and registered the machine at Einstein and Rosetta. I haven't tried any crunching on it yet as Rosetta needs 8 hours and I can't run it for that long. I will try Einstein tasks which typically take 3+ hours on the Raspberry Pis. The Altra is clocked at twice the speed of a Pi4 so might be able to get them done in 90 minutes.

25th of April

Farm status
CPU only
Running Rosetta work

Nvidia GPUs
Running Rosetta (CPU) overnight and Einstein (GPU) in the mornings

Raspberry Pis
Running Rosetta and Einstein 24/7

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
I installed one of the WD SN850 SSD's into the Ampere Altra. I've got it on the shelving unit where most of the Intel machines were sitting. I've powered it up. Its very loud for a 2U machine. While I managed to get it to boot off a USB thumb drive it wouldn't install Debian Buster. I've downloaded the Debian Bullseye image and will try that next. I have also downloaded an Ubuntu server image that I can try. Due to the noise I can't run it when other people are around.

The other two WD SN850's were installed into the Ryzen 5900X machines (one each) and I did a clean install of the operating system and BOINC on them.


Graphics cards
I am keeping an eye on my usual computer hardware shops to try and get RTX 3060 Ti cards. I've been looking for months now. One shop has RTX 3060 (non-Ti) but they want more money for them than the Ti version. They won't take orders unless they have stock and they never seem to have stock on the Ti versions.

Hopefully with Etherium changing to Proof of Stake soon and Nvidia throttling Etherium mining on their cards that should mean the crypominers lose interest, or they will switch to some other flavor of bitcoin. I won't buy off eBay or similar sites as you don't get a warranty and I don't want to support scalping.

10th of April

Farm status
CPU only
Running Rosetta overnight

Nvidia GPUs
Two running Einstein (GPU) and Rosetta overnight (CPU)

Raspberry Pis
Running Einstein and Rosetta

For news on the Raspberry Pis see Marks Rpi Cluster


Bye Bye Intel
The Intel GPU machines have been sold. They were 6 x Intel Core i7-8700 with 32GB of memory. That leaves room on the wire shelving units for the ARM server and moving the 2nd storage server.

I spent the day removing network and power cables from behind and beneath the shelving units. I removed a 16 port switch from the network and the cabling for the Pis has been consolidated to another 16 port switch. I used a lot of zip ties to bundle the network cabling together once the redundant ones had been extricated.

I now need to get another 10GbE switch so I can run the ARM and 2nd storage servers. They both have 10GbE NICs and while I have a couple of free ports on the current switch its on the other side of the room, so I have to run cables and there will be few devices on that shelving unit all wanting 10GbE connectivity.

27th of March

Farm status
CPU only
Off

Intel GPUs
Off

Nvidia GPUs
Off

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
I bought three WD SN850 NVMe SSD's in the 1TB size. Two will go into the CPU only machines (replacing the Sabrent Rocket 4 SSD's) and one is for the Ampere Altra.

The Ampere Altra arrived. I have yet to unbox it. Its a 2U rack mount and weights around 20Kg so its not exactly small or light. I have yet to work out where to put it. I don't have a rack so it will probably end up on a shelving unit. As mentioned above I have an SSD to use as a boot drive and I have downloaded the ARM64 version of Debian and written that to a USB thumb drive, so hopefully it will be a simple process to install it even though there is no standard for booting ARM64 devices (unlike PC's).

11th of March

Farm status
CPU only
Running Rosetta overnight.

Intel GPUs
Off.

Nvidia GPUs
Two doing bursts of Rosetta (CPU) and Einstein (GPU) work.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
The 2.5GbE switch arrived and is now plugged in. I put my two spare 2.5GbE NIC's into the Nvidia GPU machines.

I ordered (and received the following day) more patch cables as I use color coded ones. White for the 2.5GbE cabling to make it easier to work out which port they need to go in.

My order for more NIC's via eBay got cancelled/refunded. Apparently they were returned to the seller as undeliverable due to the mailing label getting damaged.

I made another order for different NIC's via Amazon which cost more but they are based upon an Intel i225 chip. The eBay ones are based on a Realtek chip. When they arrive I will swap some of the NIC's around.

After all this Einstein fixed their scheduler so that it doesn't keep issuing new data files. Apparently it was misconfigured which is why the large downloads. Its back to smaller downloads and I probably didn't need to upgrade the network.

28th of February

Farm status
CPU only
Did Rosetta during the week. Currently idle

Intel GPUs
Off

Nvidia GPUs
Did bursts of Einstein and Milkyway on GPU

Raspberry Pis
Pi3 running Einstein. Pi4's running Einstein and Rosetta


2.5GbE networking
That is 2500 Mbit/sec speeds using existing cat 5e or better cabling.

I've got a 5 port 2.5GbE switch on order. While looking at the QNAP model QSW-1105-5T switch I found that my usual computer shop also has TP-Link model SG105-M2 (5 port) and model SG108-M2 (8 port) 2.5GbE switches awailable.

For NIC's one can pick up an add-in PCIe card on eBay for around 50 AUD. The ones I have been buying are based around a Realtek chip and only need a 1x slot. Once the switch turns up I will install the NIC's in the Nvidia GPU machines.

This was prompted in part by Einstein's gravity wave spotlight search. Each pair of data files is approx 5MB and its downloading a lot of them. The other thing is my ISP has temporarily given me 250/15 Mbit speed (for free) while they trial higher speed internet plans.

23rd of February

Farm status
CPU only
Off most of the last week. Currently running Rosetta.

Intel GPUs
Off

Nvidia GPUs
Off most of last week. Running Einstein (GPU) and Rosetta (CPU) work.

Raspberry Pis
Pi3's running Einstein. Pi4's running Einstein and Rosetta work.

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
I'll be off-loading the Intel GPU machines. They're 8th generation CPU's and Intel are currently up to their 10th generation, with the 11th gen coming out later this year. I will wait to see what Intel have to offer when they finally get to a 10 nano meter process node. I'm inclined to just get two more Ryzen 5900X machines to replace the 6 x Core i7's.

Einstein announced today that they have a working ARM64 app for the M1 Apple computers. The Intel apps wouldn't work properly on the M1 Apple hardware so they have ported one of their apps to run as native ARM code.

I have a few computers that have 2.5GbE network capability. Finding a switch that supports it has been difficult, unless one wants to pay extra for 10G ports. I have resorted to buying a QSW-1105-5T (5 x 2.5G) switch from QNAP because the other network companies don't have anything to offer at this speed.

13th of February

Farm status
CPU only
Running Rosetta

Intel GPUs
Off

Nvidia GPUs
One running Einstein (GPU) and Rosetta (CPU).

Raspberry Pis
Running Einstein and Rosetta

For news on the Raspberry Pis see Marks Rpi Cluster


Altra on the way
I ordered an Ampere Altra Mt Snow machine. I've gone for the 80 core @ 3GHz CPU and 256GB of memory. I'll have to purchase a PCIe 4 NMVe SSD to use as a boot drive. I have some U2 drives that I could use in it.

It only has dual 1GbE network ports built-in. One is a management port and the other is available for normal use. I was going to get a 10GbE NIC but they included it in the purchase price. It will take approximately 4 weeks to arrive.

Australia Day 2021

Farm status
Everything is off. Has been for the last few weeks due to building repairs and painting. They should be finished today so I will look at restarting the farm once the weather cools down tonight. Its currently 34 degrees C.


Ampere Mt. Snow
I got a quote for a Ampere Computing Mt Snow. They're not cheap, but then I didn't expect it to be. I asked for the Q80-30 CPU (80 ARM v8 cores running at 3.0GHz). Its slightly slower than their top of the line model (3.3GHz) but uses less power.

I have to get the quote updated as its only got 128GB of memory which I don't think will be enough for 80 cores, I will bump it up to 256GB. I also need to clarify what networking it has built in, hopefully 10GbE otherwise I will have to add a network card.


Other news
I'm looking to off load the Intel GPU machines from the farm. Hopefully I can get some money for them and maybe I will add another Ryzen 5900X into the mix to make up for the lost CPU cores, although most computing projects are using GPUs these days.

January 2021

Farm status
CPU only
Doing Rosetta overnight

Intel GPUs
Off

Nvidia GPUs
Three running Einstein, Milkyway and Rosetta overnight

Raspberry Pis
Running Einstein and Rosetta

For news on the Raspberry Pis see Marks Rpi Cluster


Other news
I had to downgrade another of the Nvidia GPU machines to kernel 4.19 so I could use its GPU. The "GPL Condom" changes in the 5.9 kernel are causing me grief. I have one Nvidia GPU machine still running the 5.9 kernel which can no longer use the GPU for compute. There is a 455 driver from Nvidia which might work but its in the experimental stage in Debian.

I finally received the 2nd Ryzen 5900X CPU today so I'll be swapping out the second 3900X for a 5900X today.


GPU upgrades
I mentioned in my 19th of December 2020 post that I had plans to upgrade the graphics cards. I bought 4 x Seasonic 850w power supplies as the current machines only have 550w power supplies.

I haven't decided what Nvidia graphics card to get. I am leaning towards the RTX 3060 Ti although there are rumours of an RTX 3060 Ultra. I am waiting on the official announcement so I can compare them.