Farm Status
Intel GPUs
All running Seti work
Nvidia GPUs
Running Seti work
Raspberry Pis
Running Einstein BRP4 work
Transition to Linux
The last two machines were converted over to Linux without any issues. I simply follow the process I documented before. That means there are no Windows crunchers in the farm now.
Other news
The same person that purchased all the motherboards also wants to buy the i7-5820K machines. I decommissioned them and removed the GTX1060's on Friday. I will be replacing them with a pair of AMD Ryzen 1700 machines. The GTX1060's will go into the Ryzen's when I get them.
I have also ordered another pair of GTX1060's. These will initially be used to replace the GTX970's. Longer term I will probably replace the i3's with another pair of Ryzen machines.
My original idea was to have a low powered CPU (the i3-6100T is only 35 watts) and a pair of high performance GPU's. Unfortunately the i3 doesn't have enough PCIe lanes to run two GPU's at x16 speed. It seems to provide one slot at x16 and the second at x4 and this is on a Z170 chipset motherboard. The i3 does have built-in graphics and the motherboard is mATX sized so I may be able to use them for some other purpose.
30 April 2017
23 April 2017
23rd of April
Farm Status
Intel GPUs
All running Seti overnight
Nvidia GPUs
Two running Seti overnight. Other two are off.
Raspberry Pis
All running Einstein BRP4 work
Other news
The WD Red hard disks turned up. I expect the Drobo that they are destined for will arrive late next week.
I also sold off a couple more GPU's and the last remaining motherboard.
Now that I have sold off most of the excess hardware I will look at replacing the i7-5820K machines (6 core/12 thread) with AMD Ryzen's. I'm looking at the Ryzen 1700 (3Ghz) but also only 65 watts. The motherboard would be an ASUS Prime X370 with 32GB of DDR4 memory. I will keep the GTX1060's that are in there as they are the most up to date GPU's that I have.
Transition to Linux
In the blog post prior to this one I covered the steps used to converted two of my Nvidia GPU machines to Linux. They can run CUDA, OpenCL and CPU work now. I have two more to convert and that is all the farm converted to Linux apart from two windows-based machines. One is the file server and the other is a laptop that I use to monitor the farm using BOINCtasks.
Intel GPUs
All running Seti overnight
Nvidia GPUs
Two running Seti overnight. Other two are off.
Raspberry Pis
All running Einstein BRP4 work
Other news
The WD Red hard disks turned up. I expect the Drobo that they are destined for will arrive late next week.
I also sold off a couple more GPU's and the last remaining motherboard.
Now that I have sold off most of the excess hardware I will look at replacing the i7-5820K machines (6 core/12 thread) with AMD Ryzen's. I'm looking at the Ryzen 1700 (3Ghz) but also only 65 watts. The motherboard would be an ASUS Prime X370 with 32GB of DDR4 memory. I will keep the GTX1060's that are in there as they are the most up to date GPU's that I have.
Transition to Linux
In the blog post prior to this one I covered the steps used to converted two of my Nvidia GPU machines to Linux. They can run CUDA, OpenCL and CPU work now. I have two more to convert and that is all the farm converted to Linux apart from two windows-based machines. One is the file server and the other is a laptop that I use to monitor the farm using BOINCtasks.
16 April 2017
Setting up a Linux CUDA cruncher
Rather than using an outdated Ubuntu which seems to be the current suggestions, this is what I used with Debian. This will get you the latest Debian (Jessie), latest kernel (4.9) and the 7.6.33 BOINC client. You should be up and running with a CUDA and OpenCL capable machine after doing this.
Part 1 - Install Debian
I used the Debian 8.7 net install for this. You’ll need a thumb drive or a blank CD. Download Debian from http://www.debian.org/distrib/ and write the ISO image to CD or thumb drive.
Boot off the thumb drive or CD. It will start up the Debian installer
Install Debian. It will ask a bunch of questions like language, country, etc and prompt for partitioning your disks. When it asks for software to install select SSH server and whatever desktop you prefer and remove all other selections. Once done it will prompt for a reboot.
Part 2 - Install Nvidia software
Login as root, open a xterm window and type the following commands:
cd /etc/apt
nano sources.list (nano is a text editor)
Change “jessie main” lines to “jessie main contrib non-free” and add a jessie-backports line. It should look like this when you're done. I'm using httpredir as it will pick the fastest server.
deb http://httpredir.debian.org/debian/ jessie main contrib non-free
deb http://security.debian.org/ jessie/updates main contrib non-free
deb http://httpredir.debian.org/debian/ jessie-updates main contrib non-free
deb http://httpredir.debian.org/debian/ jessie-backports main contrib non-free
Exit out of nano and save the file (Control-O followed by Control-X)
apt update
apt install –t jessie-backports firmware-realtek (if needed). Most of my motherboards have Realtek chips.
apt install –t jessie-backports linux-image-amd64
apt install –t jessie-backports nvidia-kernel-dkms nvidia-smi nvidia-xconfig
apt install –t jessie-backports nvidia-opencl-icd (if you want OpenCL support)
nvidia-xconfig
Part 3 – Install BOINC
apt install –t jessie-backports boinc-nvidia-cuda boinc-manager
sync
reboot
Part 1 - Install Debian
I used the Debian 8.7 net install for this. You’ll need a thumb drive or a blank CD. Download Debian from http://www.debian.org/distrib/ and write the ISO image to CD or thumb drive.
Boot off the thumb drive or CD. It will start up the Debian installer
Install Debian. It will ask a bunch of questions like language, country, etc and prompt for partitioning your disks. When it asks for software to install select SSH server and whatever desktop you prefer and remove all other selections. Once done it will prompt for a reboot.
Part 2 - Install Nvidia software
Login as root, open a xterm window and type the following commands:
cd /etc/apt
nano sources.list (nano is a text editor)
Change “jessie main” lines to “jessie main contrib non-free” and add a jessie-backports line. It should look like this when you're done. I'm using httpredir as it will pick the fastest server.
deb http://httpredir.debian.org/debian/ jessie main contrib non-free
deb http://security.debian.org/ jessie/updates main contrib non-free
deb http://httpredir.debian.org/debian/ jessie-updates main contrib non-free
deb http://httpredir.debian.org/debian/ jessie-backports main contrib non-free
Exit out of nano and save the file (Control-O followed by Control-X)
apt update
apt install –t jessie-backports firmware-realtek (if needed). Most of my motherboards have Realtek chips.
apt install –t jessie-backports linux-image-amd64
apt install –t jessie-backports nvidia-kernel-dkms nvidia-smi nvidia-xconfig
apt install –t jessie-backports nvidia-opencl-icd (if you want OpenCL support)
nvidia-xconfig
Part 3 – Install BOINC
apt install –t jessie-backports boinc-nvidia-cuda boinc-manager
sync
reboot
14 April 2017
Good Friday 2017
Farm status
Intel GPUs
All running Seti work
Nvidia GPUs
Did a couple of GPUgrid work units earlier in the week. Currently they are off.
Raspberry Pis
All running Einstein BRP4 work
Other news
I have ordered three 4TB WD Red hard disks for the Drobo 5N2. The Drobo isn't due here until the end of the month. The price per terabyte increases with the larger capacity drives. Two 4TB drives are still cheaper than one 8TB drive. Seagate have a 12TB drive but they cost over a thousand dollars.
A couple of the Pi's were playing up so I had to re-image them. Unfortunately when attaching them to the Einstein project they decided the program executable was corrupted and then trashed their work units. A project reset in BOINC fixed the corrupt program. It did this on both of them so I think the checksum value given for the executable is probably wrong rather than a download issue.
I've rescheduled the ADSL connection for the 26th of April so hopefully it will get installed this time. The last technician didn't have a key to the communications cupboard.
Linux transition
The Intel GPU's are doing work faster than when they were windows machines. All 8 of them are churning through Seti work. I am trying to get the credits similar for the 3 projects they've been set up to run (Asteroids, Einstein and Seti).
One of things to try on the transition to Linux is getting the Nvidia GPU's working. Debian have Nvidia drivers in their repository so hopefully they will work. Most of the guides I have seen install older versions of Ubuntu (14.04) or Mint which is based upon Ubuntu.
I also want to look at running CPDN jobs however their programs are all 32 bit so that usually causes problems when running on a 64 bit Linux platform.
Intel GPUs
All running Seti work
Nvidia GPUs
Did a couple of GPUgrid work units earlier in the week. Currently they are off.
Raspberry Pis
All running Einstein BRP4 work
Other news
I have ordered three 4TB WD Red hard disks for the Drobo 5N2. The Drobo isn't due here until the end of the month. The price per terabyte increases with the larger capacity drives. Two 4TB drives are still cheaper than one 8TB drive. Seagate have a 12TB drive but they cost over a thousand dollars.
A couple of the Pi's were playing up so I had to re-image them. Unfortunately when attaching them to the Einstein project they decided the program executable was corrupted and then trashed their work units. A project reset in BOINC fixed the corrupt program. It did this on both of them so I think the checksum value given for the executable is probably wrong rather than a download issue.
I've rescheduled the ADSL connection for the 26th of April so hopefully it will get installed this time. The last technician didn't have a key to the communications cupboard.
Linux transition
The Intel GPU's are doing work faster than when they were windows machines. All 8 of them are churning through Seti work. I am trying to get the credits similar for the 3 projects they've been set up to run (Asteroids, Einstein and Seti).
One of things to try on the transition to Linux is getting the Nvidia GPU's working. Debian have Nvidia drivers in their repository so hopefully they will work. Most of the guides I have seen install older versions of Ubuntu (14.04) or Mint which is based upon Ubuntu.
I also want to look at running CPDN jobs however their programs are all 32 bit so that usually causes problems when running on a 64 bit Linux platform.
08 April 2017
8th of April
Farm status
Intel GPUs
3 running Seti work. The rest are off.
Nvidia GPUs
All off
Raspberry Pis
All running Einstein BRP4 work
Other news
I sold off one of the ASUS P6T's. The other one and a GTX670 are on eBay ending tomorrow. I also sold off some Seasonic power supplies. I have interest in the remaining motherboards and a couple of GTX750Ti cards. That still leaves a bunch of other parts and the two i7-5820K system to sell off.
I had a phone company technician come out to install a second ADSL line. Unfortunately he didn't have a key to the communications cupboard. Its meant to be a standard set for phone companies but it seems they don't come prepared.
I'll put up a few more graphics cards on eBay next week to get rid of them.
I've ordered a new Drobo 5N2 to replace my Drobo 5N with. I need to order some larger WD Red drives to go in it so it can backup the storage server. Once its up and running I will sell off the 5N.
Intel GPUs
3 running Seti work. The rest are off.
Nvidia GPUs
All off
Raspberry Pis
All running Einstein BRP4 work
Other news
I sold off one of the ASUS P6T's. The other one and a GTX670 are on eBay ending tomorrow. I also sold off some Seasonic power supplies. I have interest in the remaining motherboards and a couple of GTX750Ti cards. That still leaves a bunch of other parts and the two i7-5820K system to sell off.
I had a phone company technician come out to install a second ADSL line. Unfortunately he didn't have a key to the communications cupboard. Its meant to be a standard set for phone companies but it seems they don't come prepared.
I'll put up a few more graphics cards on eBay next week to get rid of them.
I've ordered a new Drobo 5N2 to replace my Drobo 5N with. I need to order some larger WD Red drives to go in it so it can backup the storage server. Once its up and running I will sell off the 5N.
02 April 2017
2nd of April
Farm Status
Intel GPUs
All except one currently running Seti work. They were running Asteroids work before.
Nvidia GPUs
All off.
Raspberry Pis
All running Einstein BRP4 work
Farm news
I've changed all except one of the Intel GPUs over to Linux. That's 7 of them. The last one had an SSD + HDD which I had to swap out with the proxy servers SSHD. Its now the same configuration as all the other Intel GPU machines and currently installing Linux. I had to rebuild the proxy server.
I found that installing Beignet (Intels OpenCL drivers for Linux) doesn't work if you are on the 3.16 kernel. It will install but clinfo is unable to detect the device. Updating the kernel to a 4.x one seems to have resolved that. I can now use the iGPU but it slows down the CPU so is not normally worth the effort.
For the moment I am leaving the Nvidia GPU machines running Windows but plan on selling off the two GTX970's and also the two 6 core/12 thread machines which currently have GTX1060's. I'm going to keep the GTX1060's but will need to put something else in there in order to sell them. I will need to investigate running the GPUs under Linux and what driver combinations work.
The computer shop rang regarding the storage server. They have received it from their supplier but it seems to only have SAS expanders for each row of drive bays. Its got 12 drive bays but each set of 3 has its own cabling. The hard disk controller has only a single SAS connector meaning it can only run 3 drives. I'll have to chase them up next week. It seems the shop don't have experience with storage servers.
Intel GPUs
All except one currently running Seti work. They were running Asteroids work before.
Nvidia GPUs
All off.
Raspberry Pis
All running Einstein BRP4 work
Farm news
I've changed all except one of the Intel GPUs over to Linux. That's 7 of them. The last one had an SSD + HDD which I had to swap out with the proxy servers SSHD. Its now the same configuration as all the other Intel GPU machines and currently installing Linux. I had to rebuild the proxy server.
I found that installing Beignet (Intels OpenCL drivers for Linux) doesn't work if you are on the 3.16 kernel. It will install but clinfo is unable to detect the device. Updating the kernel to a 4.x one seems to have resolved that. I can now use the iGPU but it slows down the CPU so is not normally worth the effort.
For the moment I am leaving the Nvidia GPU machines running Windows but plan on selling off the two GTX970's and also the two 6 core/12 thread machines which currently have GTX1060's. I'm going to keep the GTX1060's but will need to put something else in there in order to sell them. I will need to investigate running the GPUs under Linux and what driver combinations work.
The computer shop rang regarding the storage server. They have received it from their supplier but it seems to only have SAS expanders for each row of drive bays. Its got 12 drive bays but each set of 3 has its own cabling. The hard disk controller has only a single SAS connector meaning it can only run 3 drives. I'll have to chase them up next week. It seems the shop don't have experience with storage servers.
Subscribe to:
Posts (Atom)