Build your own deep learning server

Note: This article focuses on server setup and working with multiple users simultaneously, part assembly and software installation processes
Slav IvanovA simplified version of the article about creating your own DL Box.

I just started reading part 1 of the Fast. Ai course, “Practical Deep Learning for Encoders,” and I wanted to build my own server and upgrade the model with AWS P2 instances and storage. Realizing that I would be using a larger data set and didn’t want to wait hours to train my model for lack of sufficient processing power, building my own DL platform was a good choice because it would save me a lot of time and money in the long run, and also gave me great experience putting servers together.

build

Creating a part list

Before putting together any list of parts, you need to determine what you really want from your machine. For me, WHAT I want is:

I use pcPartpicker.com to point everything out because it makes it easy to compare other parts and organize everything, as well as their very good part compatibility checkers (more on that later). I’ll go into detail about why I chose each part and how it works with the overall system. For reference, you can find a link to my server build here.

GPU: GTX 1080 Ti Hybrid

When you train models with graphics cards, this is the most important part of the build, so the more powerful the GPU, the faster you can process large data sets. Higher GPU memory equals higher processing power (e.g., faster training, larger batch sizes, etc……) . I chose two of these cards for my server because my budget had enough room, and it allowed me to train my model on one card and let another user train their model on the second card. You can narrow down the type of GPU you buy to fit your budget (1070, 1060, etc.) while preventing other components from hitting bottlenecks. Hybrid 1080 Gpus are also nice because they come preloaded with aiO water cooling in addition to the normal fan cooling system that most Gpus have. The 1080 Ti heats up at full load, so high quality cooling must be maintained to extend the life of the card and maintain its performance when training the model. For more information on choosing graphics cards, this article really helped me understand what choices I should make when working in a deep learning environment.

CPU: AMD Threadripper 1900x

When you’re using a graphics card to train a neural network, your CPU is still important because it’s used for things like data preparation, so these high core counts will help speed things up. I used Treadripper because it was a very new CPU on the market and had a high number of cores (32 cores for TR2!). They cost much less than similar Intel products. 1900X was the first to get the lowest tier of TR, with only 8 cores, but my overall goal for this server was to keep it scalable.

One thing to note is that when you choose your CPU, make sure your graphics card has 8 or 16 pcie slots available, because that’s when the graphics card will perform best under load, or your system could hit a bottleneck. On high-end cpus, if you have four graphics cards on your server, you are almost guaranteed to have enough pcie slots.

The main board
: MSI X399 SLI Plus

Choose this motherboard because it is a full ATX board with space for 4 Gpus, and a maximum of 128GB of RAM. As mentioned earlier, one of the primary goals of this server is to keep it highly scalable.

memory
: 32GB Corsair Vengeance LPX DDR4 (2 x 16GB)

The more memory you have, the easier it is to process large data sets. The next thing I was planning to upgrade was to add 2 more 16GB RAM, which is why I didn’t get four channels of RAM (4x8GB OF RAM), although that would have improved my performance a bit.

Storage: 256GB Samsung SSD and 2TB hard drive

I put Ubuntu, all my libraries, and the current data I use on SSD, and all the other data I store on a 2TB mechanical hard drive.

Cooler: Corsair H100i V2 liquid cooler

Threadrippers doesn’t have an inventory cooler (and if you have a $30 backup fee, you should always upgrade to at least one cheap aftermarket cooler), so I wanted to use tools that were reliable, all-day, inexpensive and easy to maintain. The one-piece coolers are easy to install, reliable (one or two out of hundreds of thousands of units may leak coolant) and very quiet.

Power: EVGA SuperNOVA 1000W 80+ Gold

Be sure to use more power than your technique requires. PCPartpicker’s Wattage calculator is good, it gives you a general idea of how much you’re pulling (824W in my case), although it’s often inaccurate as the amount varies, so it’s best to make sure your computer doesn’t turn on. The “gold certification” simply refers to the efficiency of the PSU (how much energy is wasted as heat).

The box
Tower: Corsair 760 t

I chose this box because of how much space it has and how much it costs. While this doesn’t make your model train faster, the clear side panels and red LEDS do make you look cooler.

A year’s savings and graduation bonus in a photo

If you’re new to computers, it’s like putting together a very expensive Lego set. Anyone can do it, though it’s harder to mess up than you think. I’ll quickly cover how to build my assembly, although I highly recommend that you follow the full build videos when building your own, like this one! These instructions apply to just about any combination of widgets you’re using, like the video guide I linked to above.

Step 1: Install

CPU

This is probably the scariest part of building a computer, because you have to follow a certain sequence of steps and you can accidentally immediately destroy about $430 worth of CPU (like shredding a single CPU pin), which is easy to do.

The first

There is no right or wrong order to do things, but based on my personal experience, I like to use the PSU as the first part of the box I put in, because some situations require you to slide the PSU into a slot that allows you to move “through the motherboard” if it’s there.

The first

This is a big step, but once the motherboard is in everything else it’s easy to install. I will install them in order:

1. Installed RAM. It’s as simple as sliding your memory stick in the right direction and inserting the right slot on your motherboard (see the manual as there are specific slots to use depending on how many sticks you have

2. Install the CPU cooler. The only minor snag was that I had to replace the mounting bracket to be compatible with the TR, which just required a lot of force to change. It took eight bolts to install the radiator and I finished it.

3. Install the graphics card. Just slide them into their respective slots on the motherboard (for memory, see the manual for which slots are available) and fix the radiators in your case. Make sure your radiator is higher than your GPU. I made this mistake in the picture above and had to reinstall it on the front panel of the case.

4. Install storage devices. I removed one of the drive bays to get more airflow, so I placed SSDS and HDDS in a single drive bays in the lower right.

The first

It’s time to turn it on. At first I didn’t because the positive and negative cables on the power button of my case were reversed, but then I saw the bright whites and reds I was hoping for. If all goes well, you should see your computer published and then search for a boot device.

Setting up the server

Installing the OS

The next step is to install your operating system. I’m using Linux because that’s what most DL frameworks are designed for. I used Ubuntu Desktop V16.04 LTS and installed everything on my USB drive. There are many free tools such as UNetbootin or Rufus (Windows only) that you can download to prepare your usb drives. Here’s a very useful tutorial on how to create a bootable USB on a Mac, or if you’re running Windows.

Set up SSH

The first

Step 1: Port forwarding

You must perform a similar process for all routers, but since I have an Apple router, I follow this guide: 1. Set a static IP for your server so that it does not change to shut down each time it is shut down. 2. Log in to router 3 using Apple Airport Utility. Port of the mapping server. You’ll need to find the server’s MAC address for this section, so see this guide for how to find it on Ubuntu.

Step 2: Create a dynamic IP address

I used to create a dynamic IP address for the server that allowed me to connect to it remotely from the terminal. You can use this site to verify that it works.

When I want to connect to my server, I type a command like this:

My server is running on port 8888 and my Jupyter Notebook is running on 8889 (the -L option redirects the specified local port to a different host and port). This allows me to run the notebook locally on my machine for simultaneous testing and training with my server. However, if you don’t want to do this, you can type everything before the -l section. I will explain how to change the port on which THE Jupyter notebook is running in the next section.

Install DL/ML libraries

Now install all the libraries needed for deep learning/machine learning. I’ll walk you through the installation script section by section so you know what you can find here. It’s primarily based on Jeremy Howard’s install-Gpu.sh script, so a lot of the content here is not just because they’re useful tools, but because that’s what we used in Fastai.

First, we will make sure our system is up to date and install all the basic tools we need:

The next step is to download and install all CUDA GPU drivers:

Now we will verify that we have installed all of our CUDA content correctly:

From here we will add CUDA (Nvidia Deep Learning API) to our PATH variable:

After that, the next step is to install the CuDNN library (needed to create the neural network) :

We will now install Anaconda for the current user:

Next, we will install Tensorflow and Keras:

Next, we’ll install Python dependencies for Fastai:

The following sections will configure the Jupyter Notebook:

To change the default port to run Jupyter Notebook (port 8888), uncomment the line and enter the desired port. This allows you to run your laptop on your server and locally at the same time, allowing multiple users to use their laptop while you use your laptop:

Now we will configure TMUx, a tool that allows us to create multiple panes in terminal Windows and keep the program running when disconnected. Check out this documentation because it helps me understand all the features of TMUx. It’s very useful because you can run a notebook in one pane, monitor GPU usage in another window, and open a Linux terminal in another pane:

Next, we’ll create a TMUXP configuration file that will set up our development environment in a single command without having to configure the pane, launch Jupyter Notebook, and so on every time we want to do something. For our Fastai environment, we will start it using tMUXP Load Fastai. See the link to Using TMUXP for documentation in the bash script and for saving tMUx sessions between server restarts. Anyway, here’s how to configure our environment:

Since we no longer need them, let’s delete the installation files:

That’s it! At the time of writing this article, my server has been running 24/7, completely problem-free, completely silent, and with ease through training.

I also referred to some works:

www.digitalocean.com/community/t…

Towardsdatascience.com/building-yo…

Medium.com/impactai/se…

Dozens of ari cloud products limited time discount, quickly click on the coupon to start cloud practice!