Assembling first node for my 4-node GPU rack

I have 24 U rack waiting to house 4 nodes, each 4 double graphic cards (8 GPU-s), total 32 GPU-s. Combined arithmetic performance is about 28 000 GFLOPS = 28 TFLOPS. 
I have 4 shelves (4U height to house 1 or 2 PSU-s each, system board, 4 GTX 295 graphics cards and hard drive. Trickiest part was to mount system board to shelf. Holes didn’t match. I bought 3 mm bolts, and 20 mm spacers to make good mounting. HDD is seated into plastic HDD shelf, attached to shelf.
Components:
planning components layout

planning components layout

 

20 mm apecers between rack shelf and K9A2 Platinum sytem board

20 mm spacers between rack shelf and K9A2 Platinum sytem board

 

System board on the left, PSU with cable management on the right

System board on the left, PSU with cable management on the right

 

HDD in the bracket, cables attached

HDD in the bracket, cables attached

5 Comments

  1. Nice setup, more pictures please🙂

  2. Looking good Jaak!

    You won’t be able to leave the PSU where you show it though, clamp won’t make it down to top of GPUs. Have to move PSU back to between clamps.

    Also, I see you found the trick of the motherboard spacers. Inside two motherboard holes use spacer but do not attach to shelf. Smart guy! 😉

    I’m sure you have though about the foam on the clamps melting but it does not. It is more heat resistant than it seems.

    Jason

  3. Jason, Its very stiff and strong now. Actually I drilled 3,5 mm custom holes and attached all spacers, all 9. Clamp tip was good, but it doesn’t fit anyway, using 2 Chieftec 750W PSU-s in the next node. Insted that I’ll use custom made bracket, attached to original backplate holes in GPU-s.

    Jaak

  4. When the system is up and running, do you need to plug in anything on the display ports?

    On a 1xGTX 295 system that I’m using for CUDA I had to plug in monitors to both GPUs (DVI and HDMI) for CUDA to recognize them under Windows 7. From what I’ve heard other people had the same issue with Vista. Does running XP solve this?

    I’m going to be building a 3X GTX 295 system, and I don’t particularly fancy having to hook up 6 monitors to it…

  5. Luka

    I have dummy plufs connected into DVI outputs. My suffection, for folding, use XP only. Vistra and Win7 are going to make You more trouble. You are going to get errors. This is widely talked issue in F@H forums.,

    Jaak


Comments RSS TrackBack Identifier URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s