Author Topic: My Beowulf Cluster  (Read 23572 times)

0 Members and 1 Guest are viewing this topic.

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: My Beowulf Cluster
« Reply #15 on: December 31, 2011, 02:44:55 am »
Yeah that must be a lot of cords. You haven't seen behind my TV, though, with all the gaming consoles I got. O.O About 50% of them aren't even plugged in anymore due to running out of slots (idk if it's the right word for where you plug stuff in the wall)

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #16 on: December 31, 2011, 04:07:15 pm »
There really are a lot of cords.



I added an 8th unit to the cluster today. Cost me $24.99 for the tower, and $9.99 for a harddrive for it. It was at a local Goodwill Computer Works store, and was marked for parts only, totally fine, just needed a harddrive and an OS. I set it up today and it's currently running just like the other units. It took me a few hours to do, and a lot of typing, but it works just fine.   :)



The current computer count under my username for Folding@Home is now up to 10 computers, including my laptop and the desktop my dad uses downstairs, he asked me to add it to the project the other day. It's running the GPU client on an ATI Radeon 5700 graphics card. The thing flies through work units like nothing I've seen before, so I'm getting lots of stuff done.

Interesting side effect, I had to turn the heaters down in my apartment and crack the windows because the cluster is generating so much heat that it warms up my place. O.o

I'm curious to see what the cord bundle behind your tv looks like DJ_O. lol it's probably more diabolical than the one behind my cluster. Game systems generally require more cables than computers do to run them. My systems are running headless, so there's no need for keyboards or mice anymore, so that slimmed down the cord nightmare a bit.

Anyways I'm starting to ramble, just wanted to update you all on what's going on with the nerdiness.

Offline chattahippie

  • LV6 Super Member (Next: 500)
  • ******
  • Posts: 358
  • Rating: +27/-0
  • Super Member! :D
    • View Profile
Re: My Beowulf Cluster
« Reply #17 on: December 31, 2011, 09:03:59 pm »
This is really cool! :D
Do you know the flops of the cluster?

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #18 on: January 05, 2012, 09:20:33 am »
Not at all yet. lol. I haven't figured out a way to benchmark it yet. I've got the hostlist and whatnot setup, and installed OpenMPI on several of the machines, and I'm getting closer to getting it running as a true beowulf, but I'm still having problems getting the machines to talk to each other through the MPI stuff. I keep getting strange errors, I think it's because I don't know which flags to pass the mpiexe program (aside from the most basic things), which runs any MPI program, so I'm probably just invoking it incorrectly, or don't have the right services up on my nodes when the server unit tries to talk to them.

Cluster update:

(reposted from facebook)

I discovered some issues, wouldn't you know it? Nothing in the computer science field seems to be straight forward sometimes. I found out that several of my units have missed their deadlines that folding at home gives for when work units need to be finished, due to my startup scripts. /facepalm

So that said, I've just gone ahead and disabled them for now, and started the programs on all 8 machines individually, some of them got new units, the others seem to be attempting to finish some units, which may be pointless at this point, they might miss the deadlines too, which is okay, they will get new units soon. The deadlines coming up are tomorrow and the next day. We'll see what happens. At this point it's kind of a race against time for some of my units lol.

All of the systems have been switched over (with minor difficulty) to a completely command line based Linux, so as to save resources for the project, which is command line anyways. Initially I used the GUI to kind of ease myself into the more difficult aspects of Linux, but now it's kind of useless, so I just got rid of it, which turned out to be much easier than anticipated. I did have to fix several things that got broken when I removed the packages, but everything seems to be okay right now. I've had no issues getting the clients running or anything like that since.

Many thanks go out to Noah Putman ... yet again... saved my butt when I couldn't figure it out on my own. The guy's a whiz with Linux.

The units that did finish work got a lot done recently though. There have been 53 work units so far that have been completed. The cluster has also jumped into the middle 20th percentile in the rankings now, that's pretty impressive for such old machines.

(The server and the first four units went online in mid December, so it hasn't been very long, but it also hasn't been that long either, a couple of weeks, so if you do the math, that's a few work units a day, which is pretty good for having so many issues so far)
« Last Edit: January 05, 2012, 09:22:29 am by AaroneusTheGreat »

Offline flyingfisch

  • I'm 1337 now!
  • Members
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1620
  • Rating: +94/-17
  • Testing, testing, 1...2...3...4...5...6...7...8..9
    • View Profile
    • Top Page Website Design
Re: My Beowulf Cluster
« Reply #19 on: January 05, 2012, 09:24:54 am »
How hard is it to set something like this up? I would like to try it.... maybe :P



Quote from: my dad
"welcome to the world of computers, where everything seems to be based on random number generators"



The Game V. 2.0

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #20 on: January 05, 2012, 10:40:59 am »
You know it's really not as hard as I expected. I've had some help from the drummer in my band. That's who I mentioned in the previous post. He's a Linux guru, has been using UNIX based systems since the late 90's or so I think. Around the time I got into programming more or less. So between the two of us, it's been pretty straightforward. With the help of some of the guys on here, I'm sure you could set one up in a matter of days. It only took one day of pretty inconsistent work to get my server and the first 3 nodes online. After that, each node only took about an hour to get completely setup.

The trick is to get them running the right way. If you want to run something like I am, you really don't need an MPI or any SMP technology (http://en.wikipedia.org/wiki/Message_passing_interface if you're curious where to start reading about that). I could run the whole system here as simply a small LAN network and it would be all I needed for running a dedicated Folding at Home farm. That said, I want to use this thing for other projects of my own, so I'm setting up OpenMPI which has a mildly steep learning curve when transitioning from Window$ to Linux to MPI clustering with Linux. I have focused on calculators so much that I've kind of had to catch up with the larger units. I decided to take this on because I knew I would need to learn some things to get into the research department at the college I'm attending.

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #21 on: January 07, 2012, 11:28:30 pm »
Update: I'm quickly approaching the 100 work unit completed! As it is, while I'm writing this post, I've completed 85 work units total. I'm averaging about 4 units per 24 hours. They complete at different times, due to the size differences, but that figure is relative to when I started work with 4 running nodes, that weren't acting right. In the last week or so, I've seen about 6 to 7 units done every 24 hours. The cluster is really chugging along.

Speaking of how hard it's working. I've had to turn my air conditioner on in the middle of winter. I came home from a walk, and my house was about 80 degrees Fahrenheit, or about 26.6 degrees Celsius for those of you who live in the metric world.

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #22 on: January 09, 2012, 03:16:49 am »
UPDATE:

Very exciting news! As of 2012-01-08 23:09:00 My computers have finished 100 work units! Yay for milestones! 15 work units were finished since my last post! I'm blown away! Pretty awesome to see the cluster working so well considering the age of the machines! I discovered one of my units is 12 years old! Wow. The compaq on the far right in the picture is the one I'm speaking of. 700mhz AMD duron processor. Not quite anywhere as old as my TI-99/4a but quite the dinosaur in it's own right. The TI-99/4a is older than me by about 7 years, if you can feature that. 1981 was the year, when the pterodactyls flew.
« Last Edit: January 09, 2012, 03:18:47 am by AaroneusTheGreat »

Offline TIfanx1999

  • ಠ_ಠ ( ͡° ͜ʖ ͡°)
  • CoT Emeritus
  • LV13 Extreme Addict (Next: 9001)
  • *
  • Posts: 6173
  • Rating: +191/-9
    • View Profile
Re: My Beowulf Cluster
« Reply #23 on: January 09, 2012, 11:23:05 am »
O.O 1981 isnt *THAT* old! It's vintage I tell you, vintage!!! :P

Offline flyingfisch

  • I'm 1337 now!
  • Members
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1620
  • Rating: +94/-17
  • Testing, testing, 1...2...3...4...5...6...7...8..9
    • View Profile
    • Top Page Website Design
Re: My Beowulf Cluster
« Reply #24 on: January 09, 2012, 11:27:30 am »
Sorry if this is a dumb question, but what is a work unit?



Quote from: my dad
"welcome to the world of computers, where everything seems to be based on random number generators"



The Game V. 2.0

Offline Spyro543

  • LV9 Veteran (Next: 1337)
  • *********
  • Posts: 1189
  • Rating: +74/-3
    • View Profile
Re: My Beowulf Cluster
« Reply #25 on: January 09, 2012, 07:34:35 pm »
In folding@home, a "work unit" is just a way of measuring how much of a protein the computer has calculated so far.

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #26 on: January 10, 2012, 05:15:59 am »
Speaking of work units, Willrandship's unit has just finished it's first unit and been awarded 600 points! Congrats Will on your first unit for the team!

If you would like to see a video of why we are doing this, here's a good example of the kind of work this program is doing. Below is a successfully simulated protein fold. Proteins fold up in the body for several reasons, it creates active surfaces on the molecule that allow the individual units to form complex structures for one thing, like the cillia on your muscle cells, which allow you to move, or the cell walls of your blood cells.

Protein folding is intensely important in human beings because, when it goes wrong,  it can cause a myriad of problems. Mad cow disease is a protein misfolding disease for instance, and as it's been discovered recently, it appears that Alzheimer's is as well. Several of my units are in fact working on the Alzheimer's Amyloid Protein sequences right now.  

By the way, I would like to try to recruit as many people as possible that I know to help the project. Anyone here is more than welcome to join my team!

TEAM NUMBER: 213973

http://folding.stanford.edu/English/Download

That is the download link for the client, and here's the installation instruction page:

http://folding.stanford.edu/English/Guide

As previously mentioned, here's that nifty video:


« Last Edit: January 10, 2012, 05:20:53 am by AaroneusTheGreat »

Offline flyingfisch

  • I'm 1337 now!
  • Members
  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1620
  • Rating: +94/-17
  • Testing, testing, 1...2...3...4...5...6...7...8..9
    • View Profile
    • Top Page Website Design
Re: My Beowulf Cluster
« Reply #27 on: January 10, 2012, 12:29:51 pm »
do you have to have a powerful computer to participate in folding@home?



Quote from: my dad
"welcome to the world of computers, where everything seems to be based on random number generators"



The Game V. 2.0

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #28 on: January 10, 2012, 09:49:38 pm »
Not in the slightest. One of my machines, the very old 700 megahertz Duron processor unit, is probably much slower than your computer, unless you haven't bought a computer in 12 years, it has only finished about 2 work units, but it's contributing none the less. The minimum requirements for a system is a Pentium 3 at 450 megahertz.

The only thing I would tell you to watch out for is while setting up your client, take into account the amount of ram that you've got, that's the major thing. If your computer has 256 megabytes of ram or less, then you're going to want to set your work units to normal or small sized units, and just check on the progress every day or two to see if it's coming along quickly enough. If it looks as if the progress won't make the deadline, set the units to a size step down, and wait for the deadline to make the system get a new unit.

Chances are you'll have no problems though. The guides on the website, and the forum they've got setup are more than helpful, they make it rather easy to get started.

Here's the FAQ page, I've referred to this many times and found what I was looking for when it came to answers about what to do and/or what it does etc. Very useful info.

http://folding.stanford.edu/English/FAQ-main#ntoc1

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #29 on: January 17, 2012, 10:25:45 am »
My team has reached another milestone! As of today, we have folded over 200 work units!