Author Topic: My Beowulf Cluster  (Read 23890 times)

0 Members and 1 Guest are viewing this topic.

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
My Beowulf Cluster
« on: December 27, 2011, 11:20:57 pm »
I've started a little project in my home, it's a Debian 6 (squeeze) based Beowulf cluster! From donated and unused desktops I've made what is more or less just a small LAN network right now with partial Beowulf setup implemented. I'm still working on the OpenMPI stuff. As it is right now, I've got them all running Stanford's Folding at Home, getting mucho work units done 24/7.

If you would like to assist me and know anything about clustering or Linux in general, I'm still kind of a noob at this and could use the help. I may post questions here for that reason, if anyone can help me, a bunch of calc programmers will probably have the answer if I can't find it on my own!

Thanks in advance to anyone who helps me with the project.


Offline Juju

  • Incredibly sexy mare
  • Coder Of Tomorrow
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 5730
  • Rating: +500/-19
  • Weird programmer
    • View Profile
    • juju2143's shed
Re: My Beowulf Cluster
« Reply #1 on: December 27, 2011, 11:23:40 pm »
That looks pretty interesting.

Also welcome back!

Remember the day the walrus started to fly...

I finally cleared my sig after 4 years you're happy now?
THEGAME
This signature is ridiculously large you've been warned.

The cute mare that used to be in my avatar is Yuki Kagayaki, you can follow her on Facebook and Tumblr.

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #2 on: December 27, 2011, 11:25:02 pm »
Thanks! It's good to be back! By the way if anyone wants to join the project and my team feel free to do so:

http://folding.stanford.edu/

And the team number is 213973

Anyone can join my team, just insert the number in the configuration dialog.

Offline annoyingcalc

  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1953
  • Rating: +140/-72
  • Found in Eclipse.exe
    • View Profile
Re: My Beowulf Cluster
« Reply #3 on: December 28, 2011, 12:06:57 am »
Welcome back!
This used to contain a signature.

Offline TIfanx1999

  • ಠ_ಠ ( ͡° ͜ʖ ͡°)
  • CoT Emeritus
  • LV13 Extreme Addict (Next: 9001)
  • *
  • Posts: 6173
  • Rating: +191/-9
    • View Profile
Re: My Beowulf Cluster
« Reply #4 on: December 28, 2011, 07:48:50 am »
Oh wow, it's been a while. I hope you're doing well. :) I don't know anything about the linux netwoking stuff, but hey w/e. What are you setting it up for? Also, I see lots of Dell computers. :P

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #5 on: December 28, 2011, 10:00:26 am »
Thanks to everyone for the warm welcome back!

I'm setting it up to run the Folding@Home project for the most part, since I won't be using it myself all the time. Aside from that, when I am using it, I'm planning on writing a 3d rendering engine for OpenMPI and trying out some fun stuff there. It'll have to be in C or C++ so it can be portable to other Beowulf clusters.

OpenMPI is a Message Passing Interface, it sends parts of a large task to the units, and all the units crunch the numbers, then the nodes give their parts back to the head unit through the network. It's basically parallel processing, but on a larger scale than inside a multicore processor. Each node has it's own ram, gpu, etc, so there's a lot more to work with than a multicore cpu.

here's the wiki on what a cluster is and can do:

http://en.wikipedia.org/wiki/Beowulf_cluster

I am not well versed enough to tell you everything they are capable of or what the kind of power is in one of these things. I'm trying to figure out how to benchmark my system at the moment, and finish the OpenMPI setup. It's quite tricky when you don't know enough about Linux or networking in general. :p
« Last Edit: December 28, 2011, 10:00:52 am by AaroneusTheGreat »

Offline annoyingcalc

  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1953
  • Rating: +140/-72
  • Found in Eclipse.exe
    • View Profile
Re: My Beowulf Cluster
« Reply #6 on: December 28, 2011, 11:10:42 am »
I heard you met DJ_Omnimaga once
This used to contain a signature.

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #7 on: December 28, 2011, 12:07:14 pm »
Sho nuff did. We met up when I was in Quebec studying french. We only got to hang out for a night. Dinner and drinks with my study abroad group. And of course a bit of nerding out with the calculators and such. It was awesome. I'm trying to get back up there at some point, for one thing I love the province of Quebec. I would actually live there if I could/didn't have so many ties and connections down here that I need to keep up. And for two, I want to be able to meet up with DJ again and possibly Juju if he can make it out to that area when I'm there. The three of us chatted about that last night. I may be making a considerable sum of money with a website job for a company and some install work for dental offices soon, so if that pans out you'll be hearing about me making another, probably more leisure based, trip up there.

Offline annoyingcalc

  • LV10 31337 u53r (Next: 2000)
  • **********
  • Posts: 1953
  • Rating: +140/-72
  • Found in Eclipse.exe
    • View Profile
Re: My Beowulf Cluster
« Reply #8 on: December 28, 2011, 12:34:34 pm »
Wow you are so lucky!
This used to contain a signature.

Offline TIfanx1999

  • ಠ_ಠ ( ͡° ͜ʖ ͡°)
  • CoT Emeritus
  • LV13 Extreme Addict (Next: 9001)
  • *
  • Posts: 6173
  • Rating: +191/-9
    • View Profile
Re: My Beowulf Cluster
« Reply #9 on: December 28, 2011, 03:55:21 pm »
@ATG: Alright, sounds pretty cool. =)

Offline Juju

  • Incredibly sexy mare
  • Coder Of Tomorrow
  • LV13 Extreme Addict (Next: 9001)
  • *************
  • Posts: 5730
  • Rating: +500/-19
  • Weird programmer
    • View Profile
    • juju2143's shed
Re: My Beowulf Cluster
« Reply #10 on: December 28, 2011, 04:42:54 pm »
Yeah, that would be pretty cool if the three of us meets together.

Remember the day the walrus started to fly...

I finally cleared my sig after 4 years you're happy now?
THEGAME
This signature is ridiculously large you've been warned.

The cute mare that used to be in my avatar is Yuki Kagayaki, you can follow her on Facebook and Tumblr.

Offline Illumio

  • LV1 Newcomer (Next: 20)
  • *
  • Posts: 12
  • Rating: +0/-0
    • View Profile
Re: My Beowulf Cluster
« Reply #11 on: December 28, 2011, 04:45:43 pm »
So many in one space! :O
Spheres and Cubes.

<a href="http://www.nerdtests.com/ft_nt2.php">
<img src="http://www.nerdtests.com/images/badge/nt2/b2513faa423935d4.png" alt="NerdTests.com says I'm a High Nerd.  Click here to take the Nerd Test, get geeky images and jokes, and talk to others on the nerd forum!">
</a>

For Science! And Math! And other things!

Offline AaroneusTheGreat

  • Moderator
  • LV5 Advanced (Next: 300)
  • *****
  • Posts: 287
  • Rating: +26/-1
    • View Profile
Re: My Beowulf Cluster
« Reply #12 on: December 28, 2011, 09:37:02 pm »
@Illumio:
Yesh. :D it may get more diabolical soon. It's an odd cluster as it is, but I may be adding a netbook to it soon! O.o As it is my laptop is how I log into the cluster to do work on it. I can access each node with SSH and do installs and everything while laying in bed lololol. I set myself up for laziness. Gets more done sometimes, because I don't have to get up and scoot off the bed to work on the cluster. That was such a hassle! haha.

In all reality it was though, because I only have one kvm with two port sets on it, so I could only switch two units back and forth at the terminal easily, that's why there were so many keyboards. I did a little more work on this today and verified that my work units are getting worked on correctly, turns out 3 of the units were experiencing problems. My new stats should show up in a week or two when the units have finished enough work and the servers at stanford update.

Here's the stats link for the cluster on stanford's servers.

http://fah-web.stanford.edu/cgi-bin/main.py?qtype=teampage&teamnum=213973

This data will change every day. I have a desktop (my dad's) in the floor below me running a client on his GPU. It gets work units done MUCH faster than any of the units in this cluster. I'm not even sure if my cluster as a whole will outperform it honestly. These units are pretty old. I also haven't figured out a way to benchmark the system yet.

I'll let you all know what the final word is about processing power when the system is finally benchmarked though.

@annoyingcalc: Yes it was quite fortuitous to be able to do those things. I had to do a ton of work to get where I was though, french classes every year since the 6th grade without a gap but once in my freshman year of highschool. I'm not currently taking any french classes but I've been studying forever. I only ended up going because I just so happened to have made a good deal of money building a website, and I had run out of french classes at my community college. So I took the last thing they offered in french, which was the study abroad.

@Juju: We'll see if I can make it happen, but rest assured I'm trying my best! :)

It's really good to talk to you all again, I've missed you guys. It's really cool that you all were so quick to give me a warm welcome back. That's one of the things I love about this community. You don't get this kind of thing everywhere on the internet. Lots of things are exclusive and require a lot of rites of passage to get into. The calculator community has proven to be the most open programming community I've run into. I really hope that never changes.
« Last Edit: December 28, 2011, 09:44:57 pm by AaroneusTheGreat »

Offline DJ Omnimaga

  • Clacualters are teh gr33t
  • CoT Emeritus
  • LV15 Omnimagician (Next: --)
  • *
  • Posts: 55943
  • Rating: +3154/-232
  • CodeWalrus founder & retired Omnimaga founder
    • View Profile
    • Dream of Omnimaga Music
Re: My Beowulf Cluster
« Reply #13 on: December 29, 2011, 01:03:57 am »
Heya it's great to see you back Aaron. This project seems interesting. I can't help much though since I'm not very literrate about computer stuff. However good luck :)

Also I hope we can meet again at one point. One good spot would be during my Summer vacations where I am guaranteed to have at least two Sundays and Saturdays off. It's usually during the last two weeks of July or the first two of August of each Summer. I would most likely be able to help a bit financially as well. I usually see Juju once every month or so depending of our schedules and it would be nice if we could all see each others together. :D

Anyway glad to see you return. I hope you stay around for a while even if you don't do calc stuff anymore. Hopefully you might find some familiar faces too here as well. (Art_of_Camelot, calc84maniac and Iambian, for example)

Offline Illumio

  • LV1 Newcomer (Next: 20)
  • *
  • Posts: 12
  • Rating: +0/-0
    • View Profile
Re: My Beowulf Cluster
« Reply #14 on: December 31, 2011, 02:14:16 am »
I see.  Though I can only imagine the amount of cords everywhere.  It's a sea that will never end.
Spheres and Cubes.

<a href="http://www.nerdtests.com/ft_nt2.php">
<img src="http://www.nerdtests.com/images/badge/nt2/b2513faa423935d4.png" alt="NerdTests.com says I'm a High Nerd.  Click here to take the Nerd Test, get geeky images and jokes, and talk to others on the nerd forum!">
</a>

For Science! And Math! And other things!