Register
Events
Popular
More
Post #1
I am currently working on creating a cluster computer for my school. I'd like to know what you guys think would be a good distro to install on the nodes and on the host.
I was thinking Pelican http://idea.uab.es/mcreel/PelicanHPC/
I'd like to use it for either car crash physics simulations, or if that doesn't pan out, as a render farm for CGI.
So far, I have only the gigabit switch, and gigabit ethernet cables, I need to get the machines donated from local businesses, and possibly our school district's property redistribution warehouse. Hoping for between 20-47 nodes. I'll keep the thread updated with pictures when the project begins to move forward, and I'm sure I'll have more questions...
So facepunch linux gents, what do you think?
-
-
I think Dragonfly BSD is used for that. I know it's not Linux, but close enough.
I'm sure Scientific Linux is also pretty good at it.
-
You should call up Pixar and ask them what they use for their rendering farms.
-
For clustering like CGI you'll need FC or external SAS to provide the throughput needed. good luck though
-
I has to write an essay on this last semester, i found beowulf clusters pretty interesting. Give the openMOSIX project a read, apartently it's pretty simple to set up.
-
Cluster computing is really useful, anyone knows how can i setup my own cluster computing system in my home? i hope it will be easy.
-
Read the post above yours.
-
yeah but i noticed these tools require me to compile stuff, and even more programming,
i would like a Ready to use solution..
-
I personally don't know of any, since I've never built a cluster. Hopefully somebody can point you in the right direction soon. :)
-
So far the trickiest part is getting the hardware I need. I've got to talk to several different companies.
Edited:
Gigabit isn't fast enough?
-
...
What do you want a cluster for? I honestly doubt you have an actual need for clustering.
-
Rendering. I plan on building a Raspberry Pi cluster or a cheap computer cluster for rendering Blender.
-
125MB/s is slow
10Gig is better throughput for clustering
Fibre Channel if you want no loss, ultra performance, TCP/IP is shit for clustering
But for home stuff I guess who cares 125MB/s assuming the switch has the processing power to handle <n_ports> X 125MB/s throughput
-
have a look at baremetal node
Edited:
http://www.returninfinity.com/baremetalnode.html
-
We've got a shitton of RJ45 connectors, and I might be able to get funding for a roll of Cat6A and a handfull of NICs that support 10GB BaseT
-
nice are the switches 10GB?
if the nics support 10GbE see if they support FCoE, if they do you'll get a good speed boost
-
You can build a 300$ rig that will render shit twice as fast as 600$ worth of Raspberry Pi's.
Not to mention, due to 1 per customer limitations on first batch, you wouldn't be able to get enough of them for a cluster for ages.
-
I can back this up, I've had great communication from the author.
-
I hope so, I'm trying to get it donated by cisco.
-
If you plan on using Blender then I can't imagine that you'll need much as far as networking goes. Check out the cluster builds on graphicall and maybe blender.org if they still support it officially since the 2.5 thing. I dunno much else. How it helps.
-
Most Cisco equipment will do FCoE or at least iscsi w/ jumbo frames
-
I've seen these things being built for a physics lab, couldn't pester them with questions though - as they were the snooty Oracle consultant type.
The OS doesn't really matter, but i would go with an enterise grade linux like RHEL. As cluster configs are prob well documented.
ww.centos.org/docs/5/pdf/Cluster_Administration.pdf
CentOS is zero cost RHEL
-
Emailed a gentleman at a local university, and he said he'd forward my request for parts to their GRID processing professor, who recently oversaw upgrades of their system. Waiting to hear back from the 2nd guy.
-
Wouldn't this cause trouble when you're going to run a lot of tasks?
-
Err, how?
You create a task and then you deploy it to all the notes with the orchestrator and you're done.
If you're planning on running a lot of tasks on the same cluster, you're doing it wrong.
-
Still havent heard back, going to email them again.
-
Ok, so finally an update. We'll be using CentOS.
We got 47 nodes, IBM Xseries 330's with pentium 3's in them
Some have 512mb of ram, and 10 are mirrornet capable with 1gb of ram
We have gigabit ethernet switches that all connect back to a fiber switch
I'll be posting pictures as soon as I get them!
Heres one picture, I'm on the left. My best friend on the right.
-
Noice.
Get some liquid cooling and connect it to your heating system of your school, and you have heating for the school sorted out, as well.
-
You should run at least one Minecraft server on one of those, just for the kicks of it. (Since it doesn't take up a lot of space or RAM)Ok, so finally an update. We'll be using CentOS.
We got 47 nodes, IBM Xseries 330's with pentium 3's in them
Some have 512mb of ram, and 10 are mirrornet capable with 1gb of ram
We have gigabit ethernet switches that all connect back to a fiber switch
I'll be posting pictures as soon as I get them!
Heres one picture, I'm on the left. My best friend on the right.
-
So yesterday I moved them into their actual home, the main patch closet at the school. Tomorrow I'm going to be mounting them in the racks, and attempting to get rocks to boot to pXe. Rocks is a CentOS based distro designed for HPCs
-
OP, I officially love you. You've answered a question I've been looking for for ages.
-
-snip-
-
Well so much for mounting them in the racks, no side rails... Going to have to stack them for now, and get them functional to prove to administration that they can be a revenue stream before I can buy rails kits for then.
-
-
Compared to most other process and RAM intensive tasks, I don't think it's taking up that much. On my own VPS, running Craftbukkit-git with SpoutPlugin, it runs perfectly fine with just about 256MB virtualized, in an ArchLinux virtual machine with 512MB RAM (it's also running a few experiments of my own written in NodeJS and Meteor).
I've had loads of other services require AT LEAST 512MB of RAM, while this seems to be enough for all of my stuff.
-
My master nameserver runs bind9 (master), ntpd, fail2ban, rsyslog and cron, a irssi session in tmux, and odentd. It uses 92 MB of RAM.
One of my nameservers runs bind9 (slave), ntpd, fail2ban, rsyslog and cron and uses 19 MB of RAM.
Another server I have runs apache2, nginx, passenger, a ruby application, ntpd, mysqld, cron and rsyslog, exim4, a redis server and bugzilla (a perl app). It uses 287 MB of RAM.
I'm inclined to agree with PvtCupcakes here.
-
Really? What Linux system are you using? I'm not very efficient, as I'm using a standard Arch Linux installation without GUI.
-
I'm using Debian 6.0.5 64-bit. The numbers for the 287MB server are a bit low, the server will bump up to 350/400 when the 2-4 apps start to spin up but that's about as far as it will go.
-
Oh, so those numbers are on idle? Oh, I thought you meant during load. But I don't know Debian used that little, thats pretty cool.
After checking it out, I realize that my server does use a lot less than I had measured (I was counting the virtualization usage, instead of reading the usage directly from an SSH session using htop), but still not as low as yours. I should probably look into getting a server kernel too. Thanks for the info!
-
If your vps panel includes hard drive caching then yeah you have to discard that. You can use 'free' to get the numbers you need. My secondary dns and primary dns don't really rise up in memory, but like I mentioned in my last post, the app server does.
Friendly x 1
Funny x 1
Late x 1
Zing x 1
Agree x 1
Dumb x 4
Winner x 1
Informative x 2
Useful x 1