%expand% Lightweight Job Queuing Engine and GitHub %set(body,%html-clean({
I wrote a job queuing daemon. It sits on a head node and any number of compute nodes. Good with EC2. Replacement for condor, OGE (formerly SGE), torque, etc. So far, it's good about crash recovery, very good about latency. Other job queuing systems seem to be sooo slow with latency, you never want to use them. And they all spend so much time worrying about copying i/o files all over the place, when all I ever use is NFS anyway.

https://github.com/earonesty/grun

Written because those other systems are administratively difficult, and have more features than I need. Some day I may choose to go back to condor, or SGE, but for now, it's nice to have a smaller, faster, easier one.

I'm trying github instead of google code. The developers and gangla got me to try it out. I think I might like it. Not sure why it makes you "add" and "commit" separately.

I based the socket system off of ppcgid, since it worked so well in the past.
}))
   
Bracing against the wind  
www.documentroot.com  
Search this site:

View Archive

Free Blog Commenter & Counter

Memebot: Free Website Hosting

Americans for Approval Voting

2002 WTC Pics: Aug 12 Aug 16 Aug 30 Sep 11 Sep 23 Oct 24

Shop with Bitcoin At Amazon

Yvette's Weblog

All sites by me

Add to Google

RSS for this site

Lo! In the east against the blazing shadow of a rising sun, there come four men striding in unison, each step a journey, against the terrible winds.

[Powered by Blogger]

Saturday, May 14, 2011

Lightweight Job Queuing Engine and GitHub
Posted 8:08 PM

I wrote a job queuing daemon. It sits on a head node and any number of compute nodes. Good with EC2. Replacement for condor, OGE (formerly SGE), torque, etc. So far, it's good about crash recovery, very good about latency. Other job queuing systems seem to be sooo slow with latency, you never want to use them. And they all spend so much time worrying about copying i/o files all over the place, when all I ever use is NFS anyway.

https://github.com/earonesty/grun

Written because those other systems are administratively difficult, and have more features than I need. Some day I may choose to go back to condor, or SGE, but for now, it's nice to have a smaller, faster, easier one.

I'm trying github instead of google code. The developers and gangla got me to try it out. I think I might like it. Not sure why it makes you "add" and "commit" separately.

I based the socket system off of ppcgid, since it worked so well in the past.

[View/Post Comments] [Digg] [Del.icio.us] [Stumble]

Home | Email me when this weblog updates: | View Archive

(C) 2002 Erik Aronesty/DocumentRoot.Com. Right to copy, without attribution, is given freely to anyone for any reason.

Listed on BlogShares | Bloghop: the best pretty good | Blogarama | Technorati | Blogwise