Posts Tagged ‘Wallaby’

How accounting group configuration could work with Wallaby

February 5, 2013

Configuration of accounting groups in HTCondor is too often an expert task that requires coordination between administrators and their tools.

Wallaby provides a coordination point, so long as a little convention is employed, and can provide a task specific interface to simplify configuration.

Quick background, Wallaby provides semantic configuration for HTCondor. It models a pool as parameters aggregated into features and nodes aggregated in groups, with features and individual parameters associated with nodes and groups. It provides semantic validation of configuration before it is distributed, and has expert knowledge for minimal impact configuration changes.

And, accounting group configuration in HTCondor is spread across seven fixed parameters (GROUP_NAMES, GROUP_ACCEPT_SURPLUS, GROUP_SORT_EXPR, GROUP_NAMES, GROUP_QUOTA_ROUND_ROBIN_RATE, GROUP_AUTOREGROUP, GROUP_QUOTA_MAX_ALLOCATION_ROUNDS), and another five dynamic parameters (GROUP_ACCEPT_SURPLUS_groupname, GROUP_AUTOREGROUP_groupname, GROUP_PRIO_FACTOR_groupname, GROUP_QUOTA_groupname, GROUP_QUOTA_DYNAMIC_groupname). These are dynamic because the “groupname” in the parameter is any name listed in the GROUP_NAMES parameter.

In addition to its other features, Wallaby has an extensible shell mechanism, which can be used to create task specific porcelain.

For instance, agree that tools and administrators will store accounting group configuration on a feature called AccountingGroups, and the tools can use Wallaby’s API to manipulate the configuration while the following porcelain can simplify the task for managing that configuration by administrators.

See – wallaby_accounting_group_porcelain.txt
(embedding content requires javascript, which is not available on wordpress.com)

Wallaby: Skeleton Group

June 19, 2012

Read about Wallaby’s Skeleton Group feature. Working similar to /etc/skel for accounts on a single system, it provides base configuration to nodes as they join a pool. It is especially useful for pools with dynamic and opportunistic resources.

Manage inventory with Wallaby

January 16, 2012

Wallaby will manage your configuration, as well as an inventory of your machines. It can differentiate between machines that are expected to be present and those that opportunistically appear.

Build the roster with wallaby add-node

$ wallaby add-node node0.local node1.local node2.local
Adding the following node: node0.local
Console Connection Established...
Adding the following node: node1.local
Adding the following node: node2.local
$ for i in $(seq 3 10); do wallaby add-node node$i.local; done
Adding the following node: node3.local
Console Connection Established...
Adding the following node: node4.local
Console Connection Established...
...

List expected nodes (provisioned) –

$ wallaby inventory
Console Connection Established...
P        Node name                 Last checkin
-        ---------                 ------------
+      node0.local Wed Jan 11 07:32:33 -0500 20
+      node1.local Thu Jan 05 12:15:00 -0500 20
+     node10.local Wed Jan 11 07:31:56 -0500 20
+      node2.local Wed Jan 11 07:31:56 -0500 20
+      node3.local Wed Jan 11 07:15:21 -0500 20
+      node4.local Wed Jan 11 07:31:42 -0500 20
+      node5.local Wed Jan 11 07:16:47 -0500 20
+      node6.local                        never
+      node7.local Wed Jan 11 07:32:33 -0500 20
+      node8.local Wed Jan 11 07:32:33 -0500 20
+      node9.local Wed Jan 11 07:30:47 -0500 20
-      robin.local Thu Dec 15 14:11:35 -0500 20
-      woods.local Tue Jan 10 20:33:47 -0500 20

List opportunistic, bonus nodes (unprovisioned) –

$ wallaby inventory -o unprovisioned
Console Connection Established...
P        Node name                 Last checkin
-        ---------                 ------------
-      robin.local Thu Dec 15 14:11:35 -0500 20
-      woods.local Tue Jan 10 20:33:47 -0500 20

Provisioned nodes that have never checked in, maybe setup failed –

$ wallaby inventory -c 'last_checkin == 0 && provisioned'
Console Connection Established...
P        Node name                 Last checkin
-        ---------                 ------------
+      node6.local                        never

Provisioned node that have not checked in for the past 4 hours, maybe machine is down –

$ wallaby inventory -c 'last_checkin > 0 && last_checkin < 4.hours_ago && provisioned'
Console Connection Established...
P        Node name                 Last checkin
-        ---------                 ------------
+      node1.local Thu Jan 05 12:15:00 -0500 20

Unprovisioned nodes that have not checked in for 48 hours, candidates for wallaby remove-node

$ wallaby inventory -c 'last_checkin < 48.hours_ago && !provisioned'
Console Connection Established...
P        Node name                 Last checkin
-        ---------                 ------------
-      robin.local Thu Dec 15 14:11:35 -0500 20 

Enjoy.

Get Wallaby – Condor Configuration Management

December 4, 2010

Will Benton has been busy enhancing Wallaby, a system for Condor configuration management. He presented it along with Rob Rati at Condor Week 2010. A lot has happened since then, recently including the addition of an HTTP transport for retrieving configuration and a means to extend the Wallaby shell.


%d bloggers like this: