OpenBSD Journal

Clusterit

Contributed by mbalmer on from the ports-need-some-exposure-too dept.

Clusterit 2.4, which has been submitted by Will Maier <willmaier at ml1.net>, has been imported to our ports tree:

Clusterit is a collection of clustering tools loosely based on IBM's PSSP clustering tools. Unlike PSSP or GLUnix, Clusterit allows fast parallel execution of remote commands as it is written in architecture-independent C. Administrators can choose from a variety of authentication methods, including SSH and Kerberos.

With Clusterit, administrators can run parallelized and load balanced software builds (using jsd/jsh) or manage a heterogeneous cluster of machines (using dsh/dvt/rvt).

(Comments are closed)


Comments
  1. By Matt Van Mater (69.255.1.181) on

    This looks very similar to Tentakel, which was already in the ports tree for quite some time now. I have used tentakel on occasion and have been very happy with it for parallel operations. (in my case, i tested it to facilitate enterprise wide, distributed vulnerability scanning)

    Can someone explain how this differs in capability from tentakel? One minor difference is that some commands seem to have a kind of abstraction layer such as normalizing the results of the df command, where tentakel executes your command in the exact manner it was typed everywhere so it is more suited for homogenous environments.

    Are there any other major differences I am missing?

    Comments
    1. By Anonymous Coward (63.237.125.164) on

      ClusterIt can do what Tentakel does plus a few more things. It has 14 distinct executables.

      For example, from: http://www.garbled.net/man/run.html
      "The run utility can be used to run a command, or group of commands on a cluster of machines. The command or commands are run on a single node in the cluster, which is chosen at random."

      And from: http://www.garbled.net/man/pdf.html
      "pdf displays statistics about the amount of free disk space on the specified file_system or on the file system of which file is a part on all machines in a cluster. If neither a file or a file_system operand is specified, statistics for all mounted file systems, on all machines are displayed"

      ClusterIt seems a bit more complex. Tentakel may better suit your needs.

    2. By Will Maier (128.104.28.221) willmaier@ml1.net on

      > Can someone explain how this differs in capability from tentakel?
      > One minor difference is that some commands seem to have a kind of
      > abstraction layer such as normalizing the results of the df
      > command, where tentakel executes your command in the exact manner
      > it was typed everywhere so it is more suited for homogenous
      > environments.

      I'm the fella who submitted the port; the reason I like the suite so
      much is because it (or at least, several of its parts) is/are
      perfectly suited to mass administration. In our shop, we use dsh(1)
      for one-off maintenance and configuration; unless cfengine can do it
      better, we usually use clusterit's dsh. There are other dsh
      implementations, several of which I tested a while back, but I found
      that clusterit's was by far the most robust (thanks in part to the
      fact it's written in C) and useful.

      pdf(1) and prm(1) do have 'normalizing' effects; if you don't like
      that, just use dsh (as I do).

    3. By Tim Rightnour (207.225.166.13) on

      > Can someone explain how this differs in capability from tentakel? One minor difference is that some commands seem to have a kind of abstraction layer such as normalizing the results of the df command, where tentakel executes your command in the exact manner it was typed everywhere so it is more suited for homogenous environments.
      >
      > Are there any other major differences I am missing?

      (Author of clusterit responding here)

      I'm not that familiar with tentakel. I wrote this originally eons ago, probably before tentakel existed.

      The pdf command is specifically written for heterogenous environments. If you have a bunch of different machine types issuing a "df -k" out gives wildly different columns. For a homogenous cluster, there is no need for all that fancy stuff, so you can just issue "dsh df". The one nice thing about pdf however, is that there are a few filtering options, so you can see only filesystems that are almost full, or similar.

      prm was mostly a compatibility function for scripts I had that were originally written for PSSP.

      The only other option that is a bit bizzare to most people is the non-parallel option to pcp. I found that in huge environments, if you parallel copy a massive file to 100 nodes simultaneously, you pretty much kill the whole network. Especially if your source machine is on a fast uplink. Our original network was a hub-network, and we could drive the hub into pretty much constant collisions with pcp.

      Also.. clusterit ships with a parallel xterm-like program. This is rather nice for vi'ing config files on 30 machines at once. These days I use that program more than anything else.

  2. By Anonymous Coward (134.100.120.75) on

    I could not live without clusterit. I'm using it since several years on several architectures. For a small number of machines, dvt can be very useful, too (not to mention the look on the face of the ones watching you ;-) ...)

    Comments
    1. By Anonymous Coward (69.70.207.240) on

      > I could not live without clusterit. I'm using it since several years on several architectures. For a small number of machines, dvt can be very useful, too (not to mention the look on the face of the ones watching you ;-) ...)

      What kinda things do you use it for? I'm just curious.

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]