Building Htcondor On Linux

Overview

As of version 7.5.5, HTCondor uses cmake to configure the build. For instructions on building HTCondor prior to that, see BuildingHtcondorOnUnixOld .

Confirm the build environment

The README.building does a decent job of covering this, but usually you'd basically need these revisions, or later: wget-1.9.1, tar 1.14, autoconf-2.59.

Space Needed for a Full Build

You may need around 6 Gigs to build a releasable package of HTCondor. If you just want to build eveything up to the releasable package, then you might need only 3 Gigs or so.

Required Prereqs

One needs, as a good start these revisions, or later, of these tools: wget-1.9.1, tar 1.14, autoconf-2.59. If you are on a recent linux-flavor machine, building HTCondor is pretty easy, the farther you get into the fringe architectures, like ia64 hpux 11, the more prereqs you may need. Luckily, the configure output is pretty good about telling you about any tools you need to update.

From some experimentation, here's a list of pre-requisite RPMs necessary for a UW-style build on SL6 as of version 8.6.0:

  • cmake28 (from EPEL)
  • perl
  • gcc
  • gcc-c++
  • gcc-gfortran
  • libicu-devel
  • boost-test
  • boost-devel
  • pcre-devel
  • keyutils-libs-devel
  • krb5-libs
  • openssl-devel
  • uuid-devel
  • uuid-c++-devel
  • libuuid-devel
  • libxml2-devel
  • pam-devel
  • zlib
  • libtool-ltdl-devel
  • libtool
  • glibc-static
  • byacc
  • libX11-devel
  • libcurl-devel
  • perl-Archive-Tar
  • expat-devel
  • python-devel
  • bison
  • flex
  • nss-devel
  • openldap-devel
  • c-ares-devel

For package list for Ubuntu see: BuildingHtcondorOnUbuntu

Getting the source

Directly from the GIT repository

If you reside on the CSL networks and/or have access to our GIT repository, then follow ManagingCondorSourceTreesWithGit up to but not including the section entitled Working on a single person project .

Ensure you have checked out and are about the build the correct branch you want.

If you'd like to perform the full build process, producing the sort of package one downloads from our website with the source, then you should grab the tarball of man pages make public needs from AFS:

# sitting at the toplevel with src/ config/ externals/ etc....
% cp /p/condor/workspaces/externals/bundles/man/current/man-current.tar.gz externals/man/current

From our download pages

If you are building HTCondor sources from our download page. Then download the source tarball, it'll have a name similar to condor_src-X.Y.Z-all-all.tar.gz . X.Y.Z represents the version of HTCondor for which the source creates.

When you untar the source tarball, what you get is remarkably similar to what one would check out of GIT and should be directly buildable. You will have available in the externals directory the tarball of manual pages needed by our packaging scripts.

Externals required for Building

HTCondor may use a sizable collection of externals which implement various feature sets for HTCondor. Some examples are Kerberos, PostgreSQL, Globus. HTCondor sources include an externals/ directory which contains URLs to locate the required externals and patches to be applied. There is only a small number of externals that HTCondor absolutely requires to build, these are usually quite portable.

Required Prereqs

One needs, as a good start these revisions, or later, of these tools: cmake 2.8.3, wget-1.9.1, tar 1.14, autoconf-2.59. For a more complete list, run nmi_tools/glue/SubmitInfo.pm and look at the listed prereqs for a platform as similar to the one you are using as possible.

Externals required for Building

HTCondor may use a sizable collection of externals which implement various feature sets for HTCondor. Some examples are Kerberos , OpenSSL , Globus . There are only a small number of externals that HTCondor absolutely requires to build; these are usually quite portable. There are two ways to link with external packages, using the blessed and patched versions of the packages from the UW HTCondor externals collection, or using the native libraries installed on the build machine. We'll call these the 'UW' way and the 'proper' way. To get externals the UW way, HTCondor sources include an externals/ directory which contains URLs to locate the required externals and patches to be applied. To get externals the 'proper' way, you'll need to use your system's package manager to install the necessary development libraries.

Configure your build

See the new build instructions

The common options for configuring HTCondor to be built the 'UW way' are passed to cmake by running configure_uw . This will configure the build to use the UW externals collection rather than local system libraries.

Additional arguments to cmake may be passed on the command line of configure_uw . On most common platforms, no additional build options are required. For other platforms, there are several ways to explore the build options:

  • ccmake
  • cmake-gui
  • cmake -i
  • Use nmi_tools/glue/SubmitInfo.pm , which shows the build options that are used to build HTCondor on a wide variety of platforms in the NMI build system.
    • To list the platforms that SubmitInfo.pm knows:
      • ./nmi_tools/glue/SubmitInfo.pm -l
    • To list the options for a particular platform
      • ./nmi_tools/glue/SubmitInfo.pm < platform >
      • Example: ./nmi_tools/glue/SubmitInfo.pm x86_64_opensuse_11.3
    • You can also specify a regex:
      • ./nmi_tools/glue/SubmitInfo.pm < /regex/ >
      • Example: ./nmi_tools/glue/SubmitInfo.pm /opensuse/
  • To build with the default options for your platform:
    ./configure_uw
    
  • If you want to explicitly do a clipped port ('clipped' means no standard universe, no checkpointing, no checkpoint server)
    ./configure_uw -DCLIPPED:BOOL=ON
    

NOTE :

  • For out of source builds simply make a directory above CONDOR_SRC e.g. 7.5.5 and cd into it, then type:
    • UW: ../CONDOR_SRC/configure_uw ../CONDOR_SRC
    • Everyone else: cmake ../CONDOR_SRC
      • Development Recommendation: alias cmake='cmake
  • Builds by default cache the externals under bld_external/ in your build directory. You can change where the externals are installed by setting the environment variable CONDOR_BLD_EXTERNAL_STAGE to point to the desired location. As a given external always builds the same on a given platform, using a single externals location for several build directories can save unnecessary recompilation.
  • You might be interested in other CmakeBuildOptions

Building your source

While there are many targets to make , I will only describe the two that are most likely what you want.

install

make install will make a set of executable binaries and place them in release_dir/ . They will be dynamically linked and suitable for testing by pointing a $(RELEASE_DIR) at it from a condir_configure file.

package

make package will produce packages similar to what you can download from the UW download site for the machine upon which you are building.

Running the developer test suite

Building the tests

$ make tests

Running the tests

See TestingCondorOnUnix .