Using Alternative Batch System with SAM Station
Audience: Station Administrators who wish to adapt SAM station to their
local batch system.
Pre-requisites: running
jobs with SAM.
1. Introduction
Aside from moving the data, the SAM system manages computing resources.
The primary
goals of resource management are essentially implementation of the
experiment's policies on "fair share" resource allocation as well resource
optimization. While the data handling part of SAM manages resources connected
with the actual data transfers (MSS and network bandwidth, disk, etc),
it strives to also co-allocate "compute" (CPU etc) resources with the data
resources for user jobs. In the initial resource management in SAM, the
Station Master (SM) interacts with the underlying Batch System (BS) for
approaching both of the abo`ve stated goals.
2. The SM-BS Interaction
The SM interacts with the Batch system via an abstract adapter interface
(details later). All the specifics of the underlying Batch System are hidden
into the implementation of the adapter. As will be seen later from the
adapter interface specification, the functions of an adapter for a particular
Batch System are:
1) inform SAM how to submit jobs into the BS (i.e. the exact
command line);
2) interpret the result of a submission command and return to SAM the
string ID of the newly submitted job;
3) ensure that the submitted jobs are not actually dispatched by the
BS until SAM instructs the BS adapter that the input data required by the
job has arrived, by means of e.g., imposing an additional resource requirement
on the job. SAM
4) when notified by SAM of the data delivery for the job, (see 3) propagate
the event to the batch system so that the job can be executed whenever
its other requirements are met. Note that SAM treats the jobs as consumers
for the associated project (the user should already be familiar with the
concepts).
5) (optionally) discriminate between higher and lower priority jobs
as decided by the SAM system which takes into account the history of resource
consumption. Note that in SAM, depending on the station, the data handling
resources may play more important role than compute resources. SAM "penalizes"
jobs of users whose recent jobs have e.g., required many tape mounts or
high network traffic to the station.
In order to better understand the underlying design, please see the
relevant design pages. The following collaboration diagram depicts the
actual interaction of SAM (the Station Master as well as its clients) with
the Batch System:
3. The Code
The CD CVS package sam_station is written in C++ and contains the
abstract class sam::BSAdapter defined in the file sam_station/StationBSAdapter.hpp.
One will need to obtain a working snapshot of the code (which also relies
on other packages, namely sam_util, sam_common_idl, sam_db_server_idl,
sam_optimizer_idl and sam_station_idl) and develop the code in one's working
area prior to committing the code back to SAM. Some tips
from Lauri on how to work with SAM code may be useful. Contact sam-design@fnal.gov
with questions.
=============================================================================
Project : SAM
Package : sam
$Id: bs_adapters.html,v 1.2 2001/05/30 16:32:29 terekhov Exp $
This work is part of a development project, called SAM, which consists
of a
number of coordinated packages each named sam_xxxx .
Notice of authorship, copyright status, and terms and conditions,
should
the software eventually become available for use outside Fermilab,
can be
found in the README and LICENCE files in the top level directory
of the main
sam package.
==============================================================================