Before you install sam_bootstrap, you should install and tailor sam via:
The ups tailor sam command will allow you to add SAM configuration for each environment your SAM servers will be using. For example, if you plan to run one or more of your SAM station servers in the station_prd environment, SAM configuration with qualifier station_prd has to be added to the sam product. For more information on managing SAM configurations, please refer to the full SAM Config documentation.upd install sam -q -G -c ups tailor sam
SAM Bootstrap requires a separate directory with write privileges for the user (usually sam) who will be running the servers. This directory is used for configuration files, server log files, etc. All servers controlled by the package should be listed in the server list file. Each line of this file controls one server, and each line must have the form:
Note: as of sam_bootstrap v4_1_4, you may include comment lines in your server list file. Comments are lines that begin with the "#" character.server sam_configuration_qualifier ups_product_version [station_name] [server options]
Currently, the supported servers are:
| Server | Ups Product | Notes |
| station* | sam_station | SAM station manager and cache management. There should be ONE station master per SAM station. |
| fss* | sam_station | File Storage server, required only for storing files into the SAM system. There should be at most ONE FSS per SAM station. |
| stager* | sam_station | The station manager will, by default, start its own stagers for communicating with all of the cache disks. Separate stagers need to be started if you are using additional disks for file storage (i.e., running an FSS). If you are running an FSS, stagers are required on all nodes (master and workers). |
| bbftp | sam_bbftp | SAM bbftp file transfer protocol daemon, required if you will be using sam_bbftp as a file transfer protocol. |
| gridftp | sam_gridftp | SAM gridftp file transfer protocol daemon, required if you will be using sam_gridftp as a file transfer protocol. |
| fcp | fcp | FCP server is used for limiting total number of incoming and outgoing file transfers that use kerberos rcp as a file transfer protocol. |
| nameservice | orbacus -q services | Central name and address registry for all SAM servers. At this time, the nameservice is only supported on d0db-prd, d0db-dev and d0db-int. |
| logger | sam_logger | Central logging service for all SAM servers; currently supported only on d0db-prd, d0db-dev, and d0db-int. |
| dbserver | sam_db_server | Central SAM database service |
| dbs | sam_db_srv (replacement for sam_db_server) | Central SAM database service after the v5_1_1 schema upgrade. |
| dim | sam_dimension_server_prototype | Central SAM dimension server, required for sam_db_srv post b6_1_0_0 |
| pnfs | sam_pnfs_srv | Central SAM pnfs server, required for sam_db_srv post v6_4_0_1 |
| optimizer | sam_optimizer | Central SAM optimizer, for ordering file transfer requests; currently supported only on d0db-prd, d0db-dev and d0db-int. |
Servers which require a valid station name (i.e., station name that is known to the SAM database) are denoted by an asterisk (*). The "SAM configuration qualifier" should be one of the valid sam_config qualifiers (added during the ups tailor sam phase mentioned above), while the "ups product version" for the different servers must refer to their corresponding ups products given in the above table. Each server may be given one or more runtime options which will override the default ones.
A SAM station will typically contain one "master" node, which runs the SAM station master and is responsible for the cache management of all nodes operating within the cluster; and zero or more "worker" nodes. Sam_bootstrap will need to be tailored separately for master and worker nodes.
| Important: |
|---|
|
| Server | Option | Description |
|---|---|---|
| station | --constrain-delivery=d0mino.fnal.gov --min-delivery=1k | Files are routed through d0mino. |
| fss | <none> | Usally no options are required |
| stager | --with-fss --without-sm --max-transfers=N | MASTER node stager options |
| stager | --with-fss --with-sm --max-transfers=N | WORKER node stager options |
d0mino> setup sam_bootstrap d0mino> echo $SAM_BOOTSTRAP_ENV /usr/products/upsdb/sam_bootstrap/sam_bootstrap.env d0mino> cat $SAM_BOOTSTRAP_ENV SAM_SERVER_HOME=/home/sam/private export SAM_SERVER_HOME SAM_MAIL_RECIPIENT=local-admin@my.node export SAM_MAIL_RECIPIENT SAM_SERVER_LIST=node_based export SAM_SERVER_LIST SAM_ENCP_CONFIG="d0en" export SAM_ENCP_CONFIG
In the above example SAM Bootstrap uses directory /home/sam/private as its working directory, all e-mail notifications will be sent to local-admin@my.node, and the server list file is based on the node name, in which case
it is set to ${SAM_SERVER_HOME}/${HOST}_server_list.txt. The encp configuration is d0en.
Here is an example run on sammy (a Linux development node):
sammy:> ups tailor sam_bootstrap
$ ups tailor sam_bootstrap
Enter SAM_BOOTSTRAP_ENV [default: /fnal/ups/db/sam_bootstrap/sam_bootstrap.env]?
Using /fnal/ups/db/sam_bootstrap/sam_bootstrap.env as sam_bootstrap environment file.
Enter SAM_SERVER_HOME [default: /home/sam/private]?
SAM_SERVER_HOME is set to /home/sam/private.
Enter SAM_MAIL_RECIPIENT [default: sam-auto@fnal.gov]? lauri@fnal.gov
SAM_MAIL_RECIPIENT is set to lauri@fnal.gov.
Enter SAM_SERVER_LIST [default: /home/sam/private/sammy_server_list.txt]?
SAM_SERVER_LIST is based on node and set to /home/sam/private/sammy_server_list.txt.
Enter appropriate ENCP configuration [default: d0en]?
SAM_ENCP_CONFIG is set to d0en.
Server list file /home/sam/private/sammy_server_list.txt does not exist.
During configuration process temporary file /tmp/lauri_sammy_server_list.txt
will be created. This file will have to be moved by user sam to
/home/sam/private/sammy_server_list.txt.
Now determining default versions of server software, please stand by...
.....................
Ready to configure your servers.
Configure a station? [default: false]: y
Enter the environment? station_prd
Enter the sam_station product version: [default: v4_1_0_3]?
Station name? d0-demo-station
Options to the server [default: None]? --constrain-delivery=d0mino.fnal.gov --min-delivery=1k
Configure another station? [default: false]:
Configure a stager? [default: false]: y
Enter the environment [default: station_prd]?
Enter the sam_station product version: [default: v4_1_0_3]?
Station name [default: d0-demo-station]?
Optional server configuration file [default: None]?
Configure another stager? [default: false]:
Configure a stager_old? [default: false]: n
Configure an fss? [default: false]: y
Enter the environment [default: station_prd]?
Enter the sam_station product version: [default: v4_1_0_3]?
Station name [default: d0-demo-station]?
Optional server configuration file [default: None]?
Configure another fss? [default: false]:
Configure an optimizer? [default: false]:
Configure a dbserver? [default: false]:
Configure a logger? [default: false]:
Configure an infoserver? [default: false]:
Configure a nameservice? [default: false]:
Configure fcp? [default: false]:
Configure a sam_bbftp daemon? [default: false]: y
Enter the sam_bbftp product version: [default: f1_9_4c]?
Configure a gridftp? [default: false]: y
Enter the environment? dev
Enter the sam_gridftp product version: [default: v1_6]?
Options to the server [default: None]?
Configure another gridftp? [default: false]:
Using file /tmp/lauri_sammy_server_list.txt:
station station_prd v4_1_0_3 d0-demo-station
stager station_prd v4_1_0_3 d0-demo-station
fss station_prd v4_1_0_3 d0-demo-station
bbftp station_prd f1_9_4c
gridftp station_prd v1_6
for installing needed ups products...
Installing encp -G "-c -q d0en"...
informational: encp v2_17 already exists on local node, skipping.
Installing orbacus -q sam_station -G "-c"...
informational: orbacus v3_3_4 already exists on local node, skipping.
Installing sam_bbftp f1_9_4c -G "-c"...
informational: beginning install of sam_bbftp.
informational: transferred
/ftp/products/sam_bbftp/f1_9_4c/Linux+2/sam_bbftp_f1_9_4c_Linux+2
from fnkits.fnal.gov to
/fnal/ups/prd/sam_bbftp/f1_9_4c/Linux+2
informational: transferred
/ftp/products/sam_bbftp/f1_9_4c/Linux+2/sam_bbftp_f1_9_4c_Linux+2/ups/.
from fnkits.fnal.gov to
/fnal/ups/prd/sam_bbftp/f1_9_4c/Linux+2/ups
informational: transferred
/ftp/products/sam_bbftp/f1_9_4c/Linux+2/sam_bbftp_f1_9_4c_Linux+2.table
from fnkits.fnal.gov:/ to
/fnal/ups/db/sam_bbftp/f1_9_4c.table.new
DECLARE: A UPS tailor exists for this product
informational: ups declare succeeded
DECLARE: A UPS start/stop exists for this product
informational: ups declare succeeded
Installing sam_station v4_1_0_3...
informational: sam_station v4_1_0_3 already exists on local node, skipping.
Installing sam_user -G "-c"...
informational: sam_user v4_1_1 already exists on local node, skipping.
Execute the following to resolve chains:
ups declare -f Linux+2.2 -q "" -g current sam_user v3_0_2 -z /fnal/ups/db
==================================================================
If you haven't already done so, you will need to tailor sam_bbftp.
To do this, from a product maintainer account:
ups tailor sam_bbftp f1_9_4c
Then, from the sam account:
ups installAsSam sam_bbftp f1_9_4c
You need to initialize the server_list file.
Please execute the following command as user sam:
cp /tmp/lauri_sammy_server_list.txt \
/home/sam/private/sammy_server_list.txt
In the above example the default values for many variables were accepted (note the empty response lines). The server list file did not exist, and therefore had to be created. The user is prompted for the necessary information for each type of server that might be needed. All required products are then automatically installed. (Any ups declare commands printed out after installation of the needed ups products can usually be ignored).
At the end of the configuration, the temporary server list file (owned by the product maintainer) must be copied into the actual server list file owned by user sam.
If sam_bbftp hasn't already been installed on this node, you will need to perform the tailoring and installAsSam steps:
$ id
uid=1275(lauri) gid=4525(products) groups=1530(g023),4525(products),4525(products)
$
$ ups tailor sam_bbftp
The sam_bbftp product requires an authorization file
that is owned by, and readable only by, user sam.
This file should not be in an NFS-exported area.
Ideally, it should be located in a secure area
where only 'root' has privileges. If this is
not possible, a non-shared file system such as
/tmp is satisfactory.
NOTE, this file must exist and be readable by user
sam from both sam_bbftp clients and servers.
Enter the full path to the sam_bbftp authorization file:
(default: /tmp/.sam_bbftp/.sam_bbftp_cookie)
You should now create the sam_bbftp authorization file
by using the command
ups installAsSam sam_bbftp f1_9_4c
You will need to know the appropriate authorization string
to enter when requested.
You must use an account which can create the
/home/sam/.sam_bbftp/.sam_bbftp_cookie file and set the file ownership
to user sam. (Usually, this is the sam account).
$
$ ksu sam
Authenticated lauri@PILOT.FNAL.GOV
ksu[10130]: 'ksu sam' authenticated lauri@PILOT.FNAL.GOV for lauri on /dev/pts/0
Account sam: authorization for lauri@PILOT.FNAL.GOV successful
ksu[10130]: Account sam: authorization for lauri@PILOT.FNAL.GOV successful
Changing uid to sam (7816)
sammy%
sammy% ups installAsSam sam_bbftp
This is the sam_bbftp authorization procedure.
Please enter the sam_bbftp authorization string:
Now creating sam_bbftp authorization file /tmp/.sam_bbftp/.sam_bbftp_cookie
Authorization is complete.
In case fcp server is used for controlling the maximum number
of rcp transfers, the fcp ups product has to be configured.
Note that this is only useful if fcp -c
wrapper is used for sam_kerberos_rcp.
Assuming that the fcp root directory is set to $FCP_ROOT
the following commands have to be executed from a product
maintainer account:
The necessary directories have to be created and configuration files have to be edited from the sam account:ups tailor -O $FCP_ROOT -f $OS_FLAVOR $FCP_PRODUCT_VERSION ups tailor sam_kerberos_rcp (use "fcp -c" for rcp wrapper)
At this point edit the $FCP_ROOT/cfg/fcp.cfg file, and decide on is the maximum number simultaneous transfers. If logging isn't necessary, comment out the log_file variable, otherwise set it to $FCP_ROOT/log.mkdir -p $FCP_ROOT/cfg mkdir -p $FCP_ROOT/bin mkdir -p $FCP_ROOT/log setup fcp $FCP_PRODUCT_VERSION cp $FCP_DIR/cfg/fcp.cfg.example $FCP_ROOT/cfg/fcp.cfg
If sam_gridftp hasn't already been installed on this node, you will need to install it and it's dependencies, by following these temporary instuctions. This is due to a bug in ups.
INSTALLATION OF GLOBUS DATA HANDLING TOOLS AND
GLOBUS SECURITY INFRASTRUCTURE CONFIGURATION
---------------------------------------------
Each product gives instructions on what to do next.
Please follow instructions on the screen carefully.
The following steps describe the procedure
for a system that is installing globus for the first
time.
-------------------
Installation of the globus data handling server tools
- as products:
> upd install globus_dh_server -G-c
-------------------
Installation of the globus security infrastructure
- as root:
# setup globus_dh_server
# ${GLOBUS_LOCATION}/setup/globus/setup-gsi
-------------------
Installation of the globus data handling client tools
- as products:
> upd install globus_dh_client -G-c
-------------------
Installation of the globus security infrastructure
configuration tools
- as products:
> upd install sam_gsi_config -G-c
-------------------
Configuration of the globus security infrastructure
and request of the sam server certificate: ONLY THE
FERMILAB Kerberos CA IS TRUSTED.
-as root:
# ups tailor sam_gsi_config
# setup sam_gsi_config
# sam_cert_request
You will need to repeat the last 2 steps on each machine requiring
gridftp server or client access.
-------------------
Configuration of the authorization file (grid-mapfile)
for the sam servers (putting the following line in
a cron job updates the list automatically)
-as sam
> ups get_gridmap sam_gsi_config
---------------------------------------------------------
INSTALLATION PROCEDURE FOR SAM_GRIDFTP.
This procedure should be improved by listing sam_gridftp
in the tailoring script of sam_bootstrap
-------------------
Install the globus data handling tools and GSI (instructions
above)
-------------------
Install the new sam products that know about gridftp:
-as products:
>upd install sam_gridftp
Eventually this step will be done by sam_cp. Currently I prefer
to do it by hand, since a bug in upd (ticket 27906) may cause
the installation of globus (if not there already) to hang the
sam_cp installation.
>setup globus_location
>ups tailor sam_gridftp
(defaults OK)
>upd install sam_cp -G-c
>edit sam_cp_config file to include sam_gridftp as a protocol
for your sam installation and include the new protocol, as
by instructions on the screen (file to modify:
${UPS_THIS_DB}/sam_cp_config/Config/sam_cp_config.py)
d0mino> ups inquire sam_bootstrap SAM_SERVER_HOME is set to "/home/sam/private". SAM_MAIL_RECIPIENT is set to "sam-auto@fnal.gov". SAM_SERVER_LIST is based on node and set to "/home/sam/private/d0mino_server_list.txt". SAM_ENCP_CONFIG is set to "d0en".
In addition to the above commands, one can also start or stop a particular server using the following procedure (as user sam):
logon as sam
setup sam_bootstrap
ls ~sam/private/*.txt (retrieves a text file)
cat text_file found above
from the text_file, find the version dbserver,logger, etc. is running
exec run.sh ACTION SERVER QUALIFIER UPS_PRODUCT_VERSION [STATION_NAME] &
Here, "action" can be either start or stop, while the other arguments are the same as before. For example, in order to start the development DB server from the ups product sam_db_server v2_2_2 on d0ora3, one would login to d0ora3 as user sam, and execute:
<d0ora3> setup sam_bootstrap <d0ora3> exec run.sh start dbserver dev v2_2_2 ""&
As another example, in order to stop the development station central-analysis on d0mino, from the ups product sam_station v2_2_3 one would do (again as user sam):
<d0mino> setup sam_bootstrap <d0mino> run.sh stop station dev v2_2_3 central-analysis
| Updated 20Aug2002 to reflect new server list file configuration!!! |
Node d0mino hosts the central-analysis sam station, the fss stagers for several remote sam stations, and the station master for the cab (central-analysis-backend) sam station. Each of these "types" of server are listed in separate server list files.
Before invoking any sam_bootstrap command, you must make sure that your SAM_BOOTSTRAP_ENV environmental variable is pointing to the correct server list environment:
| central-analysis | What to set up: | $ setup sam |
| $SAM_BOOTSTRAP_ENV: | /usr/products/upsdb/sam_bootstrap/CENTRAL-ANALYSIS_bootstrap.env | |
| Server_list file: | ~sam/private/CENTRAL-ANALYSIS_server_list.txt | |
| cab | What to set up: | $ setup -q cab sam |
| $SAM_BOOTSTRAP_ENV: | /usr/products/upsdb/sam_bootstrap/CAB_bootstrap.env | |
| Server_list file: | ~sam/private/CAB_server_list.txt | |
| remote-stager | What to set up: | $ setup -q remote-stager sam |
| $SAM_BOOTSTRAP_ENV: | /usr/products/upsdb/sam_bootstrap/REMOTE-STAGER_bootstrap.env | |
| Server_list file: | ~sam/private/REMOTE-STAGER_server_list.txt |
The default d0mino_server_list.txt file is a placeholder only, and should not contain any "real" server entries.
To upgrade the central-analysis station, use the following steps:
# # as user sam: # $ setup sam # set default value for SAM_BOOTSTRAP_ENV $ setup sam_bootstrap # initialize default bootstrap environment $ ups inquire sam_bootstrap # determine location of server list file $ <edit> server_list_file # change product versions, add/remove servers, etc. # # as user products: # $ ups tailor sam_bootstrap # install appropriate SAM products # # as user sam: # $ ups update sam_bootstrap # stop obsolete servers, start new servers
Similar steps are used to update the other stations/servers, but make sure to set up the correct SAM_BOOTSTRAP_ENV environment first.
The IO node the upgrade procedure would be as follows:
#
# as user sam on node d0bbin:
#
$ setup sam_bootstrap # initialize default bootstrap environment
$ export SAM_BOOTSTRAP_ENV=/d0farm/ups/db/sam_bootstrap/sam_bootstrap_d0bbin.env
# use d0bbin bootstrap configuration
$ ups inquire sam_bootstrap # determine location of server list file
$ <edit> server_list_file # change product versions, add/remove servers, etc.
#
# as user d0flib on node d0bbin:
#
$ setup sam_bootstrap
$ export SAM_BOOTSTRAP_ENV=/d0farm/ups/db/sam_bootstrap/sam_bootstrap_d0bbin.env
$ ups tailor sam_bootstrap # install appropriate SAM products
#
# as user sam:
#
$ setup sam_bootstrap
$ export SAM_BOOTSTRAP_ENV=/d0farm/ups/db/sam_bootstrap/sam_bootstrap_d0bbin.env
$ ups update sam_bootstrap # stop obsolete servers, start new servers
For the worker nodes one cannot use the update command, since all nodes use the same startup log file, and the update command would work correctly only for the first node on which it would be executed. Because of that, one has to restart all servers on all worker nodes.
#
# as user sam on one of the worker nodes:
#
$ setup sam_bootstrap # initialize default bootstrap environment
$ export SAM_BOOTSTRAP_ENV=/d0farm/ups/db/sam_bootstrap/sam_bootstrap_worker.env
# use worker node bootstrap configuration
$ ups inquire sam_bootstrap # determine location of server list file
$ <edit> server_list_file # change product versions, add/remove servers, etc.
#
# as user d0flib on one of the worker nodes:
#
$ setup sam_bootstrap
$ export SAM_BOOTSTRAP_ENV=/d0farm/ups/db/sam_bootstrap/sam_bootstrap_worker.env
$ ups tailor sam_bootstrap # install appropriate SAM products
#
# as user sam on all worker nodes:
#
$ setup sam_bootstrap # initialize default bootstrap environment
$ export SAM_BOOTSTRAP_ENV=/d0farm/ups/db/sam_bootstrap/sam_bootstrap_worker.env
# use worker node bootstrap configuration
$ ups restart sam_bootstrap # stop/start all services
# # as user root: # - create sam account - create sam working directory (typically, ~sam/private) and set ownership to user sam # # as user products: # $ setup upd $ upd install sam # install sam_user portion $ upd tailor sam # add needed configurations $ upd install -G"-c" sam_bootstrap # install sam server portion $ ups tailor sam_bootstrap # follow the instructions # # as user sam: # - copy the temporary server_list.txt file into the actual server_list.txt file $ setup sam_bootstrap $ ups start sam_bootstrap
Note that, while the above steps are sufficient to start all configured SAM servers, in the case of SAM station one has to perform certain administrative tasks after station is up and running. These tasks include adding and configuring groups, adding disks to the station cache, etc.
Places to look for information:
============================================================================= Project : SAM Package : sam_bootstrap $Id: index.shtml,v 1.2 2002/04/16 15:43:42 lauri Exp $ This work is part of a development project, called SAM, which consists of a number of coordinated packages each named sam_xxxx. Notice of authorship, copyright status, and terms and conditions, should the software eventually become available for use outside Fermilab, can be found in the README and LICENCE files in the top level directory of the main sam package. =============================================================================