SAM WBS Tasklist (updated 2001/01/30)

1 SAM Infrastructure and Project ManagementJune 1998 to March 2001 0.25 FTE average+reviews13 FTEweeks to March + 6 FTE weeks to Nov 01 -
11  Project ManagementTasklists and WBS - -
12tasklist Web pages - - -
13  Product installation and buildsOrbacus, Fnorb, Jclass, Java and Python, Oracledb - -
14  Database Insfrastructureinitial tests and setup -> support databases - -
15tasklist Project standards and code organizationand version cutting methodology - -
16  Training and Education of Team memberse.g. Python, Softreltools, CORBA, Java Servlets - -
17tasklist Demos, Reviews and Talks - - -
18tasklist Code Reviews - - -
19tasklist User documentation and help facilities - -
2 Requirements and Conceptual DesignsOctober 1997 to June 1999(extended)1 FTEweek1 EXPweek
21  Initial Requirements Specification - - - 100%
22  Glossary of Termsneeds updating in December2 FTEdays - 90%
23  Architecture and conceptual design - - - 100%
24  Revised Requirements Specs ??wasnt done after V0 - limited opportunities now needs Expt input - ? EXPdays0%
3 Designs and Interfaces to other systems and ProductsJune 1998 to November 15 1999 (extended and further extended)17 FTEweeks (to March) + 20 FTEdays after March -
31  Enstore Interface? verification, control of shelf tape 2 FTEdays - 95%
32  SAM Servers Design - - 100%
33tasklist Farm System Interfacefile merge and Output node remains5 FTEdays5 Expdays85%
34  RIP and Online System interface (re-opened) - - - 100%
35  User Interface C++ V0now only in context of framework - - 100%
36  User Interface command line V0 - - - 100%
37  User Interface C++ V1all in context of framework - - 100%
38  User Interface command line V1 - - - 100%
39  D0 Framework interface, input package and RCP design - - - 100%
310  Database Schema V0including minimal server state persistency - - 100%
311  Database Schema V1including minimal Triggers and limited browsing capability and reports - - 100%
312  Database Schema V2final full database design and design of browsing capability and reports, including Luminosity, Triggers, System config10 FTEdays -
313  File Catalog Meta-data design - - - 100%
314  Tape format and Tape Import via Enstore direct tape import and sync with SAM - - 100%
315  Monte Carlo import facility - 1 FTEday10EXPdays50%
316  Optimizerinitial algorithm for control by access mode12 FTEdays - 5%
317  Batch system interface and other Resource Management15 FTEdays - 35%
318  Event Catalogsand full indexes5 FTEdays - 50%
319  File Delivery and StorageProject-Servers-User interaction for files read and write + File Merging - - 100%
320  Event Delivery and StorageProject-Servers-User interaction for Events (Pick Events)5 FTEdays -
321  Log file specification V0 - - - 100%
322  Log file specification V1 - - - 100%
323  System Statistics and views V0 - - - 100%
324  System Statistics, Information and views V1integration with Netlogger? Ops? 5 FTEdays - 50%
325  Archival of system state and statisticsextracts of log files? (after March) 5 FTEdays -
326  Data ExportData export facility and exportable SAM system and database (after March)15 FTEdays - 10%
327  Error Handling, Robustness and Redundancy study - - - 100%
328  Database server -multi-connection/multi-thread design - 3 FTEdays - 70%
329  Project definition extensions - 2 FTEdays - 90%
330  System Design Documentjoin of all above- for a review10 FTEdays - 0%
332  Database Schema V2.5coordinated with online database schema5 FTEdays - 5%
4 Data Organization and ContentOctober 1997 to December 19991 FTEweek? 17+ Expweeks
41  Streaming Model 2+ EPW20%
42  Farm Input and Output Files and Streams Specification-> merge/split design - ? FTEdays50%
43  Data Tiers size and content ? FTEdays30%
44  Data Export to remote institutions - strategies 5 FTEdays10 EPW
5 Demonstration Prototype of SAMJune 1998 to December 1998--COMPLETED--December 1999 -
51  CORBA Infrastructure use of IDL and Name service - - 100%
52  Data Delivery (User) example C++ consumer with waits, timeouts and errors - - 100%
53  Project and snapshot definitionprototype GUI and scripts - - 100%
54  Database - - - 100%
55  Log Server - - - 100%
56  Database Serverfor Consumer registration and File Consumption - - 100%
57  Optimizerdummy version with random wait - - 100%
58  Project Masterimplementing Freight Train and Farm mode with few 10s of consumers for each project - - 100%
59  Administrative Console for system exploration and statistics and graphs of latency and throughput - - 100%
6 V0 Working Prototype of SAMDecember 1998 to May 15, 1999--COMPLETED-- July 15 1999 -
61  Operate prototype and monitor database as real data added/used - - - 100%
62tasklist Database V0 production database on d0ora2 - - - 100%
63tasklist Database ServerPython with generated code and more persistency and retrieval services + full exception handling and error reporting - SW,MV100%
64  Log Serveradditional events as determined by running stress tests and viewing statistics - - 100%
65  Administrative Tool/Web viewable system status and graphsBuild on standalone Java program - - 100%
66  Command line project and snapshot definition and investigationallow saving of simple queries and auxiliary file containing file list - - 100%
67tasklist Database Browsing/Web based queries and static pagesSpecific Pages for MC - -
68tasklist Project Master/scalable to several hundred clients - data deliveryremove thread restrictions ??IT100%
69tasklist Data Storage (User)command line interface and meta-data verification - IT100%
610  Data Storage (System)Station servers and Enstore interaction - IT100%
611  Data Delivery (User)command line interface - defer V0 C++ package - IT100%
612  Project Master w/o real SMminimal Cache management, real SM deferred to V1 - - 100%
613  Optimizer deferred to V1 - -
614tasklist Support Tables Maintenance - - - 100%
615  Overview of SAM and User's Guide V0partially done - rest deferred to V1 - -
7 V1 Limited Functional SAM System - for UsersJune 15, 1999 to October 1, 1999--COMPLETED-- Oct 15,1999
71  SAM Manager integrated in D0 frameworkincluding RCP,metadata,cmd line options - - 100%
72  V1 of Database Schema on d0ora2need cutover scripts - - 100%
73  First Event Catalog in dbwith primitive web browser - - 100%
74  DB Server with support for data loggerfirst version - not extensively tested, needs upgrades to event catalog loading - - 100%
75  File Storage Serverfor RAW, MC and reconstructed data files - - 100%
76  Improved Project Masterwith preferred location and // fetch control - - 100%
77  User python scriptsfor launching applications - - 100%
78  Admin/Statistics tools available on web - - - 100%
79  V1 of Project Toolswith project editor GUI - - 100%
710  Test harness to exercise system continuouslysam cluster and d02ka, only simple.py used - - 100%
711  Special support for RIP/Data Logger? verification, batch updates? will need more - - 100%
712  Initial User Guide - - - 100%
8 V1.5 SAM System - with Station and Disk CacheTarget date December 15, 1999--COMPLETED Mar 22, 2000-- -
81  Bug fixes and urgent extensions - - - 100%
82tasklist Project Master extra features restart, run on failed files, subset of files - - 100%
83  Configure d0 systems so ALL servers start up at boot timeups/upd issues - - 100%
84tasklist File storage server and store commands - extra features - - 100%
85  V2 Project Tools -- Deltas, Sums, combinations of constraints, get rid of SQL bag it until V2 - design only - see 3.29 - - X
86tasklist Station with disk cache and buffer management - - 100%
87tasklist Framework metadata for output files - - 100%
88tasklist MC card files and other parameter files -
8A V1.6 SAM System - minor fixes and improvements to V1.5 approx 3.0 FTEs on feature development. Target date May 1 2000-- COMPLETED -- July 5, 2000 -
8A1  Bug fixes and urgent extensions - - -
8A2  D0om name expander - - -
8A3  sam_bootstrap productfor ensuring startup of all servers - -
8A6  Station/disk cache and buffer management improvements and bug fixes, file locking - -
8A9  CORBA utilities into sam_util work on sam_manager - -
8A10  Database server changes in support of station - - -
9 V2.0 SAM System - for cosmic ray data-taking approx 3.0 FTEs on feature development. Target date June 26 2000--COMPLETED(alpha) V2.1-August 17, 2000 -
91  Bug fixes and urgent extensions - 7 FTEdays - 80%
92tasklist Project Master/Consumer robustness restart consumer on PM crash1FTEdays - 95%
93tasklist File storage server and store commands - async mode DLSAM and Farm i/f2 FTEdays - 95%
94tasklist File merge/split implementation(after design completed)7 FTEdays
95tasklist Project definition and classificationimplement new design2 FTEdays - 80%
96tasklist Auto destination mapping1st version of1 FTEdays - 85%
97tasklist MC file import facility automate + extensions for MC request fns2 FTEdays - 10%
98tasklist Database and tools upgrade for V2.0in support of 9.4-9.6 - 100%
99tasklist Status information, browsing and diagnostics - 11 FTEdays - 10%
9A V2.2 SAM System - parallel small upgrade while working on V3.0 approx 1.5 FTEs on development. Target date September 15, 20007.5 FTEweeks -
9A1  Bug fixes and urgent extensions - 8 FTEdays -
9A2tasklist Project Master/Consumer robustness unit test and add to test harness1FTEdays
9A3tasklist File storage server and store commands - async modefinish DLSAM and Farm i/f2 FTEdays10 Expdays
9A4tasklist File merge/split implementation(after design completed)7 FTEdays
9A5tasklist Project definition and classification2nd version5 FTEdays
9A6tasklist Auto destination mappingimprove1 FTEdays
9a7tasklist MC file import facility automate + extensions for MC request fns2 FTEdays -
9A8tasklist Database and tools upgrade for V2.0in support of 9.4-9.61 FTEdays
9A9tasklist Status information, browsing and diagnostics - 11 FTEdays -
10 V3.0 SAM System - with luminosity, database sync, resource management,batch interface, file merge/split avg. 3.0 FTEs on feature development. New Target date November 15, 200027 FTEweeks -
101  - - -
11 V4.0 SAM System - ready for data taking approx 3.0 FTEs on feature development. Target date January, 30 2000 27 FTEweeks -
111  - - -
12 V5.0 SAM System - thumbnail support, tune resource management approx 3.0 FTEs on feature development. Target date March 15 200119 FTEweeks -
121  - - -
13 V6.0 SAM System - full remote site/export support Target date May 15 2001 - 3.0 FTEs on feature development 21 FTEweeks -
131  - - -
19 Test systems and software - procure and installSeptember 1998 to May 19991FTEweek -
191  SAM PC Testbed10 Linux PCs - - 100%
192  JClass graphing package - - - 100%
193  D0 Database Servers and disksneed to size final machine and disks for FY20015FTEdays - 90%
194  Find machine to test batch softwared0test - - 100%
195  Prototype Analysis Server Machine to run loads ond0test - - 100%
20 Access Control and AuthenticationMarch 2000 to May 20004+ FTEweeks -
201tasklist Servers recovery and restartmade part of test harness->22.16
202  Databaseexercise recovery procedures-> test harness 22.17
203  Access control and authentication designcoordinate with Strong Authentication for file transfer/FTP4FTEweeks -
204  Implement authentication and access controlASSUME NOT! - -
21 Operations and SupportMarch 1999 to 2001 and beyond22 FTEweeks to March 01 @ 0.5 -> 1.0 FTE constant -
211  Set up SAM User's group and meetingsongoing help -
212  Mailing listplus mail archive - - 100%
213  Bug report and fixes facilityUse of Remedy - -
214  Prepare Run I Data for import? Not being done in SAM - - X
215  Import Run I Production catalog and move Run I data? possibly only catalog- not in SAM - - X
216  RunII operationsuser help,problem resolution etc -
217  Data access statistics and monitoring - 4FTEweeks -
218  Supporting data maintenancee.g. application,group,data tier, etc - ? Exp FTEs
219  User and group maintenance - -
22 Tests, Simulations and Integration and Monitoring ToolsOctober 1997 to March 01 and beyond 22 FTEweeks -
221  Spreadsheet simulations of data ratesSeasons of SAM - - 100%
222  Tests with Enstore to fake robot - - - 100%
223  Tests with Enstore to STK robot - - - 100%
224  Tests with Enstore to EMASS robotinitial tests - - 100%
225  Overnight tests with Enstore to EMASSpermanently assigned cartridges and library - - 100%
226  Farm software integration and testsneed to convert to framework code and scripts3FTEdays - 80%
227  D0 Framework integrationinput and output packages and RCPs - - 100%
228  Farm Merge and "O" node tests - 15FTEdays
229  Tests with Monte-Carlo import facility - 5FTEdays
2210  RIP and Online integrationcreate instrumented logger for Test harness2 FTEdays - 50%
2211  Revisit data rates spreadsheetor do new type of simulation using test harness - - X
2212  Test Harness - stress testssimple.py tests only on sam cluster and d02ka - - 100%
2213  d0test disk throughput and contention tests - 5 FTEdays - 50%
2214  d0test network throughput and round-robinwith Enstore2 FTEdays 90%
2215  Test Plan requested by Ruth -
2216tasklist Test Harness and stress tests using all systems- sam cluster, d0test,farms 15 FTEdays -
2217tasklist Database load tests and monitoring activities incorporate work from 23.10 5 FTEdays -
2218tasklist Information server and sam_monitor upgradefor use with and outside of test harness25 FTEdays -
23 Joint work/reliance on Support Databases most of the Manpower in Support Databases WBS. Continuous interface @0.2 FTE14 FTEweeks to Nov 01 -
231  Planning and procurement and installation - - -
232  Database upgrades - - -
233  Investigation of suitable 'free' or 'cheap' database for remote institution - - -
234  DBA - operational support - - -
235  Backup strategy and implementation and testing of backups - - -
236  Database Server -> C++, as necessary for Calibration (migrate back to SAM if needed) - - -
237  Use of Oracle queueing for online/offline database uploads and synchronization - - -
238  Further studies of partitioning strategies and effects - - -
239  Monitoring tools - OEM and DB instrumentation (e.g. query timings, page hits, etc.) - - -
2310  Very large database tests with populated tables - - -

Detailed Tasklists

################################################################################
# Project               : SAM
# Package               : sam_doc
#
# Most Recent Update by : lauri
#
# This work is part of a development project, called SAM, which consists of a
# number of coordinated packages each named sam_xxxx .
#
# Notice of authorship, copyright status,  and terms and conditions, should
# the software eventually become available for use outside Fermilab, can be
# found in the README and LICENCE files in the top level directory of the main
# sam package.
################################################################################