Index

Information Management

Computer Information Resources Management

Introduction/Background

Date of last assessment: October 1999

Departmental Overview

Laboratory Mission

The Stanford Linear Accelerator Center (SLAC) is dedicated to experimental and theoretical research in elementary particle physics and in those fields that make use of its synchrotron radiation facilities, including biology, chemistry, geology, material science and electrical engineering. This includes the development of new techniques in particle acceleration and detection, and of synchrotron radiation sources and associated instrumentation. The Center is operated as a national user facility for the Department of Energy by Stanford University.

Organizational Mission

The Computer Information Resource Management functional area is responsible for coordinating Information Management activities within the Laboratory. This coordination effort includes encouragement of information standards to ensure broad availability of information resources, of computer and systems procurements that have Laboratory support, and are part of Laboratory wide information planning practices.

The Computer Information Resource Management functional area self-assessment is based on, and measured against, performance objectives and standards as reflected in the SLAC contract that were defined by SLAC managers and DOE points of contact in order to address customer satisfaction, cost efficiency, and contract compliance.

Identification of Self-Assessment Report Staff

Names, titles, affiliations of participants

Bill Johnson, IRM Computing Manager, BSD

Douglas Kreitz, Assistant Director, BSD

Scope of Self-Assessment

Status of Open Items from 1999 Review

The BaBar program has been extremely productive during FY2000.  Luminosity in PEP II has increased more rapidly than expected, reaching nearly 80% of design luminosity in August.  By virtue of this luminosity, the BaBar detector has recorded more than 10 million B meson pairs, producing a flood of data that pressed the BaBar computing resources to the limit.  In order to accommodate this plethora of data,  SLAC Computing Services (SCS) and BaBar physicists have been expanding computing resources as rapidly as possible.  Three different areas deserve special mention:

• Compute Farms

The IBM RS/6000 AIX Farm has been phased out in favor of Sun Microsystems Batch Workers. The BaBar Collaboration has made the decision that SLAC will provide primary support for the Sun Microsystems components of computing for the Collaboration while other Laboratories will provide support for other architectures.   At the beginning of FY00 there were roughly 300 Sun Ultra 5 systems.  These were upgraded to Netra T1 systems and 600 new T1 systems were added for a total of 900 Batch worker machines.  These are deployed in different ways depending on BaBar’s needs.  At present 150 are dedicated to “prompt reconstruction.”  These machines are an adjunct of the On-Line Data Acquisition system and must be present before the BaBar Detector can collect data.  Another 150 machines are dedicated to BaBar reconstruction to process BaBar events that have been recorded to tape.  Another 50 machines are dedicated to the BaBar Database Group for testing purposes.  The remainder of machines (550 in number) are Batch Farm processors available to the Laboratory High Energy Physics Program and are currently heavily used by BaBar but with some usage by the End Station A Program and the GLAST Program.  These machines are actively used for Monte Carlo calculations and general data analysis.

BaBar (via SCS) is evaluating a Pilot Farm project deploying Intel processors running Red Hat Linux.  Presently 16 Dell dual processor machines are attached to this farm.  This Intel Farm appears to be a cost effective way of supplying raw computing power for the BaBar program.

• The STK Silos continue to be the major source of Mass Storage for the High Energy Physics Program.  At the beginning of FY00, there were 100 TB of available storage and 150 TB at the close of the year.  The silos remain an extremely flexible and cost effective resource for supplying the Laboratory’s needs in this area.  During the year, the IBM product HPSS (High Performance Storage System) has been introduced, both for BaBar and other HEP programs, to provide a seamless Managed Storage environment that automatically provides appropriate storage media (disk space for frequently used data, tape for more static data) for Laboratory data.
• BaBar data is collected in an Objectivity Database that can be accessed world-wide by members of the Collaboration.  This object-oriented database presently contains 140 Terabytes of data and perhaps comprises the largest “known” database in the world (“classified” databases may surpass it, but their size {or even existence} is rarely disclosed).

Intel Commodity Computer Purchases

• SLAC continues to buy the majority of its Intel machines through the DOE established BOA with Dell.  This has provided a substantial cost saving during the year and perhaps an even greater cost saving in user support costs through standardization to one manufacturer.  This has allowed SCS to develop a standard installation procedure for Windows NT systems that pervades the vast majority of Laboratory systems.
• The Microsoft site license program continues to provide both substantial savings and assures that Laboratory systems are in full conformity with license regulations with regard to Microsoft products.  The latter assurance has freed an estimated 2 FTEs of human support resources from tracking license activities and paid for the program almost entirely in salary costs.

 Virus Detection

The InocuLAN anti-virus product has completely replaced the McAfee product at SLAC and pervaded the Windows culture at the Laboratory.  In a year when computer users everywhere have been assaulted by a continual barrage of viruses and worms, this network-based virus detection package has been enormously effective in protecting the Laboratory.  InocuLAN has been extremely responsive in supplying signature updates, sometimes several times per week.  (More details about virus activities at SLAC are given in the Unclassified Computer Security Self-Assessment).

Computing and Information Management Policies

Laboratory Policy Committees, which had been in a formative state in FY99, have been fully functional throughout the year.  There are two committees responsible for addressing computing policies

• Associate Directors Committee on Computing (ADCC), which met 5 times during FY00.
• Computer Coordinating Committee (CCC), which met 6 times during FY00.  This group discusses issues of concern and submits policy recommendations to ADCC for approval or other actions.

In the discussion below, policies attributed to these committees are noted and discussed where appropriate.

New Items of Interest in 2000

Y2K Status

Like most US institutions and businesses, SLAC experienced no significant Y2K failures during or after the transition to 2000.  ADCC was actively involved in decisions preparing for the transition and in reviewing transition events.

• During preliminary analyses, very few problems were discovered that required attention, certainly none in any major application or program.
• No problems occurred during the actual transition, or on any subsequent “sensitive” dates such as February 29, 2000 or March 1, 2000.
• With the clear vision of hindsight, it would appear that the more-than-two FTEs of effort that were expended in searching for vulnerabilities were largely wasted.

Intel Systems Summary

• Windows NT growth peaked during the year at about 1500 systems and has stabilized at approximately 1400.  Use of Microsoft Systems Management Server to manage these systems suffered a rather severe setback with the system upgrade from SMS 1.2 to SMS 2.0.  Instabilities in SMS 2.0 caused most System Administrators to turn off SMS client services in late spring.  After considerable research and consultation with Microsoft, Service Updates have been applied to SMS 2.0 that restore stable conditions, and System Administrators are beginning to re-instate SMS in many areas.  Since SMS is the only available means to automatically apply security upgrades and new software to SLAC systems, the loss of management functionality has to be of concern.
• The number of Linux systems at SLAC has increased sharply during FY00, growing from an estimated 140 systems in September 1999, to more than 250 in September 2000.  Precise figures are hard to obtain since Linux and Windows NT are often installed as dual-boot possibilities on Intel systems.  Because of security concerns, this growth has led to the development/extensions of Unix management tools by SCS to include Linux under the SCS Unix support umbrella.  Principal among these is a Linux version of “tailor,” the management script that runs nightly on SCS managed Unix systems.  “Tailor” provides an even broader set of management tools than SMS does for Windows NT.

Because of the growing number of Linux security exploits, the SCS Security group has strongly recommended that Linux administrators use the standard SCS installation scripts when installing new Red Hat Linux systems.  When vulnerable systems are detected using the ISS scanning tool available to us through the Stanford University site license, system owners are contacted and guided by the SCS Security group to apply the appropriate patches.

• Use of Citrix Windows Terminal Server (WTS) Farms continues to grow at SLAC.  Five WTS clusters are now operative: 
•  a general HEP cluster;
•  a general SSRL cluster;
•  a high security Business Information Services (BIS) cluster;
• two clusters offering special software applications for the Klystron and CAD/CAM groups respectively

The secure BIS cluster requires 128 bit encrypted ICA clients and carefully restricted accounts on the WTS server.  This cluster will disappear behind a firewall that allows only ICA protocols when the Secure Business Services Project is completed.

Macintosh Support

Because of limited support resources and lack of qualified candidates for support positions, SCS support for Macintosh systems was discontinued during FY99 and users pointed to available commercial resources when support is essential.  This reduction was not made on technical grounds but purely to provide adequate support at SLAC for one PC platform (Windows NT).  Because of Macintosh popularity at SLAC, ADCC and CCC took a further step this year by requiring Associate Director approval of all new Macintosh purchases.

Windows 2000 Support

Because Active Directory Domains in Windows 2000 are such a departure from the NT 4 Domain concept, SLAC is taking precautions to limit the introduction of Windows 2000 into the community.  This has been accomplished by the following measures:

• A Windows 2000 (Win2K) Working Group has been established to propose a suitable active domain structure for SLAC.

• Windows 2000 machines must be registered to assure that Browser services, Dynamic DNS services, and Active Domain components are disabled to avoid disruption of the existing NT 4 Domains.

• Installation of registered Windows 2000 desktops has been discouraged until support issues are fully understood.  This has been successfully achieved through cooperation with the NT Administrator and Support Group at SLAC.

• Installation of Windows 2000 is allowed/encouraged on Laptops because plug-and-play features greatly simplify support for laptops.  At the same time this provides experience for supporting Win 2K in the NT 4 Domain environment.

• A test Active Domain, following Win2K Working Group design parameters, will be established and offered to adventurous Domain Administrators.

• SCS has acquired and is deploying a set of Planning, Analysis, and Management tools from FastLane that will aid in the transition to Win2K.  They are also closely following industry experience in making the transition.  We anticipate beginning the transition during calendar 2001.

Two other important policy issues have been addressed in conjunction with this transition:  1) Computer replacement cycle and 2) staff training for System Administrators.

• Because of the resource demands of Windows 2000 and SCS’s inability to support archaic operating systems, CCC recommended to ADCC that all departments adopt a 3.5 year replacement cycle for computing systems.

• In order for System Administrators to keep pace with new technologies, CCC also recommended a training budget of $3000 for each NT Administrator.

 Institutional Data:

For a number of years, the SLAC Institutional Data Committee (IDC) has been identifying data resources that need to be integrated into a single Institutional Data Structure at SLAC.  Since these data span all Divisions but have grown out of internal functional needs within each department without central planning, the data is often duplicative and, more seriously, lacks definitive ownership. Intra-divisional coordination is very strongly required in order to correct this situation. 

IDC presented its findings at a CCC meeting and subsequently reported them to ADCC with a recommendation to establish a “Data Czar” within the Directors Office to achieve this coordination.  A job description for this post has been created and a preliminary search begun to fill the position.

Performance Side-Bar Indicators

 STK Silos

The Silos remain an extremely effective mechanism for providing mass storage for the Laboratory, even after more than a decade of evolution.  The history depicted in the two charts below amply show the versatility and cost effectiveness of this medium.  During the past year the cost per Terabyte of storage space has dropped by 30% while the volume of storage in use has increased by 50%.  A further sign of system flexibility is the ease with which this mass storage media has been integrated in the managed storage system HPSS, giving the Laboratory a seamless repository for all its data.

 World Wide Web Usage

The SLAC Web also has grown in sophistication and functionality through the use of virtual Web technology.  New Web analysis tools have been purchased and deployed during FY00.  The charts below were generated using these tools.

The first two charts show the majority presence of educational institutions in the SLAC Web traffic patterns.  The .COM and .NET traffic for the most part are the inevitable Web search engines that perpetually index the data at all Internet Web sites.  In the “Hits by Country” plot we see evidence of our international collaborators and a surprisingly large (and presently inexplicable) presence of hits from the Netherlands.

The third chart shows weekly distribution of hits, indicating that approximately half the traffic is from non-U.S. Domains above a fairly constant pedestal of .EDU and .COM traffic.

The final chart shows the client Operating System composition of SLAC’s Web traffic.  Not surprisingly this has a dominant Windows NT component since much of the traffic is internal to SLAC where Windows NT is the dominant system.  Linux also has a very strong presence, reflecting the trend toward Linux in the High Energy Physics community worldwide.  The Macintosh component, once strong at SLAC and HEP in general, has fallen to 4%.  This may reflect SLAC’s (reluctant) reduced support policies for the platform.

Compute Farms

As in FY99, SLAC Compute Farms have undergone rapid, continuing growth and change.  Indicators to track this change numerically cannot be meaningfully defined at this time because of the pace of change.  From the point of view of satisfying the needs of the scientific community, however, great progress has been made to supply an immense and flexible computing resource.  At the beginning of the year there were 300 Sun Ultra 5 systems in the Farm operating at 333MHz; at the close of FY00 there are 900 Sun Textra 1 systems operating at 440MHz with a resulting 400% increase in computing capacity.   In less than a decade, there has been an increase of more than two orders of magnitude in available computing power for physics at SLAC.  Furthermore, the present farm is dramatically more flexible in its ability to dedicate portions of the farm to a particular activity.  In the case of BaBar, the principal user of the Farm resource, 150 processors are dedicated to prompt reconstruction during the BaBar data taking cycle in order to provide monitoring statistics for the detector while data is being accumulated.

In addition to the Sun farm, two Intel Compute Farms are in operation:  one for Accelerator Research (ERDA) consisting of 16 Dell processors running Linux; the other is a BaBar pilot project, recently upgraded to 34 Dell processors running Linux.  These Intel Farms show great promise for providing highly cost effective mass computing resources. 

Status of Goals during FY00

1. Coherent strategy for Windows 2000 (discussion of “achievements)

This goal was fully accomplished, as indicated above through the establishment of

• The Windows 2K Working Group
• Registration and regulation of Win2K systems
• Deployment of FastLane Reporting, Management, and Analysis Tools
• Use of Windows 2000 for Laptop installations

2. Development of DOE Performance Measures based on SCS indicators—no progress

No significant progress has been made in this area.  With the Peer Review process imminent in the future, it was impractical to establish SCS based measures until the requirements for Peer Review are fully known.

Improvement Action Plan/Goals

Goals for FY 2001

1. Migrate Windows Desktops to Windows 2000
2. Develop Performance Measures based on the tools available to the Laboratory to measure performance.