SLC Functional Description Document for History Buffering

  1. Title/Name
  2. This subsystem is referred to as History Buffering by the SLC Control System. To the outside world, it is referred to as Data Archiving.

  3. Purpose
  4. A. What is the application for and why is it necessary?

    History Buffering retrieves and stores data from the SLC database and the EPICs IOC databases at a constant rate specified by users. This facility is used to look at the history of device settings or beam parameters in order to troubleshoot problems or monitor the machine.  Any piece of floating point data put into the SLC or EPICs database can be history buffered; conversely, any piece of data not in the SLC or EPICs database cannot be history buffered.

    As part of the History Buffering facility, data can be retrieved and viewed.   Retrieved data can be any single device parameter specified by a starting and ending timestamp. Viewing is performed through the SLC Control System.  The retrieved data can also be correlated, fit, averaged, etc.

    B. Who uses it?

    Most people associated with the Control System will have a need to look at history data at some point. Operators in the control room and machine physicists probably have the most frequent need to look at buffered data. They need to have access to the history of each device and the state of the machine constantly.  Maintanence people may need to monitor individual devices to troubleshoot hardware problems, and software people may need to look at beam parameters or device histories to troubleshoot software problems.

  5. Operating Environment

    Since the data buffered is located in the SLC database, this facility must run on the production mainframe.  A test version may run on the development mainframe but is used mainly to test software changes to the facility. It has off-line and on-line components which run on the mainframe.

  1. Functional Requirements

    The initial specification of database items to be history buffered is an off-line activity.  Each database item to be history buffered is specified in a data file along with the buffering rate.  These files are "compiled" off-line which means they are checked for errors, the database locations are resolved, etc. 

    The history buffering process which actually retrieves the data and stores it in a data file is performed by standalone processes running on the host machine.  There are two processes which reads all the files and stores the data.  One process for the SLC data and the other process for EPICs data.  When a parameter is to be added or deleted, the new file which specifies the devices, must be reinstalled.

    The viewing part of the history data is integrated into the SCP.  This part of the process has access to the file system and can retrieve the data specified by users.  It has the built-in capability to correlate data with other data or with time, filter data, or perform simple calculations on the data.

    There is another off-line batch process which runs once a day (usually at night).  This process adds the daily data to the weekly data files, reorganizes the data by devices (rather then by time) and compresses the data so the amount of data kept long-term is manageable and much less than the data initially taken.

    A description of the steps performed in the current history buffering process can be found in document Device History Facility.

  1. Performance Requirements

    Since the rate that data is buffered is very slow (on the order of minutes), there aren't real timing constraints.  If the rate were drastically increased (on the order of seconds or subsecond rates), performance would become an issue.

    The performance of data viewing can become an issue with users.  Since users can ask to view data from previous months, the algorithm to find the location of the data is very important and the ability to display the data quickly is very important. If the data takes more than a couple of seconds to find and display, users become impatient.

  1. Resource Dependencies

    DATA

    Any data placed in the SLC Database or EPICs IOCs database can be history buffered.   The units, accuracy and precision are subsystem dependent and not a concern for history buffering.

    CPU

    Since history buffering runs at such a slow rate, the CPU intensity will be bursty depending on the number of devices being buffered and the rate. CPU usage will be intensive when the once a day filtering is performed (usually late at night). All of the current history buffer processing is performed on the Alphas.

    Access to Accelerator Hardware

    The access to hardware is indirect since history buffering pulls data from the database.  The quantity of data is based on user specification.

    Access to Processed Data

    The source for history buffers is the SLC database and the EPICs IOCs. This facility does generate data files which could be a data source for other applications.

  1. Hindsight & Foresight [optional]

    The current history buffer system is not very flexible.  It is not meant for fast data taking, it does not easily handle additions or deletions of signals, and  the data can only be viewed from the host machine.

    The requirements for the NLC will be much more complex and involved.  The amount of data archived will be orders of magnitude more than the current system so the flexibility needed to specify the devices and data rates must be handled easily and seemlessly. Also, data viewing for the NLC will need to be performed via the Web.   Since the requesters of archive data may be located at other sites,  it is important to have access to machine data from anywhere in the world.

    A first cut at the requirements for NLC Archiving can be seen from the presentation, NLC Archiving Requirements.