| [ Intro | Pubs | WebTables | Workbook | Software | SLACVX | Search ] | |
| [ WebTables : Help | Maintenance ] |
This manual is currently in the midst of a rewrite. Parts of it still reflect the 1996 SLD Plot and Table System rather than the new WebTables System.
A block diagram is provided for quick reference.
In addition to this manual, you may want to study the user's help manual, Help on the WebTables System.
The major components of the WebTables System are as follows:
* indicates files for which there is one copy per interface set
(i.e., one copy for the SLD Main Tables, one copy for the SLD Cater Tables, etc.)
Most maintenance just invovles changing DATANAMES.DAT
and then rerunning WTFORMWRITE.COM.
This file contains information describing the datanames used in the WebTables interfaces. One copy of this file exists for each interface set.
The file contains:
Edit this file to add new datanames to the interface pages, to remove datanames from the interface pages or to change other information associated with a given dataname. Then, to implement the changes, call WTFORMWRITE.COM, the Interface Form Writing Exec, as described below.
Pay careful attention to spacing when editing this file. The column position of the colons must not be changed.
Keep in mind that the database only contains date, no time, for some datanames.
The glossary files are more important than you might think since some crucial information needed by STANDINT.COM, the Interface Form Reading Exec and INTERMED.COM, the Intermediate Exec, is hidden in comment lines at the start of each glossary file.
The two previous copies of all output files are kept in case you make such a horrible mess that you just want to delete your new versions.
WTFORMWRITE.COM should be run any time DATANAMES.DAT is changed.
The exec should be run by hand from the directory
DISK$SLD_FAC0:[SLDWWW.WEBTABLES]and must be run from an account that has write access to that directory.
Rebuilding the Run Information Interface pages is fairly slow since there are dozens of glossary files to be written. Be patient.
The GIFs used as submit buttons were created by viewing the HTML file DISK$SLD_FAC0:[SLDWWW.WEBTABLES]SUBMIT_BUTTON_LAYOUT.HTML, capturing the relevant part of the table (using xv or any similar product) and then re-coloring the image as needed. This gives the submit buttons roughly the same look as table headers. There should rarely be any need to repeat this process.
To create a new interface set:
Performs the layout of the range options that appear at the top of the SIMPLE INTERFACE and the Resubmission Forms.
Performs the layout of the qualifiers radio buttons that appear at the top of the SIMPLE INTERFACE and the Resubmission Forms.
The glossary files are more important than you might think. Hidden in their comment lines is some crucial information needed by STANDINT.COM, the Interface Form Reading Exec and INTERMED.COM, the Intermediate Exec.
Do not edit these files by hand. Use WTFORMWRITE.COM.
The Interface Form Reading Exec starts by calling an executable that loads the submission values into a set of symbols prefixed by "form_fld". This executable is
DISK$SLD_USR0:[SLDWWW.HTTPD.HTBIN]SLD_SYMBOLS.EXEThe executable is compiled from a slightly modified version of CGI_SYMBOLS.C, a routine that comes with the OSU VMSthreads Server. The SLD modification is necessary because the OSU version does not correctly handle symbols for field names that contain a period. This comes up in our forms for the symbols that contain the x and y image map positions, form_fld_point.x and form_fld_point.y.
The problem is that DCL symbol names cannot include a period. The OSU code fails to handle this case, making no symbols for these critical fields. The SLD version works around this by converting any periods to underscores in symbol names. We then end up with the symbols form_fld_point_x and form_fld_point_y.
If a problem is found with the submit fields, a page is sent back containing an error message. Otherwise, a new URL is formed which uniquely identifies what kind of plot or table is to be made. STANDINT.COM redirects the browser to this new URL which is then handled by INTERMED.COM, the Intermediate Exec. This URL is described in detail in the document: URL for the Intermediate Exec.
You can get the Interface Form Reading Exec to show you some diagnostics by inserting the secret debug string, "db" (for DeBug) into the "Last" part of the submit form. This works whether or not you have actually selected "Last" as the range_type.
For more information on debugging the Interface Form Reading Exec, see To Debug a COM File Running on the WWW-SLD Web Server.
The Intermediate Exec is the intermediary that handles all of the mechanics of plotting and tabulating.
Since several users may be using the Run Information System almost simultaneously, unique filenames must be constructed for each user. Filenames are made unique by including a string that is composed of the job start time in hundredths of a second. The resulting integer is the * in the above filenames.
The Intermediate Exec begins by unpacking the data from the URL and creating some filenames for temporary files and output files.
The next step is to interpret the range options. If the form used was "Last n Days," it is converted to a begin and end date based on the current date from f$time. If the form used was "Last n Runs," the exact run numbers come later from Oracle.
The datanames part of the URL can contain one or more datanames from any of three classes.
The datanames part of the URL is used to create a number of strings for input to the Oracle Interface Exec, for page, plot and column headers, for input to the Column Math Engine and for prefixes and suffixes used in link datanames.
For each dataname, the system searches for an appropriate glossary file. If such a file is found, the comment lines at the top of the file yield information on title, submit options, precision, error type and default y scale limits. If no such glossary file is found, we still allow the dataname but we have to use the dataname name instead of a proper title string, and we have to rely on defaults for submit options, precision, etc.
It is an important feature of the Run Information System that it does not insist that the dataname be in DATANAMES.DAT and therefore have a glossary file. Any dataname in one of the supported Oracle tables will be allowed. This means that one never needs to rush to update DATANAMES.DAT just because someone wants to see a new piece of information.
The actual Oracle query is not done by the Intermediate Exec. It is done by a query engine that lives on Unix, WTQUERY.REX, the Oracle Interface Exec.
The Intermediate Exec packs all of the query input into a URL described in the document:URL for the Oracle Interface Exec. It then calls this URL using the "vmsproxy" system developed by Tony Johnson.
You can get the Intermediate Exec to show you the exact URL by which it called the Oracle Interface Exec by inserting the secret debug string, "do" (for Debug Oracle url) into the "Last" part of the submit form. This works whether or not you have actually selected "Last" as the range_type.
The data returned by the Oracle Interface Exec is trapped in an output file of the form OO*.TMP (for Oracle Output) by a temporary reassignment of SYS$OUTPUT.
The Intermediate Exec then does some of the work involved in creating the resubmission form. The form is initialized with the same values as the original submission. One of several resubmission GIFs are used depending on what resubmission options are valid. For example, if more than one dataname was selected, "Plot by Day" and "Plot by Run" cannot be done (we only allow plots of a single quantity).
The output from the Oracle Interface Exec, OO*.TMP, is then read and merged with the non-Oracle columns of data (constant datanames and link datanames).
In the case of Plots by Day, the first column must be converted from Oracle date format to Topdraw date format. This format is integer month followed by decimal point followed by integer day. For example, 31-JAN becomes 1.31.
A column of data is merged into the Oracle output if there was a constant dataname. What INTERMED.COM considers a constant dataname is any dataname for which the dataname name starts with a digit or a decimal point. The new column contains the same constant in every row. The constant is the dataname name itself.
A column of data is merged into the Oracle output if there was a link dataname. What INTERMED.COM considers a link dataname is any dataname for which the dataname name starts with "LINK_". The new column contains file specifications. Each file specfication is constructed by combining information from the dataname name with the run or date strings from the first column.
The new set of columns, which can now contains all three classes of data (Oracle information, Constants and Links), ends up in a file of the form CI*.TMP (for Column math Input).
Some clarification is required here to understand the difference between the links that appear in the Standard Interface tables and the links that appear in the DUCS Test tables.
The next step in forming the plots or tables is to handle any math that needs to be performed on the columns of data. This occurs when the user has requested ratios or when errors must be calculated. Since DCL cannot perform math on real numbers, this math processing is done in a C program, the Column Math Engine.
The C program reads the columns of data from CI*.TMP, performs the necessary calculations, and outputs the data to another temporary file, CO*.TMP (for Column math Output).
Columns of data for which no operation needs to be performed are handled as strings, written back out just as then came in. This allows a simplified program flow for INTERMED.COM. All of the data it needs is present in the output of the Column Math Engine.
If the user has requested a plot, the output from the Column Math Engine is converted to a file suitable for input to Topdraw, TI*.TD (for Topdraw Input).
If there are any gaps in the list of runs in a Plot by Run, extra work must be done to bring the histogram down to YMIN for missing runs. Otherwise, Topdraw draws a misleading stair-step in the missing run range. It uses the previous run's value for the first half of the gap and the next run's value for the second half of the gap. It would be good to include a similar fix for gaps in the list of dates in a Plot by Date, but date manipulation is too ugly to handle.
A special version of Topdraw is called to do the actual plotting. Its output is a file of the form TO*.GIF (for Topdraw Output). This file is displayed as an inline image.
If the user has requested a table or sum, the output from the Column Math Engine is converted to a table. Whenever possible, the table headers are links to glossary files.
If the data is a file specification, the table should contain either a link to that file or a statement that the file is missing. These file specifications may be ones we constructed for link datanames (generally the case for run information) or they have come out directly out of Oracle (the case for daily DUCS test information).
In either case, what tells INTERMED.COM that the table element should be presented as a file specification is that it contains one and only one colon. INTERMED.COM then looks to see if the file exists. Currently, it just does an f$search. Eventually it needs to also be aware of file shelving.
A first step in debugging the Intermediate Exec is to study the various temporary and output files mentioned above. All of these files are preserved for at least the last five minutes (subsequent queries trigger a delete/before=-:05).
For more information on debugging the Intermediate Interface Exec, see To Debug a COM File Running on the WWW-SLD Web Server.
The Oracle Interface Exec handles the actual Oracle query. It is called from the middle of INTERMED.COM, the Intermediate Exec, via the "vmsproxy" system developed by Tony Johnson.
The Oracle Interface Exec forms the appropriate SQL code based on the information provided in the URL for the Oracle Interface Exec plus some other lists of rules.
The tables SLDATA.DATLOG, SLDPM.PASS1RUN, SLDPM.PASS2RUN and SLDPM.OFFLDAY are accessed by joins to the key table SLDATA.RUNLOG.
If the range if specified in the form "LAST n RUNS", all runs are selected with results requested in descending order. Output is truncated after n rows. The order of these n rows is then inverted before returning them.
INTERMED.COM, the Intermediate Exec, traps the output by a temporary redefinition of SYS$OUTPUT into a file of the form OO*.TMP (for Oracle Output).
You can get the Intermediate Exec to show you the exact URL by which it called the Oracle Interface Exec by inserting the secret debug string, "do" (for Debug Oracle url) into the "Last" part of the submit form. This works whether or not you have actually selected "Last" as the range_type.
To debug the Oracle Interface Exec, set Trace to R at the top of the exec and rerun the query. The query will no longer display correctly through the Web interface, but the full trace will be available in the SYS$OUTPUT file DISK$SLD_SCR0:[SLDWWW]OO*.TMP
Be sure to turn Trace back to O when you have finished debugging, or else the system will remain disabled.
This is the Column Math Engine called by INTERMED.COM, the Intermediate Exec. It is currently able to handle division and Poisson errors. It could eventually be expanded to handle other math functions.
Characters in flagstr have the following function:
For example, the flag string "o*od3o2p" means
Columns of data for which no operation needs to be performed are handled as strings, written back out just as then came in. This allows a simplified program flow for INTERMED.COM. All of the data it needs is present in the output of the Column Math Engine.
Modifications to COLMATH.C can be tested by recompiling and relinking COLMATH.C and then calling it from COLTEST.COM (located in the same directory).
This is the version of Topdraw called by INTERMED.COM, the Intermediate Exec. It was made by taking the version of Topdraw in the SLAC$TD directory on the SLD cluster and then making the following changes:
Put SET VERIFY into the COM file.
The server will run on SLDB4 with a process name prefixed by:
NET_
To see what processes are running under the server:
show sys/net/node=sldb4Issue this command as the server is working. One of these processes will show activity.
Immediately after the server is done, kill the process with the command:
STOP/ID=If you don't kill the process, it will be kept for reuse by other web work. While it is the wait state it will be renamed with the prefix:
SERVER_It is OK to kill any or all of these processes. The server automatically restarts them as needed.
Immediately after killing the process,
EVE DISK$SLD_USR0:[SLDWWW]NETSERVER.LOGThe latest of these log files will be the one from the server just killed. If you killed the right process, your debug output will be there.
Be sure to turn off the SET VERIFY when you are done. Too much debug output can cause the servers to run out of disk space.