Chapter 9. GRS-1 Record Model and Filter Modules

Although input data can take any form, it is sometimes useful to describe the record processing capabilities of the system in terms of a single, canonical input format that gives access to the full spectrum of structure and flexibility in the system. In Zebra, this canonical format is an "SGML-like" syntax.

To use the canonical format specify grs.sgml as the record type.

Consider a record describing an information resource (such a record is sometimes known as a locator record). It might contain a field describing the distributor of the information resource, which might in turn be partitioned into various fields providing details about the distributor, like this:

      <Distributor>
      <Name> USGS/WRD </Name>
      <Organization> USGS/WRD </Organization>
      <Street-Address>
      U.S. GEOLOGICAL SURVEY, 505 MARQUETTE, NW
      </Street-Address>
      <City> ALBUQUERQUE </City>
      <State> NM </State>
      <Zip-Code> 87102 </Zip-Code>
      <Country> USA </Country>
      <Telephone> (505) 766-5560 </Telephone>
      </Distributor>
     

The keywords surrounded by <...> are tags, while the sections of text in between are the data elements. A data element is characterized by its location in the tree that is made up by the nested elements. Each element is terminated by a closing tag - beginning with </, and containing the same symbolic tag-name as the corresponding opening tag. The general closing tag - </> - terminates the element started by the last opening tag. The structuring of elements is significant. The element Telephone, for instance, may be indexed and presented to the client differently, depending on whether it appears inside the Distributor element, or some other, structured data element such a Supplier element.

       <gils>
       <title>Zen and the Art of Motorcycle Maintenance</title>
       </gils>
      

       <title>
       <var lang lang "eng">
       Zen and the Art of Motorcycle Maintenance</>
       <var lang lang "dan">
       Zen og Kunsten at Vedligeholde en Motorcykel</>
       </title>
      

       <title>
       <var lang lang "eng"><var body iana "text/plain">
       Zen and the Art of Motorcycle Maintenance
       </title>
      

       <title>
       <var body iana "text/plain>
       <var lang lang "eng">
       Zen and the Art of Motorcycle Maintenance
       <var lang lang "dan">
       Zen og Kunsten at Vedligeholde en Motorcykel
       </title>
      

In order to handle general input formats, Zebra allows the operator to define filters which read individual records in their native format and produce an internal representation that the system can work with.

Input filters are ASCII files, generally with the suffix .flt. The system looks for the files in the directories given in the profilePath setting in the zebra.cfg files. The record type for the filter is grs.regx.filter-filename (fundamental type grs, file read type regx, argument filter-filename).

Generally, an input filter consists of a sequence of rules, where each rule consists of a sequence of expressions, followed by an action. The expressions are evaluated against the contents of the input record, and the actions normally contribute to the generation of an internal representation of the record.

An expression can be either of the following:

An action is surrounded by curly braces ({...}), and consists of a sequence of statements. Statements may be separated by newlines or semicolons (;). Within actions, the strings that matched the expressions immediately preceding the action can be referred to as $0, $1, $2, etc.

The available statements are:

The following input filter reads a Usenet news file, producing a record in the WAIS schema. Note that the body of a news posting is separated from the list of headers by a blank line (or rather a sequence of two newline characters.

      BEGIN                { begin record wais }

      /^From:/ BODY /$/    { data -element name $1 }
      /^Subject:/ BODY /$/ { data -element title $1 }
      /^Date:/ BODY /$/    { data -element lastModified $1 }
      /\n\n/ BODY END      {
      begin element bodyOfDisplay
      begin variant body iana "text/plain"
      data -text $1
      end record
      }
     

If Zebra is compiled with support for Tcl enabled, the statements described above are supplemented with a complete scripting environment, including control structures (conditional expressions and loop constructs), and powerful string manipulation mechanisms for modifying the elements of a record.