|
|
by William Robert Stanek
Using CGI scripts, you can create powerful, personalized, and professional Web publications that readers can interact with. CGI scripts are external programs that act as gateways between the Web server and other applications. You can use CGI scripts to process input from readers and thus open a two-way communication channel with your readers. Reader input can be data from fill-out forms, keywords for a database query, or values that describe the reader's browser and connection.
Your CGI scripts can use this input to add entries to an index, to search databases, to create customized documents on the fly, and much more. Yet the most wonderful thing about CGI scripts is that they hide their complexities from users. If you've used a fill-out form or an image map on the Web, you've probably used a gateway script without even knowing it. With CGI scripts, everything seems to happen automatically. You enter data and click a mouse button, and a moment later, a result is displayed. Learning what happens between the click of the mouse button and the display of the result is what this chapter is all about. This chapter explains what you need to know about CGI scripts: what they are, how to use them, and why to use them.
CGI scripts are external programs that run on the Web server. You can use CGI scripts to create highly interactive Web publications. The standard that defines how external programs are used on Web servers and how they interact with other applications is the common gateway interface. The three keywords that comprise the name of the standard-common, gateway, and interface-describe how the standard works:
Common. By specifying a common way for scripts to be accessed, CGI enables anyone, regardless of platform, to pass information to a CGI script.
Gateway. By defining the link (gateway) between the script, the server, and other applications, CGI makes it possible for external programs to accept generalized input and pass information to other applications.
Interface. By describing the way external programs can be accessed by users (the interface), CGI reduces the complex process of interfacing with external programs to a few basic procedures.
The developers of CGI worked these key concepts into the CGI standard to create a powerful and extendable advanced feature for Web publishers that shields readers of your publications from its complexities. Readers need only click an area of an image map or submit their fill-out forms after completing them. Everything after the click of the mouse button seems to happen automatically, and the readers don't have to worry about the how or why. Web publishers, however, must understand how CGI scripts work, especially if they want to take advantage of the ways CGI can be used to create powerful Web publications.
Although readers see only the results of their submissions or queries, behind the scenes many things are happening. The process consists of the following steps:
On UNIX systems, CGI scripts are located in a directory called cgi-bin in the usr file system, and CGI utilities are located in a directory called cgi-src in the usr file system. On other systems, your Web server documentation explains where CGI scripts and utilities should be placed.
CGI scripts are also called gateway scripts. The term script comes from the UNIX environment, in which shell scripts abound, but gateway scripts don't have to be in the format of a UNIX script. You can write gateway scripts in almost any computer language that produces an executable file. The most common languages for scripts are the following:
Bourne Shell
C Shell
C/C++
Perl
Python
Tcl
Visual Basic
When you use the traditional scripting languages, your scripts are located on the server and are executed when a user submits a fill-out form. Scripts located on a server are called server-side scripts. In contrast, some scripts, called client-side scripts, are embedded in HTML documents and are executed by your browser. JavaScript and VBScript are two up-and-coming client-side scripting languages.
The best programming language to write your script in is one that works with your Web server and meets your needs. Preferably, the language should already be available on the Web server, and you should be proficient in it (or at least have some knowledge of the language). Keep in mind that most user input is in the form of text that must be manipulated in some way, which makes support for text strings and their manipulation critical.
The easiest way to determine whether a language is available on a Web server is to ask the Webmaster or system administrator responsible for the server. Because most Web servers operate on UNIX systems, you may also be able to use the which or whereis UNIX commands to check on the availability of a particular language. To use commands on UNIX systems, type which or whereis at the shell prompt and follow the command with a keyword on which you want to search, such as the name of the programming language you want to use. For example, to see whether your UNIX server supports Perl, type either one of the following lines:
which perl whereis perl
As Perl, C/C++, and UNIX shell are the most popular languages for scripts, the following sections look briefly at these languages, with emphasis on why and when to use them. Each section contains a checklist for features and systems supported that contains the following categories:
Operating system support lists the operating systems on which the language can be used.
Programming level specifies the level of difficulty involved in using and learning the language.
Complexity of processing indicates the complexity of the tasks you can process with the language.
Text-handling capabilities are the capability of the language to manipulate text and strings.
The sections on common scripting languages are followed by discussions of the newest scripting languages: JavaScript and VBScript. Both JavaScript and VBScript are hot topics on the Net right now. If you want to be on the cutting edge of Internet technologies, keep both eyes on these languages.
Operating system support: UNIX
Programming level: Basic
Complexity of processing: Basic
Text-handling capabilities: Moderately advanced
The UNIX operating system is in wide use in business, education, and research sectors. There are almost as many variations of the UNIX operating system as there are platforms that use it. Even platforms produced by the same manufacturer use different variants of the UNIX operating system. For example, DEC has variants of UNIX for the Dec-Alpha, Decstation, and Dec OSF platforms.
What these operating systems have in common is the core environment on which they are based. Most UNIX operating systems are based on Berkeley UNIX (BSD), AT&T System V, or a combination of BSD and System V. Both BSD and System V support three shell scripting languages:
Bourne shell
C shell
Korn shell
| Tip |
You can quickly identify the shell scripting language that is being used by examining the first line of a script. Bourne shell scripts generally have this first line: #!/bin/sh |
All UNIX shells are interpreted languages, which means the scripts you create do not have to be compiled. Bourne shell is the most basic shell. C shell is an advanced shell with many features of the C programming language. Because Bourne shell uses a completely different syntax than C shell, scripts written in Bourne are not compatible with scripts written in C. If you create a script in Bourne shell and later want to use C shell to interpret the script, you must rewrite the script for C shell.
Many programmers often want to merge the simplicity of Bourne shell with the advanced features of C shell, and this is where Korn shell comes in handy. Korn shell has the same functionality as the Bourne shell and incorporates many features of the C shell. Any shells you've written in Bourne shell can be interpreted directly by the Korn interpreter. This capability prevents you from having to rewrite a Bourne shell script when you later find you want to use a feature supported by Korn. Although the Korn shell is gaining popularity, Bourne and C shell are the two most widely used UNIX shells.
You can change your current shell any time from the shell prompt by typing one of the following:
/bin/sh to change to Bourne shell
/bin/csh to change to C shell
/bin/ksh to change to Korn shell
Usually, you will see differences between the various shells immediately. For example, the default command prompt for Bourne shell is the dollar sign, and the default command prompt for C shell is usually your host name and user ID followed by a colon. Beyond this difference, C shell supports a history function, aliasing of commands, and many other controls that the Bourne shell does not. However, these differences are generally not important to the CGI programmer. Your primary concern should be the features that the shell directly supports and how scripts behave when executed in it.
Bourne shell is the smallest of the shells and the most efficient. Consequently, a Bourne shell script generally executes faster and uses fewer system resources. When you want more advanced features, such as arrays, use Korn shell. Korn shell has more overhead than Bourne shell and requires slightly more system resources. When you want to make advanced function calls or assignments, use C shell. Because C shell is larger than Bourne and Korn shell, scripts written in C shell generally have higher overhead and use more system resources.
Although UNIX shells have good built-in facilities for handing text, such as sed, awk, and grep, they are not as powerful or extendable as traditional programming languages. Consider using shell scripts when you want to perform simple tasks and moderately advanced text or file manipulation.
Operating system support: UNIX, DOS, Windows, Mac, and others
Programming level: Advanced
Complexity of processing: Advanced
Text-handling capabilities: Difficult to use
When you want your scripts to perform complex tasks, you call in the big guns. Two of the most advanced languages used in CGI scripts are C and C++. C is the most popular programming language in use today. C++ is the object-oriented successor to C. Both C and C++ are advanced programming languages that require you to compile your scripts before you can use them. The major advantages of C and C++ is that they enjoy widespread use and versions are available for virtually every operating system you can think of.
Use C (rather than C++) when your scripts must execute swiftly and use minimal system resources. Compiled C programs are very small-tiny compared to programs with similar functionality programmed in other languages. Small programs use minimal system resources and execute quickly. However, C is a very complex language with difficult-to-use facilities for manipulating text. Therefore, if you are not proficient in C, be wary of using C to perform advanced text string processing.
Use C++ when certain functions of your scripts will be reused and when long-term development costs are a major concern. C++ is an object-oriented language that enables you to use libraries of functions. These functions form the core of your CGI scripts and can be reused in other CGI scripts. For example, you can use one function to sort the user's input, another function to search a database using the input, and another function to display the output as an HTML document. Keep in mind that C++ is an object-oriented language that is very different from other languages. If you have not used an object-oriented language before, are not familiar with C, and plan to use C++ for your CGI scripts, be prepared for a steep learning curve.
Operating system support: UNIX, DOS, Windows, Mac, and others
Programming level: Advanced
Complexity of processing: Advanced
Text-handling capabilities: Easy to use
If you want to be on the inside track of CGI programming, learn and use the Practical Extraction and Report Language (Perl). Perl combines elements of C with UNIX shell features like awk, sed, and grep to create a powerful language for processing text strings and generating reports. Because most of the processing done by CGI scripts involves text manipulation, Perl is rapidly becoming the most widely used language for CGI scripts. As with C and C++, a major advantage of Perl is its widespread use. Versions of Perl are available for virtually every operating system you can think of. You can use Perl to do the following tasks:
Perl, like Bourne and C shell, is an interpreted language. However, Perl does not have the limitations of most interpreted languages. You can use Perl to manipulate extremely large amounts of data, and you can quickly scan files using sophisticated pattern-matching techniques. Perl strings are not limited in size. The entire contents of a file can be used as a single string. Perl's syntax is similar to C's. Many basic Perl constructs, like if, for, and while statements, are used just as you use them in C.
| Tip |
Like a UNIX shell script, a Perl script will usually specify the path to the source routines in the first line. Therefore, the first line of a Perl script should specify the path to where Perl is installed on the system. This path is usually one of the following: #!/usr/local/perl |
Perl is surprisingly easy to learn and use, especially if you
know the basics of C or UNIX shell. Perl scripts are usually faster
than UNIX shell scripts and are slightly slower than compiled
C/C++ scripts. You should use Perl whenever you have large amounts
of text to manipulate.
JavaScript is a client-side scripting language based on the Java programming language developed by Sun Microsystems. This powerful up-and-coming scripting language is being developed by Netscape Communications Corporation, and, as you may have guessed, Netscape Navigator 2.0/3.0 fully supports JavaScript.
Netscape Navigator 2.0/3.0 interprets JavaScript programs embedded directly in an HTML page, and, just like Java applets, these programs are fully interactive. JavaScript can recognize and respond to mouse clicks, form input, and page navigation. This capability means your pages can "intelligently" react to user input. The JavaScript language resembles the Java programming language-with a few important exceptions, as you can see from the comparisons in the following lists:
JavaScript is the following:
Java is the following:
JavaScript is designed to complement the Java language and has some terrific features for Web publishers. For example, by creating a JavaScript program that passes parameters to a Java applet, you could use the JavaScript program as an easy-to-use front-end for your Java applets. Because a Web publisher is not required to know about classes to use JavaScript and to pass parameters to a Java applet, JavaScript provides a simple solution for publishers who want to use the features of the Java language but don't want to learn how to program in Java. The JavaScript scripting language is featured in the chapters in Part VIII, "JavaScript and Java."
With VBScript, Microsoft proves once again that it understands the tools developers need. Visual Basic Script is a subset of Visual Basic and is used to create highly interactive documents on the Web. Similar to JavaScript, programs written in VBScript are embedded in the body of your HTML documents.
Visual Basic Script also enables dynamic use of OLE scripting management with ActiveX Controls. The Object Linking and Embedding of scripts enables Web publishers to dynamically embed VBScript run-time environments. Basically, this capability enables you to use VBScripts as plug-in modules. You can, for example, embed a VBScript program in your Web document that calls other VBScript programs to use as plug-ins. The exact plug-in calls could be dynamically selected based on user input. The VBScript scripting language is featured in the chapters of Part VII, "ActiveX and VBScript."
At this point, you may be worried about having to program. You may also be wondering why you would want to use gateway scripts at all. These are valid concerns. Learning a programming language isn't easy, but as you will see later, you may never have to program at all. Dozens of ready-to-use CGI scripts are freely available on the Web. Often you can use these existing programs to meet your needs.
The primary reason to use CGI scripts is to automate what would otherwise be a manual and probably time-consuming process. Using CGI scripts benefits both you and your reader. The reader gets simplicity, automated responses to input, easy ways to make submissions, and fast ways to conduct searches. Gateway scripts enable you to automatically process orders, queries, and much more. CGI programs are commonly used to do the following tasks:
Gateway scripts are used to process input submitted by readers of your Web publications. The input usually consists of environment variables that the Web server passes to the gateway script. Environment variables describe the information being passed, such as the version of CGI used on the server, the type of data, the size of the data, and other important information. Gateway scripts can also receive command-line arguments and standard input. To execute a CGI script, the script must exist on the server you are referencing. You must also have a server that is both capable of executing gateway scripts and configured to handle the type of script you plan to use.
Readers pass information to a CGI script by activating a link containing a reference to the script. The gateway script processes the input and formats the results as output that the Web server can use. The Web server takes the results and passes them back to the reader's browser. The browser displays the output for the reader.
The output from a gateway script begins with a header containing a directive to the server. Currently, the three valid server directives are Content-type, Location, and Status. The header can consist of a directive in the format of an HTTP header followed by a blank line. The blank line separates the header from the data you are passing back to the browser. Output containing Location and Status directives usually consists of a single line because the directive contained on the Location or Status line is all that the server needs, and when there is no subsequent data, you do not need to insert a blank line. The server interprets the output, sets environment variables, and passes the output to the client.
Any transaction between a client and server has many parts. These parts can be broken down into nine steps as follows:
When a user activates a link to a gateway script, input is sent to the server. The server formats this data into environment variables and checks to see whether additional data was submitted via the standard input stream.
Input to CGI scripts is usually in the form of environment variables. The environment variables passed to gateway scripts are associated with the browser requesting information from the server, the server processing the request, and the data passed in the request. Environment variables are case-sensitive and are normally used as described in this section. Although some environment variables are system-specific, many environment variables are standard. The standard variables are shown in Table 26.1.
As later examples show, environment variables are set automatically
whenever reader input is passed to a server. The primary reason
to learn about these variables is to better understand how input
is passed to CGI scripts, but you should also learn about these
variables so you know how to take advantage of them when necessary.
| Variable | Purpose |
| AUTH_TYPE | Specifies the authentication method and is used to validate a user's access. |
| CONTENT_LENGTH | Used to provide a way of tracking the length of the data string as a numeric value. |
| CONTENT_TYPE | Indicates the MIME type of data. |
| GATEWAY_INTERFACE | Indicates which version of the CGI standard the server is using. |
| HTTP_AccEPT | Indicates the MIME content types the browser will accept, as passed to the gateway script via the server. |
| HTTP_USER_AGENT | Indicates the type of browser used to send the request, as passed to the gateway script via the server. |
| PATH_INFO | Identifies the extra information included in the URL after the identification of the CGI script. |
| PATH_TRANSLATED | Set by the server based on the PATH_INFO variable. The server translates the PATH_INFO variable into this variable. |
| QUERY_STRING | Set to the query string (if the URL contains a query string). |
| REMOTE_ADDR | Identifies the Internet Protocol address of the remote computer making the request. |
| REMOTE_HOST | Identifies the name of the machine making the request. |
| REMOTE_IDENT | Identifies the machine making the request. |
| REMOTE_USER | Identifies the user name as authenticated by the user. |
| REQUEST_METHOD | Indicates the method by which the request was made. |
| SCRIPT_NAME | Identifies the virtual path to the script being executed. |
| SERVER_NAME | Identifies the server by its host name, alias, or IP address. |
| SERVER_PORT | Identifies the port number the server received the request on. |
| SERVER_PROTOCOL | Indicates the protocol of the request sent to the server. |
| SERVER_SOFTWARE | Identifies the Web server software. |
The AUTH_TYPE variable provides access control to protected areas of the Web server and can be used only on servers that support user authentication. If an area of the Web site has no access control, the AUTH_TYPE variable has no value associated with it. If an area of the Web site has access control, the AUTH_TYPE variable is set to a specific value that identifies the authentication scheme being used. Otherwise, the variable has no value associated with it. A simple challenge-response authorization mechanism is implemented under current versions of HTTP.
Using this mechanism, the server can challenge a client's request and the client can respond. To do this, the server sets a value for the AUTH_TYPE variable and the client supplies a matching value. The next step is to authenticate the user. Using the basic authentication scheme, the user's browser must supply authentication information that uniquely identifies the user. This information includes a user ID and password.
Under the current implementation of HTTP, HTTP 1.0, the basic authentication scheme is the most commonly used authentication method. To specify this method, set the AUTH_TYPE variable as follows:
AUTH_TYPE = Basic
The CONTENT_LENGTH variable provides a way of tracking the length of the data string. This variable tells the client and server how much data to read on the standard input stream. The value of the variable corresponds to the number of characters in the data passed with the request. If no data is being passed, the variable has no value.
As long as the characters are represented as octets, the value of the CONTENT_LENGTH variable will be the precise number of characters passed as standard input or standard output. Thus, if 25 characters are passed and they are represented as octets, the CONTENT_LENGTH variable will have the following value:
CONTENT_LENGTH = 25
| Note |
An octet is a set of eight bits. Thus, when characters are represented as octets, they are passed in eight-bit groupings. |
The CONTENT_TYPE variable indicates the data's MIME type. MIME typing is a feature of HTTP 1.0 and is not available on servers using HTTP 0.9. This variable is set only when attached data is passed using the standard input or output stream. The value assigned to the variable identifies the MIME type and subtype as follows:
CONTENT_TYPE = type/subtype
MIME types are broken down into basic type categories. Each data
type category has a primary subtype associated with it. Table
26.2 shows the basic MIME types and their descriptions.
| Type | Description |
| application | Binary data that can be executed or used with another application |
| audio | A sound file that requires an output device to preview |
| image | A picture that requires an output device to preview |
| message | An encapsulated mail message |
| multipart | Data consisting of multiple parts and possibly many data types |
| text | Textual data that can be represented in any character set or formatting language |
| video | A video file that requires an output device to preview |
| x-world | Experimental data type for world files |
MIME subtypes are defined in three categories: primary, additionally
defined, and extended. The primary subtype is the primary type
of data adopted for use as a MIME content type. Additionally defined
data types are additional subtypes that have been officially adopted
as MIME content types. Extended data types are experimental subtypes
that have not been officially adopted as MIME content types. You
can easily identify extended subtypes because they begin with
the letter x followed by a hyphen. Table 26.3 lists common
MIME types and their descriptions.
| Type/Subtype | Description |
| application/mac-binhex40 | Macintosh binary-formatted data |
| application/msword | Microsoft Word document |
| application/octet-stream | Binary data that can be executed or used with another application |
| application/pdf | ACROBAT PDF document |
| application/postscript | Postscript-formatted data |
| application/rtf | Rich Text Format (RTF) document |
| application/x-compress | Data that has been compressed using UNIX compress |
| application/x-dvi | Device-independent file |
| application/x-gzip | Data that has been compressed using UNIX gzip |
| application/x-latex | LATEX document |
| application/x-tar | Data that has been archived using UNIX tar |
| application/x-zip-compressed | Data that has been compressed using PKZip or WinZip |
| audio/basic | Audio in a nondescript format |
| audio/x-aiff | Audio in Apple AIFF format |
| audio/x-wav | Audio in Microsoft WAV format |
| image/gif | Image in gif format |
| image/jpeg | Image in JPEG format |
| image/tiff | Image in TIFF format |
| image/x-portable-bitmap | Portable bitmap |
| image/x-portable-graymap | Portable graymap |
| image/x-portable-pixmap | Portable pixmap |
| image/x-xbitmap | X-bitmap |
| image/x-xpixmap | X-pixmap |
| message/external-body | Message with external data source |
| message/partial | Fragmented or partial message |
| message/rfc822 | RFC 822-compliant message |
| multipart/alternative | Data with alternative formats |
| multipart/digest | Multipart message digest |
| multipart/mixed | Multipart message with data in multiple formats |
| multipart/parallel | Multipart data with parts that should be viewed simultaneously |
| text/html | HTML-formatted text |
| text/plain | Plain text with no HTML formatting included |
| video/mpeg | Video in the MPEG format |
| video/quicktime | Video in the Apple QuickTime format |
| video/x-msvideo | Video in the Microsoft AVI format |
| x-world/x-vrml | VRML world file |
Some MIME content types can be used with additional parameters. These content types include text/plain, text/html, and all multipart message data. The charset parameter, which is optional, is used with the text/plain type to identify the character set used for the data. If a charset is not specified, the default value charset=us-ascii is assumed. Other values for charset include any character set approved by the International Standards Organization. These character sets are defined by ISO-8859-1 to ISO-8859-9 and are specified as follows:
CONTENT_TYPE = text/plain; charset=iso-8859-1
The version parameter, which also is optional, is used with the text/html type to identify the version of HTML used. If this parameter is set, the browser reading the data interprets the data if the browser supports the version of HTML specified. The following document conforms to the HTML 3.2 specification:
CONTENT_TYPE = text/html; version=3.2
The boundary parameter, which is required, is used with multipart data to identify the boundary string that separates message parts. The boundary value is set to a string of 1 to 70 characters. Although the string cannot end in a space, it can contain any valid letter or number and can include spaces and a limited set of special characters. Boundary parameters are unique strings that are defined as follows:
CONTENT_TYPE = multipart/mixed; boundary=boundary_string
The GATEWAY_INTERFACE variable indicates which version of the CGI specification the server is using. The value assigned to the variable identifies the name and version of the specification used as follows:
GATEWAY_INTERFACE = name/version
The current version of the CGI specification is 1.1. A server conforming to this version would set the GATEWAY_INTERFACE variable as follows:
GATEWAY_INTERFACE = CGI/1.1
The HTTP_AccEPT variable defines the types of data the client will accept. The acceptable values are expressed as a type/subtype pair. Each type/subtype pair is separated by commas, as in
type/subtype, type/subtype
Most clients accept dozens of MIME types. The following lines identify all the MIME content types accepted by this particular client:
HTTP_AccEPT = application/msword, application/octet-stream, application/postscript, application/rtf, application/x-zip-compressed, audio/basic, audio/x-aiff, audio/x-wav, image/gif, image/jpeg, image/tiff, image/x-portable-bitmap, message/external-body, message/partial, message/rfc822, multipart/alternative, multipart/digest, multipart/mixed, multipart/parallel, text/html,
text/plain, video/mpeg, video/quicktime, video/x-msvideo
The HTTP_USER_AGENT variable identifies the type of browser used to send the request. The acceptable values are expressed as software type/version or library/version. The following HTTP_USER_AGENT variable identifies Netscape Navigator Version 2.0:
HTTP_USER_AGENT = Mozilla/2.0
As you can see, Netscape uses the alias Mozilla to identify itself. The primary types of clients that set this variable are browsers, Web spiders, and robots. Although this variable is useful for identifying the type of client used to access a script, keep in mind that not all clients set this variable.
Here's a list of software type values used by popular browsers:
Arena
Enhanced ncSA Mosaic
Lynx
MacWeb
Mozilla
ncSA Mosaic
NetCruiser
WebExplorer
WinMosaic
These values are used by Web spiders:
Lycos
MOMSpider
WebCrawler
The PATH_INFO variable specifies extra path information and can be used to send additional information to a gateway script. The extra path information follows the URL to the gateway script referenced. Generally, this information is a virtual or relative path to a resource that the server must interpret. If the URL to the CGI script is specified in your document as
/usr/cgi-bin/formparse.pl/home.html
then the PATH_INFO variable would be set as follows:
PATH_INFO = /home.html
Servers translate the PATH_INFO variable into the PATH_TRANSLATED variable by inserting the default Web document's directory path in front of the extra path information. For example, if the PATH_INFO variable were set to home.html and the default directory were /usr/documents/pubs, the PATH_TRANSLATED variable would be set as follows:
PATH_TRANSLATED = /usr/documents/pubs/home.html
The QUERY_STRING variable specifies an URL-encoded search string. You set this variable when you use the GET method to submit a fill-out form or when you use an ISINDEX query to search a document. The query string is separated from the URL by a question mark. The user submits all the information following the question mark separating the URL from the query string. The following is an example:
/usr/cgi-bin/formparse.pl?string
When the query string is URL-encoded, the browser encodes key parts of the string. The plus sign is a placeholder between words and acts as a substitute for spaces:
/usr/cgi-bin/formparse.pl?word1+word2+word3
Equal signs separate keys assigned by the publisher from values entered by the user. In the following example, response is the key assigned by the publisher, and never is the value entered by the user:
/usr/cgi-bin/formparse.pl?response=never
Ampersand symbols separate sets of keys and values. In the following example, response is the first key assigned by the publisher, and sometimes is the value entered by the user. The second key assigned by the publisher is reason, and the value entered by the user is I am not really sure:
/usr/cgi-bin/formparse.pl?response=sometimes&reason=I+am+not+really+sure
Finally, the percent sign is used to identify escape characters. Following the percent sign is an escape code for a special character expressed as a hexadecimal value. Here is how the previous query string could be rewritten using the escape code for an apostrophe:
/usr/cgi-bin/formparse.pl?response=sometimes&reason=I%27m+not+really+sure
The REMOTE_ADDR variable is set to the Internet Protocol (IP) address of the remote computer making the request. The IP address is a numeric identifier for a networked computer. The following is an example of the REMOTE_ADDR variable:
REMOTE_ADDR = 205.1.20.11
The REMOTE_HOST variable specifies the name of the host computer making a request. This variable is set only if the server can figure out this information using a reverse lookup procedure. If this variable is set, the full domain and host name are used as follows:
REMOTE_HOST = www.tvp.com
The REMOTE_IDENT variable identifies the remote user making a request. The variable is set only if the server and the remote machine making the request support the identification protocol. Further, information on the remote user is not always available, so you should not rely on it even when it is available. If the variable is set, the associated value is a fully expressed name that contains the domain information as well:
REMOTE_IDENT = william.www.tvp.com
The REMOTE_USER variable is the user name as authenticated by the user, and as such is the only variable you should rely upon to identify a user. As with other types of user authentication, this variable is set only if the server supports user authentication and if the gateway script is protected. If the variable is set, the associated value is the user's identification as sent by the client to the server:
REMOTE_USER = william
The REQUEST_METHOD variable
specifies the method by which the request was made. For
HTTP 1.0, the methods could be any of the following:
GET
HEAD
POST
PUT
DELETE
LINK
UNLINK
The GET, HEAD, and POST methods are the most commonly used request methods. Both GET and POST are used to submit forms. The HEAD method could be specified as follows:
REQUEST_METHOD = HEAD
The SCRIPT_NAME variable specifies the virtual path to the script being executed. This information is useful if the script generates an HTML document that references the script. If the URL specified in your HTML document is
http://tvp.com/cgi-bin/formparse.pl
the SCRIPT_NAME variable is set as follows:
SCRIPT_NAME = /cgi-bin/formparse.pl
The SERVER_NAME variable identifies the server by its host name, alias, or IP address. This variable is always set and could be specified as follows:
SERVER_NAME = tvp.com
The SERVER_PORT variable specifies the port number on which the server received the request. This information can be interpreted from the URL to the script if necessary. However, most servers use the default port of 80 for HTTP requests. If the URL specified in your HTML document is
http://www.ncsa.edu:8080/cgi-bin/formparse.pl
the SERVER_PORT variable is set as follows:
SERVER_PORT = 8080
The SERVER_PROTOCOL variable identifies the protocol used to send the request. The value assigned to the variable identifies the name and version of the protocol used. The format is name/version, such as HTTP/1.0. The variable is set as follows:
SERVER_PROTOCOL = HTTP/1.0
The SERVER_SOFTWARE variable identifies the name and version of the server software. The format for values assigned to the variable is name/version, such as CERN/2.17. The variable is set as follows:
SERVER_SOFTWARE = CERN/2.17
Most input sent to a Web server is used to set environment variables, yet not all input fits neatly into an environment variable. When a user submits data to be processed by a gateway script, this data is received as an URL-encoded search string or through the standard input stream. The server knows how to process this data because of the method (either POST or GET in HTTP 1.0) used to submit the data.
Sending data as standard input is the most direct way to send data. The server tells the gateway script how many eight-bit sets of data to read from standard input. The script opens the standard input stream and reads the specified amount of data. Although long URL-encoded search strings may get truncated, data sent on the standard input stream will not. Consequently, the standard input stream is the preferred way to pass data.
You can identify a submission method when you create your fill-out forms. Under HTTP 1.0, two submission methods for forms exist. The HTTP GET method uses URL-encoded search strings. When a server receives an URL-encoded search string, the server assigns the value of the search string to the QUERY_STRING variable.
The HTTP POST method uses the standard input streams. When a server receives data by the standard input stream, the server assigns the value associated with the length of the input stream to the CONTENT_LENGTH variable.
Supose you have a Web document containing a form with three key fields: NAME, ADDRESS, and PHONE_NUMBER. Assume the URL to the script is http://www.tvp.com/cgi-bin/survey.pl, and the user responds as follows:
Sandy Brown 12 Sunny Lane WhoVille, USA
987-654-3210
Identical information submitted using the GET and POST methods is treated differently by the server. When the GET method is used, the server sets the following environment variables, and then passes the input to the survey.pl script:
PATH =/bin:/usr/bin:/usr/etc:/usr/ucb SERVER_SOFTWARE = CERN/3.0 SERVER_NAME = www.tvp.com GATEWAY_INTERFACE = CGI/1.1 SERVER_PROTOCOL = HTTP/1.0 SERVER_PORT=80 REQUEST_METHOD = GET HTTP_AccEPT = text/plain, text/html, application/rtf, application/postscript, audio/basic, audio/x-aiff, image/gif, image/jpeg, image/tiff, video/mpeg PATH_INFO = PATH_TRANSLATED = SCRIPT_NAME = /cgi-bin/survey.pl QUERY_STRING = NAME=Sandy+Brown&ADDRESS=12+Sunny+Lane+WhoVille,+USA &PHONE_NUMBER=987-654-3210 REMOTE_HOST = REMOTE_ADDR = REMOTE_USER = AUTH_TYPE = CONTENT_TYPE = CONTENT_LENGTH =
When the POST method is used, the server sets the following environment variables, and then passes the input to the survey.pl script:
PATH =/bin:/usr/bin:/usr/etc:/usr/ucb SERVER_SOFTWARE = CERN/3.0 SERVER_NAME = www.tvp.com GATEWAY_INTERFACE = CGI/1.1 SERVER_PROTOCOL = HTTP/1.0 SERVER_PORT=80 REQUEST_METHOD = POST HTTP_AccEPT = text/plain, text/html, application/rtf, application/postscript, audio/basic, audio/x-aiff, image/gif, image/jpeg, image/tiff, video/mpeg PATH_INFO = PATH_TRANSLATED = SCRIPT_NAME = /cgi-bin/survey.pl QUERY_STRING = REMOTE_HOST = REMOTE_ADDR = REMOTE_USER = AUTH_TYPE = CONTENT_TYPE = application/x-www-form-urlencoded
CONTENT_LENGTH = 81
The following POST-submitted data is passed to the gateway script through the standard input stream:
NAME=Sandy+Brown&ADDRESS=12+Sunny+Lane+WhoVille,+USA&PHONE_NUMBER=987-654-3210
After the script finishes processing the input, the script should return output to the server. The server will then return the output to the client. Generally, this output is in the form of an HTTP response that includes a header followed by a blank line and a body. Although the CGI header output is strictly formatted, the body of the output is formatted in the manner you specify in the header. For example, the body can contain an HTML document for the client to display.
CGI headers contain directives to the server. Currently, these three server directives are valid:
A single header can contain one or all of the server directives. Your CGI script outputs these directives to the server. Although the header is followed by a blank line that separates the header from the body, the output does not have to contain a body.
The Content-Type field in a CGI header identifies the MIME type of the data you are sending back to the client. Usually the data output from a script is a fully formatted document, such as an HTML document. You could specify this output in the header as follows:
Content-Type: text/html
The output of your script doesn't have to be a document created within the script. You can reference any document on the Web using the Location field. The Location field references a file by its URL. Servers process location references either directly or indirectly depending on the location of the file. If the server can find the file locally, it passes the file to the client. Otherwise, the server redirects the URL to the client and the client has to retrieve the file. You can specify a location in a script as follows:
Location: http://www.tvpress.com/
| Note |
Some older browsers don't support automatic redirection. Consequently, you may want to consider adding an HTML -formatted message body to the output. This message body will only be displayed if a browser cannot use the location URL. |
The Status field passes a status line to the server for forwarding to the client. Status codes are expressed as a three-digit code followed by a string that generally explains what has occurred. The first digit of a status code shows the general status as follows:
1XX Not yet allocated
2XX Success
3XX Redirection
4XX Client error
5XX Server error
Although many status codes are used by servers, the status codes you pass to a client via your CGI script are usually client error codes. Suppose the script could not find a file and you have specified that in such cases, instead of returning nothing, the script should output an error code. Here is a list of the client error codes you may want to use:
Status: 401 Unauthorized Authentication has failed. User is not allowed to access the file and should try again.
Status: 403 Forbidden. The request is not acceptable. User is not permitted to access file.
Status: 404 Not found. The specified resource could not be found.
Status: 405 Method not allowed. The submission method used is not allowed.
Creating the output from a CGI script is easier than it may seem. All you have to do is format the output into a header and body using your favorite programming language. This section contains two examples. The first example is in the Perl programming language. The second example is in the UNIX Bourne shell.
If you wanted the script to output a simple HTML document using Perl, here is how you could do it:
#!/usr/bin/perl #Create header with extra line space print "Content-Type: text/html\n\n"; #Add body in HTML format print <<"MAIN"; <HTML><HEAD><TITLE>Output from Script</TITLE></HEAD> <BODY> <H1>Top 10 Reasons for Using CGI</H1> <P>10. Customer feedback.</P> <P>9. Obtaining questionnaire and survey responses.</P> <P>8. Tracking visitor count.</P> <P>7. Automating searches.</P> <P>6. Creating easy database interfaces.</P> <P>5. Building gateways to other protocols.</P> <P>4. HTML 2.0 image maps.</P> <P>3. User authentication.</P> <P>2. Online order processing.</P> <P>1. Generating documents on the fly.</P> </BODY> MAIN
If you wanted the script to output a simple HTML document in Bourne shell, here's how you could do it:
#!/bin/sh #Create header with extra line space echo "Content-Type: text/html" #Add body in HTML format cat << MAIN <HTML><HEAD><TITLE>Output from Script</TITLE></HEAD> <BODY> <H1>Top 10 Reasons for Using CGI</H1> <P>10. Customer feedback.</P> <P>9. Obtaining questionnaire and survey responses.</P> <P>8. Tracking visitor count.</P> <P>7. Automating searches.</P> <P>6. Creating easy database interfaces.</P> <P>5. Building gateways to other protocols.</P> <P>4. HTML 2.0 Image maps.</P> <P>3. User authentication.</P> <P>2. Online order processing.</P> <P>1. Generating documents on the fly.</P> </BODY> MAIN
The server processing the output sets environment variables, creates an HTTP header, and then sends the data on to the client. Here is how the HTTP header might look coming from a CERN Web server:
HTTP/1.0 302 Found MIME-Version: 1.0 Server: CERN/3.0 Date: Monday, 4-Mar-96 23:59:59 HST Content-Type: text/html Content-Length: 485 <HTML><HEAD><TITLE>Output from Script</TITLE></HEAD> <BODY> <H1>Top 10 Reasons for Using CGI</H1> <P>10. Customer feedback.</P> <P>9. Obtaining questionnaire and survey responses.</P> <P>8. Tracking visitor count.</P> <P>7. Automating searches.</P> <P>6. Creating easy database interfaces.</P> <P>5. Building gateways to other protocols.</P> <P>4. HTML 2.0 image maps.</P> <P>3. User authentication.</P> <P>2. Online order processing.</P> <P>1. Generating documents on the fly.</P>
</BODY>
Server-push technology allows you to make dynamic updates to your documents. Server-push and gateway scripts make perfect combinations. The output of your gateway scripts generally has a defined HTTP header. Using this header, you can create a document that will reload itself or call another document.
Server-push is designed to complement client-pull and behaves similarly. Client-pull is used to refresh or retrieve an entire document with header directives in the document specifying time intervals; server-push is generally used to dynamically update a component of a document, such as an image, with server-side scripts setting the time intervals.
Using client-pull, the client must contact the server, wait for a reply, establish a connection, retrieve the data, close the connection, and repeat the entire process when requesting new data. Using server-push, the client contacts the server, waits for a reply, establishes a connection, retrieves the data, and then retrieves any new data it requests without repeating the process. Besides giving the server total control over when and how often data is sent, server-pull enables users to easily interrupt data updates using the Stop feature of their browsers.
To create a document that updates itself using server-push, you can use the Content-Type directive. The Content-Type field in a header identifies the MIME type of data you are sending back to the client. Use the following MIME content type with client-pull:
multipart/x-mixed-replace
Although most HTTP responses consist of only a single message or document, the multipart message type enables a single response to contain multiple messages or documents. The multipart message type encapsulates other message types within the document you are sending to the client.
When you use the MIME type multipart/mixed, you must also specify the boundary string that separates the messages. This string should be unique and not used elsewhere in the text of the message. After the boundary string, you specify the MIME type of the individual message in a header. The header should be followed by a blank line if the message has a body section as well.
The following sample response header for a multipart message contains three messages that are separated by a unique boundary string:
Content-type: multipart/mixed;boundary=--YourUniqueBoundaryString --YourUniqueBoundaryString Content-type: text/plain Text for the first message. --YourUniqueBoundaryString Content-type: text/plain Text for the second message. --YourUniqueBoundaryString Content-type: text/plain Text for the third message.
--YourUniqueBoundaryString
| Caution |
Some servers have problems processing the multipart message type if you put any spaces after the multipart/mixed assignment. For this reason, do not insert spaces after the semicolon or around the equal sign when you assign the mandatory boundary string. |
To use server-push, you use Netscape's experimental multipart/x-mixed-replace message type. This message type tells the client to replace the current data with the new data. The client finishes loading the current document before loading and displaying the new document. Transforming the multipart/mixed message type into a multipart/x-mixed replace message is easy. You simply change the type assignment in the first line as follows:
Content-type: multipart/x-mixed-replace;boundary=--YourUniqueBoundaryString --YourUniqueBoundaryString Content-type: text/plain Text for the first message. --YourUniqueBoundaryString Content-type: text/plain Text for the second message. --YourUniqueBoundaryString Content-type: text/plain Text for the third message.
--YourUniqueBoundaryString
Ideally, you would send the client a multipart message as the output from a gateway script. Because the multipart message can contain as many sections as you want, you can use a for loop or a do-while loop to keep the script running indefinitely. By building hypertext links into the output documents, you can enable the reader to interact with your publication. The links could even call other gateway scripts that use server-pull animation.
This simple script uses server-pull to display the current time once each second:
#!/bin/sh # Build multipart message using the x-mixed-replace format # Ensure header is separated from body with space cat << MAIN Content-type: multipart/x-mixed-replace;boundary=--YourUniqueBoundaryString --YourUniqueBoundaryString MAIN # Set while to true to loop continuously while true do #Create Document cat << TIME Content-type: text/html <HTML> <HEAD> <TITLE>Current Time in Hawaii</TITLE> </HEAD> <BODY BGCOLOR="#0000ff" text="#ffff00" link="#fffbf0" vlink="#808000" alink="#ff0000"> <BIG> date </BIG> </BODY> </HTML> --YourUniqueBoundaryString TIME #wait 1 second before repeating the loop sleep 1 done
| Caution |
Although this Bourne shell script works, you should not use UNIX shell scripts with serverpush if the connection is going to be open indefinitely. The primary reason for this is that shell scripts will not stop running when the user severs the connection, and if the script is running on your server, it is using system resources. For this reason, if the server-push will open a connection of indefinite duration, use a language that handles interrupts well, such as C or Perl. |
A neat trick you can do with server-push is to have the server update an inline image in the current document. Creating an inline animation is as easy as adding an inline image with an URL to a gateway script that uses server-push. Although the current document will not get updated, the image will. The following document contains an inline image that is updated:
<HTML> <HEAD> <TITLE>Server Push</TITLE> </HEAD> <BODY> <CENTER> <IMG WIDTH=64 HEIGHT=64 SRC="http://tvp.com/cgi-bin/doit.cgi"> </CENTER> <P>Only the image in this document will get updated. <P>The rest of the document will not be updated. </BODY> </HTML>
Creating a gateway script that uses server-push is also easy. All you have to do is create a script that builds a document in multipart message format. The following script in the C programming language called doit.c was created by Rob McCool. The script enables you to animate a series of images. The number of stages for the animation is determined by the variable LASTchAR.
With the LASTchAR variable, you set the last character of the file name for your images, which are uniquely named and end in an alphabetic letter from lowercase a to the value for LASTchAR. To use this script with images in gif format, all you have to do is modify the LASTchAR variable as necessary and compile the script using your favorite compiler. Listing 26.1 shows the doit.c script.
Listing 26.1. Modified doit.c script.
/*
* doit.c: Quick hack to play a sequence of gif files.
*
* This is a modified version of Rob McCool's original script.
*
* This code is released into the public domain. Do whatever
* you want with it.
*
*/
/* the following lines set up the libraries and variables
* The most important variables are: HEADER and LASTchAR
* With the HEADER variable, you set the HTTP response header.
* With the LASTchAR variable, you set the last character of the
* file name for your images. As this animation has 10 stages,
* the image names will end in a to j.
*/
#include <sys/types.h>
#include <sys/mman.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <stdio.h>
#define LASTchAR 'j'
#define HEADER \
"Content-type: multipart/x-mixed-replace;boundary=YourUniqueBoundaryString\n" \
#define RANDOMSTRING "\n--YourUniqueBoundaryString\n"
#define ENDSTRING "\n--YourUniqueBoundaryString--\n"
#define CTSTRING "Content-type: image/gif\n\n"
int main(int argc, char *argv[])
{
struct stat fi;
char fn[32];
caddr_t fp;
unsigned char x;
int fd;
if(write(STDOUT_FILENO, HEADER, strlen(HEADER)) == -1)
exit(0);
if(write(STDOUT_FILENO, RANDOMSTRING, strlen(RANDOMSTRING)) == -1)
exit(0);
x = 'a';
while(1) {
sleep(1);
if(write(STDOUT_FILENO, CTSTRING, strlen(CTSTRING)) == -1)
exit(0);
/* The next line defines the name of the images to use for creating
* the inline animation. Here, the images are located in the images
* subdirectory of the current directory which should be cgi-bin.
* The image files must be named as follows:
* images/Aa.gif
* images/Ab.gif
* images/Ac.gif
* images/Ad.gif
* images/Ae.gif
* images/Af.gif
* images/Ag.gif
* images/Ah.gif
* images/Ai.gif
* images/Aj.gif
*/
sprintf(fn, "images/A%c.gif", (char) x);
if( (fd = open(fn, O_RDONLY)) == -1)
continue;
fstat(fd, &fi);
fp = mmap(NULL, fi.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if(fp == (caddr_t) -1)
exit(0);
if(write(STDOUT_FILENO, (void *) fp, fi.st_size) == -1)
exit(0);
munmap(fp, fi.st_size);
close(fd);
if(write(STDOUT_FILENO, RANDOMSTRING, strlen(RANDOMSTRING)) == -1)
exit(0);
if(x == LASTchAR)
exit(0);
else ++x;
The common gateway interface opens the door for adding advanced features to your Web publications. This workhorse running quietly in the background lets you use fill-out forms, database queries, and index searches and create documents on the fly. You use CGI whenever you want to open a two-way communication channel with the reader.
Although CGI enhancement is a click of the mouse button away for most readers, CGI enhancement means extra work for Web publishers. Still, the payoff associated with CGI makes the extra effort truly worthwhile.
