blob: 56643d05fc4993fc4b5cff3d4f59772aed9895a3 [file] [log] [blame]
Guido van Rossum470be141995-03-17 16:07:09 +00001\section{Standard Module \sectcode{cgi}}
Guido van Rossume47da0a1997-07-17 16:34:52 +00002\label{module-cgi}
Guido van Rossuma12ef941995-02-27 17:53:25 +00003\stmodindex{cgi}
4\indexii{WWW}{server}
5\indexii{CGI}{protocol}
6\indexii{HTTP}{protocol}
7\indexii{MIME}{headers}
8\index{URL}
9
Guido van Rossum86751151995-02-28 17:14:32 +000010\renewcommand{\indexsubitem}{(in module cgi)}
11
Guido van Rossuma29cc971996-07-30 18:22:07 +000012Support module for CGI (Common Gateway Interface) scripts.
Guido van Rossuma12ef941995-02-27 17:53:25 +000013
Guido van Rossuma29cc971996-07-30 18:22:07 +000014This module defines a number of utilities for use by CGI scripts
15written in Python.
Guido van Rossuma12ef941995-02-27 17:53:25 +000016
Guido van Rossuma29cc971996-07-30 18:22:07 +000017\subsection{Introduction}
18\nodename{Introduction to the CGI module}
Guido van Rossuma12ef941995-02-27 17:53:25 +000019
Guido van Rossuma29cc971996-07-30 18:22:07 +000020A CGI script is invoked by an HTTP server, usually to process user
21input submitted through an HTML \code{<FORM>} or \code{<ISINPUT>} element.
22
23Most often, CGI scripts live in the server's special \code{cgi-bin}
24directory. The HTTP server places all sorts of information about the
25request (such as the client's hostname, the requested URL, the query
26string, and lots of other goodies) in the script's shell environment,
27executes the script, and sends the script's output back to the client.
28
29The script's input is connected to the client too, and sometimes the
30form data is read this way; at other times the form data is passed via
31the ``query string'' part of the URL. This module (\code{cgi.py}) is intended
32to take care of the different cases and provide a simpler interface to
33the Python script. It also provides a number of utilities that help
34in debugging scripts, and the latest addition is support for file
35uploads from a form (if your browser supports it -- Grail 0.3 and
36Netscape 2.0 do).
37
38The output of a CGI script should consist of two sections, separated
39by a blank line. The first section contains a number of headers,
40telling the client what kind of data is following. Python code to
41generate a minimal header section looks like this:
Guido van Rossuma12ef941995-02-27 17:53:25 +000042
Guido van Rossume47da0a1997-07-17 16:34:52 +000043\bcode\begin{verbatim}
44print "Content-type: text/html" # HTML is following
45print # blank line, end of headers
46\end{verbatim}\ecode
47%
Guido van Rossuma29cc971996-07-30 18:22:07 +000048The second section is usually HTML, which allows the client software
49to display nicely formatted text with header, in-line images, etc.
50Here's Python code that prints a simple piece of HTML:
Guido van Rossum470be141995-03-17 16:07:09 +000051
Guido van Rossume47da0a1997-07-17 16:34:52 +000052\bcode\begin{verbatim}
53print "<TITLE>CGI script output</TITLE>"
54print "<H1>This is my first CGI script</H1>"
55print "Hello, world!"
56\end{verbatim}\ecode
57%
Guido van Rossuma29cc971996-07-30 18:22:07 +000058(It may not be fully legal HTML according to the letter of the
59standard, but any browser will understand it.)
Guido van Rossum470be141995-03-17 16:07:09 +000060
Guido van Rossuma29cc971996-07-30 18:22:07 +000061\subsection{Using the cgi module}
62\nodename{Using the cgi module}
63
64Begin by writing \code{import cgi}. Don't use \code{from cgi import *} -- the
65module defines all sorts of names for its own use or for backward
66compatibility that you don't want in your namespace.
67
68It's best to use the \code{FieldStorage} class. The other classes define in this
69module are provided mostly for backward compatibility. Instantiate it
70exactly once, without arguments. This reads the form contents from
71standard input or the environment (depending on the value of various
72environment variables set according to the CGI standard). Since it may
73consume standard input, it should be instantiated only once.
74
75The \code{FieldStorage} instance can be accessed as if it were a Python
76dictionary. For instance, the following code (which assumes that the
77\code{Content-type} header and blank line have already been printed) checks that
78the fields \code{name} and \code{addr} are both set to a non-empty string:
Guido van Rossum470be141995-03-17 16:07:09 +000079
Guido van Rossume47da0a1997-07-17 16:34:52 +000080\bcode\begin{verbatim}
81form = cgi.FieldStorage()
82form_ok = 0
83if form.has_key("name") and form.has_key("addr"):
84 if form["name"].value != "" and form["addr"].value != "":
85 form_ok = 1
86if not form_ok:
87 print "<H1>Error</H1>"
88 print "Please fill in the name and addr fields."
89 return
90...further form processing here...
91\end{verbatim}\ecode
92%
Guido van Rossuma29cc971996-07-30 18:22:07 +000093Here the fields, accessed through \code{form[key]}, are themselves instances
94of \code{FieldStorage} (or \code{MiniFieldStorage}, depending on the form encoding).
Guido van Rossum470be141995-03-17 16:07:09 +000095
Guido van Rossuma29cc971996-07-30 18:22:07 +000096If the submitted form data contains more than one field with the same
97name, the object retrieved by \code{form[key]} is not a \code{(Mini)FieldStorage}
98instance but a list of such instances. If you expect this possibility
99(i.e., when your HTML form comtains multiple fields with the same
100name), use the \code{type()} function to determine whether you have a single
101instance or a list of instances. For example, here's code that
102concatenates any number of username fields, separated by commas:
Guido van Rossum470be141995-03-17 16:07:09 +0000103
Guido van Rossume47da0a1997-07-17 16:34:52 +0000104\bcode\begin{verbatim}
105username = form["username"]
106if type(username) is type([]):
107 # Multiple username fields specified
108 usernames = ""
109 for item in username:
110 if usernames:
111 # Next item -- insert comma
112 usernames = usernames + "," + item.value
113 else:
114 # First item -- don't insert comma
115 usernames = item.value
116else:
117 # Single username field specified
118 usernames = username.value
119\end{verbatim}\ecode
120%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000121If a field represents an uploaded file, the value attribute reads the
122entire file in memory as a string. This may not be what you want. You can
123test for an uploaded file by testing either the filename attribute or the
124file attribute. You can then read the data at leasure from the file
125attribute:
126
Guido van Rossume47da0a1997-07-17 16:34:52 +0000127\bcode\begin{verbatim}
128fileitem = form["userfile"]
129if fileitem.file:
130 # It's an uploaded file; count lines
131 linecount = 0
132 while 1:
133 line = fileitem.file.readline()
134 if not line: break
135 linecount = linecount + 1
136\end{verbatim}\ecode
137%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000138The file upload draft standard entertains the possibility of uploading
139multiple files from one field (using a recursive \code{multipart/*}
140encoding). When this occurs, the item will be a dictionary-like
141FieldStorage item. This can be determined by testing its type
142attribute, which should have the value \code{multipart/form-data} (or
143perhaps another string beginning with \code{multipart/} It this case, it
144can be iterated over recursively just like the top-level form object.
145
146When a form is submitted in the ``old'' format (as the query string or as a
147single data part of type \code{application/x-www-form-urlencoded}), the items
148will actually be instances of the class \code{MiniFieldStorage}. In this case,
149the list, file and filename attributes are always \code{None}.
150
151
152\subsection{Old classes}
153
154These classes, present in earlier versions of the \code{cgi} module, are still
Guido van Rossuma5a4c2a1996-10-24 14:47:44 +0000155supported for backward compatibility. New applications should use the
156FieldStorage class.
Guido van Rossuma29cc971996-07-30 18:22:07 +0000157
158\code{SvFormContentDict}: single value form content as dictionary; assumes each
159field name occurs in the form only once.
160
161\code{FormContentDict}: multiple value form content as dictionary (the form
162items are lists of values). Useful if your form contains multiple
163fields with the same name.
164
165Other classes (\code{FormContent}, \code{InterpFormContentDict}) are present for
166backwards compatibility with really old applications only. If you still
167use these and would be inconvenienced when they disappeared from a next
168version of this module, drop me a note.
169
170
171\subsection{Functions}
Fred Drake4b3f0311996-12-13 22:04:31 +0000172\nodename{Functions in cgi module}
Guido van Rossuma29cc971996-07-30 18:22:07 +0000173
174These are useful if you want more control, or if you want to employ
175some of the algorithms implemented in this module in other
176circumstances.
177
178\begin{funcdesc}{parse}{fp}: Parse a query in the environment or from a file (default \code{sys.stdin}).
179\end{funcdesc}
180
181\begin{funcdesc}{parse_qs}{qs}: parse a query string given as a string argument (data of type
182\code{application/x-www-form-urlencoded}).
183\end{funcdesc}
184
185\begin{funcdesc}{parse_multipart}{fp\, pdict}: parse input of type \code{multipart/form-data} (for
186file uploads). Arguments are \code{fp} for the input file and
187 \code{pdict} for the dictionary containing other parameters of \code{content-type} header
188
189 Returns a dictionary just like \code{parse_qs()}: keys are the field names, each
190 value is a list of values for that field. This is easy to use but not
191 much good if you are expecting megabytes to be uploaded -- in that case,
192 use the \code{FieldStorage} class instead which is much more flexible. Note
193 that \code{content-type} is the raw, unparsed contents of the \code{content-type}
194 header.
195
196 Note that this does not parse nested multipart parts -- use \code{FieldStorage} for
197 that.
198\end{funcdesc}
199
200\begin{funcdesc}{parse_header}{string}: parse a header like \code{Content-type} into a main
201content-type and a dictionary of parameters.
202\end{funcdesc}
203
204\begin{funcdesc}{test}{}: robust test CGI script, usable as main program.
205 Writes minimal HTTP headers and formats all information provided to
206 the script in HTML form.
207\end{funcdesc}
208
209\begin{funcdesc}{print_environ}{}: format the shell environment in HTML.
210\end{funcdesc}
211
212\begin{funcdesc}{print_form}{form}: format a form in HTML.
213\end{funcdesc}
214
215\begin{funcdesc}{print_directory}{}: format the current directory in HTML.
216\end{funcdesc}
217
218\begin{funcdesc}{print_environ_usage}{}: print a list of useful (used by CGI) environment variables in
219HTML.
220\end{funcdesc}
221
222\begin{funcdesc}{escape}{}: convert the characters ``\code{\&}'', ``\code{<}'' and ``\code{>}'' to HTML-safe
223sequences. Use this if you need to display text that might contain
224such characters in HTML. To translate URLs for inclusion in the HREF
225attribute of an \code{<A>} tag, use \code{urllib.quote()}.
226\end{funcdesc}
227
228
229\subsection{Caring about security}
230
231There's one important rule: if you invoke an external program (e.g.
232via the \code{os.system()} or \code{os.popen()} functions), make very sure you don't
233pass arbitrary strings received from the client to the shell. This is
234a well-known security hole whereby clever hackers anywhere on the web
235can exploit a gullible CGI script to invoke arbitrary shell commands.
236Even parts of the URL or field names cannot be trusted, since the
237request doesn't have to come from your form!
238
239To be on the safe side, if you must pass a string gotten from a form
240to a shell command, you should make sure the string contains only
241alphanumeric characters, dashes, underscores, and periods.
242
243
244\subsection{Installing your CGI script on a Unix system}
245
246Read the documentation for your HTTP server and check with your local
247system administrator to find the directory where CGI scripts should be
248installed; usually this is in a directory \code{cgi-bin} in the server tree.
249
250Make sure that your script is readable and executable by ``others''; the
251Unix file mode should be 755 (use \code{chmod 755 filename}). Make sure
252that the first line of the script contains \code{\#!} starting in column 1
253followed by the pathname of the Python interpreter, for instance:
254
Guido van Rossume47da0a1997-07-17 16:34:52 +0000255\bcode\begin{verbatim}
256#!/usr/local/bin/python
257\end{verbatim}\ecode
258%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000259Make sure the Python interpreter exists and is executable by ``others''.
260
261Make sure that any files your script needs to read or write are
262readable or writable, respectively, by ``others'' -- their mode should
263be 644 for readable and 666 for writable. This is because, for
264security reasons, the HTTP server executes your script as user
265``nobody'', without any special privileges. It can only read (write,
266execute) files that everybody can read (write, execute). The current
267directory at execution time is also different (it is usually the
268server's cgi-bin directory) and the set of environment variables is
269also different from what you get at login. in particular, don't count
270on the shell's search path for executables (\code{\$PATH}) or the Python
271module search path (\code{\$PYTHONPATH}) to be set to anything interesting.
272
273If you need to load modules from a directory which is not on Python's
274default module search path, you can change the path in your script,
275before importing other modules, e.g.:
276
Guido van Rossume47da0a1997-07-17 16:34:52 +0000277\bcode\begin{verbatim}
278import sys
279sys.path.insert(0, "/usr/home/joe/lib/python")
280sys.path.insert(0, "/usr/local/lib/python")
281\end{verbatim}\ecode
282%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000283(This way, the directory inserted last will be searched first!)
284
285Instructions for non-Unix systems will vary; check your HTTP server's
286documentation (it will usually have a section on CGI scripts).
287
288
289\subsection{Testing your CGI script}
290
291Unfortunately, a CGI script will generally not run when you try it
292from the command line, and a script that works perfectly from the
293command line may fail mysteriously when run from the server. There's
294one reason why you should still test your script from the command
295line: if it contains a syntax error, the python interpreter won't
296execute it at all, and the HTTP server will most likely send a cryptic
297error to the client.
298
299Assuming your script has no syntax errors, yet it does not work, you
300have no choice but to read the next section:
301
302
303\subsection{Debugging CGI scripts}
304
305First of all, check for trivial installation errors -- reading the
306section above on installing your CGI script carefully can save you a
307lot of time. If you wonder whether you have understood the
308installation procedure correctly, try installing a copy of this module
309file (\code{cgi.py}) as a CGI script. When invoked as a script, the file
310will dump its environment and the contents of the form in HTML form.
311Give it the right mode etc, and send it a request. If it's installed
312in the standard \code{cgi-bin} directory, it should be possible to send it a
313request by entering a URL into your browser of the form:
314
Guido van Rossume47da0a1997-07-17 16:34:52 +0000315\bcode\begin{verbatim}
316http://yourhostname/cgi-bin/cgi.py?name=Joe+Blow&addr=At+Home
317\end{verbatim}\ecode
318%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000319If this gives an error of type 404, the server cannot find the script
320-- perhaps you need to install it in a different directory. If it
321gives another error (e.g. 500), there's an installation problem that
322you should fix before trying to go any further. If you get a nicely
323formatted listing of the environment and form content (in this
324example, the fields should be listed as ``addr'' with value ``At Home''
325and ``name'' with value ``Joe Blow''), the \code{cgi.py} script has been
326installed correctly. If you follow the same procedure for your own
327script, you should now be able to debug it.
328
329The next step could be to call the \code{cgi} module's test() function from
330your script: replace its main code with the single statement
331
Guido van Rossume47da0a1997-07-17 16:34:52 +0000332\bcode\begin{verbatim}
333cgi.test()
334\end{verbatim}\ecode
335%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000336This should produce the same results as those gotten from installing
337the \code{cgi.py} file itself.
338
339When an ordinary Python script raises an unhandled exception
340(e.g. because of a typo in a module name, a file that can't be opened,
341etc.), the Python interpreter prints a nice traceback and exits.
342While the Python interpreter will still do this when your CGI script
343raises an exception, most likely the traceback will end up in one of
344the HTTP server's log file, or be discarded altogether.
345
346Fortunately, once you have managed to get your script to execute
347*some* code, it is easy to catch exceptions and cause a traceback to
348be printed. The \code{test()} function below in this module is an example.
349Here are the rules:
350
351\begin{enumerate}
352 \item Import the traceback module (before entering the
353 try-except!)
354
355 \item Make sure you finish printing the headers and the blank
356 line early
357
358 \item Assign \code{sys.stderr} to \code{sys.stdout}
359
360 \item Wrap all remaining code in a try-except statement
361
362 \item In the except clause, call \code{traceback.print_exc()}
363\end{enumerate}
364
365For example:
366
Guido van Rossume47da0a1997-07-17 16:34:52 +0000367\bcode\begin{verbatim}
368import sys
369import traceback
370print "Content-type: text/html"
371print
372sys.stderr = sys.stdout
373try:
374 ...your code here...
375except:
376 print "\n\n<PRE>"
377 traceback.print_exc()
378\end{verbatim}\ecode
379%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000380Notes: The assignment to \code{sys.stderr} is needed because the traceback
381prints to \code{sys.stderr}. The \code{print "$\backslash$n$\backslash$n<PRE>"} statement is necessary to
382disable the word wrapping in HTML.
383
384If you suspect that there may be a problem in importing the traceback
385module, you can use an even more robust approach (which only uses
386built-in modules):
387
Guido van Rossume47da0a1997-07-17 16:34:52 +0000388\bcode\begin{verbatim}
389import sys
390sys.stderr = sys.stdout
391print "Content-type: text/plain"
392print
393...your code here...
394\end{verbatim}\ecode
395%
Guido van Rossuma29cc971996-07-30 18:22:07 +0000396This relies on the Python interpreter to print the traceback. The
397content type of the output is set to plain text, which disables all
398HTML processing. If your script works, the raw HTML will be displayed
399by your client. If it raises an exception, most likely after the
400first two lines have been printed, a traceback will be displayed.
401Because no HTML interpretation is going on, the traceback will
402readable.
403
404
405\subsection{Common problems and solutions}
Guido van Rossum470be141995-03-17 16:07:09 +0000406
407\begin{itemize}
Guido van Rossuma29cc971996-07-30 18:22:07 +0000408\item Most HTTP servers buffer the output from CGI scripts until the
409script is completed. This means that it is not possible to display a
410progress report on the client's display while the script is running.
411
412\item Check the installation instructions above.
413
414\item Check the HTTP server's log files. (\code{tail -f logfile} in a separate
415window may be useful!)
416
417\item Always check a script for syntax errors first, by doing something
418like \code{python script.py}.
419
420\item When using any of the debugging techniques, don't forget to add
421\code{import sys} to the top of the script.
422
423\item When invoking external programs, make sure they can be found.
424Usually, this means using absolute path names -- \code{\$PATH} is usually not
425set to a very useful value in a CGI script.
426
427\item When reading or writing external files, make sure they can be read
428or written by every user on the system.
429
430\item Don't try to give a CGI script a set-uid mode. This doesn't work on
431most systems, and is a security liability as well.
Guido van Rossum470be141995-03-17 16:07:09 +0000432\end{itemize}
433