blob: 3f2639823ee2a24213a2255f00c9087bad993f72 [file] [log] [blame]
Guido van Rossum470be141995-03-17 16:07:09 +00001\section{Standard Module \sectcode{cgi}}
Guido van Rossuma12ef941995-02-27 17:53:25 +00002\stmodindex{cgi}
3\indexii{WWW}{server}
4\indexii{CGI}{protocol}
5\indexii{HTTP}{protocol}
6\indexii{MIME}{headers}
7\index{URL}
8
Guido van Rossum86751151995-02-28 17:14:32 +00009\renewcommand{\indexsubitem}{(in module cgi)}
10
Guido van Rossuma29cc971996-07-30 18:22:07 +000011Support module for CGI (Common Gateway Interface) scripts.
Guido van Rossuma12ef941995-02-27 17:53:25 +000012
Guido van Rossuma29cc971996-07-30 18:22:07 +000013This module defines a number of utilities for use by CGI scripts
14written in Python.
Guido van Rossuma12ef941995-02-27 17:53:25 +000015
Guido van Rossuma29cc971996-07-30 18:22:07 +000016\subsection{Introduction}
17\nodename{Introduction to the CGI module}
Guido van Rossuma12ef941995-02-27 17:53:25 +000018
Guido van Rossuma29cc971996-07-30 18:22:07 +000019A CGI script is invoked by an HTTP server, usually to process user
20input submitted through an HTML \code{<FORM>} or \code{<ISINPUT>} element.
21
22Most often, CGI scripts live in the server's special \code{cgi-bin}
23directory. The HTTP server places all sorts of information about the
24request (such as the client's hostname, the requested URL, the query
25string, and lots of other goodies) in the script's shell environment,
26executes the script, and sends the script's output back to the client.
27
28The script's input is connected to the client too, and sometimes the
29form data is read this way; at other times the form data is passed via
30the ``query string'' part of the URL. This module (\code{cgi.py}) is intended
31to take care of the different cases and provide a simpler interface to
32the Python script. It also provides a number of utilities that help
33in debugging scripts, and the latest addition is support for file
34uploads from a form (if your browser supports it -- Grail 0.3 and
35Netscape 2.0 do).
36
37The output of a CGI script should consist of two sections, separated
38by a blank line. The first section contains a number of headers,
39telling the client what kind of data is following. Python code to
40generate a minimal header section looks like this:
Guido van Rossuma12ef941995-02-27 17:53:25 +000041
42\begin{verbatim}
Guido van Rossuma29cc971996-07-30 18:22:07 +000043 print "Content-type: text/html" # HTML is following
44 print # blank line, end of headers
Guido van Rossuma12ef941995-02-27 17:53:25 +000045\end{verbatim}
46
Guido van Rossuma29cc971996-07-30 18:22:07 +000047The second section is usually HTML, which allows the client software
48to display nicely formatted text with header, in-line images, etc.
49Here's Python code that prints a simple piece of HTML:
Guido van Rossum470be141995-03-17 16:07:09 +000050
51\begin{verbatim}
Guido van Rossuma29cc971996-07-30 18:22:07 +000052 print "<TITLE>CGI script output</TITLE>"
53 print "<H1>This is my first CGI script</H1>"
54 print "Hello, world!"
Guido van Rossum470be141995-03-17 16:07:09 +000055\end{verbatim}
56
Guido van Rossuma29cc971996-07-30 18:22:07 +000057(It may not be fully legal HTML according to the letter of the
58standard, but any browser will understand it.)
Guido van Rossum470be141995-03-17 16:07:09 +000059
Guido van Rossuma29cc971996-07-30 18:22:07 +000060\subsection{Using the cgi module}
61\nodename{Using the cgi module}
62
63Begin by writing \code{import cgi}. Don't use \code{from cgi import *} -- the
64module defines all sorts of names for its own use or for backward
65compatibility that you don't want in your namespace.
66
67It's best to use the \code{FieldStorage} class. The other classes define in this
68module are provided mostly for backward compatibility. Instantiate it
69exactly once, without arguments. This reads the form contents from
70standard input or the environment (depending on the value of various
71environment variables set according to the CGI standard). Since it may
72consume standard input, it should be instantiated only once.
73
74The \code{FieldStorage} instance can be accessed as if it were a Python
75dictionary. For instance, the following code (which assumes that the
76\code{Content-type} header and blank line have already been printed) checks that
77the fields \code{name} and \code{addr} are both set to a non-empty string:
Guido van Rossum470be141995-03-17 16:07:09 +000078
79\begin{verbatim}
Guido van Rossuma29cc971996-07-30 18:22:07 +000080 form = cgi.FieldStorage()
81 form_ok = 0
82 if form.has_key("name") and form.has_key("addr"):
83 if form["name"].value != "" and form["addr"].value != "":
84 form_ok = 1
85 if not form_ok:
86 print "<H1>Error</H1>"
87 print "Please fill in the name and addr fields."
88 return
89 ...further form processing here...
Guido van Rossum470be141995-03-17 16:07:09 +000090\end{verbatim}
91
Guido van Rossuma29cc971996-07-30 18:22:07 +000092Here the fields, accessed through \code{form[key]}, are themselves instances
93of \code{FieldStorage} (or \code{MiniFieldStorage}, depending on the form encoding).
Guido van Rossum470be141995-03-17 16:07:09 +000094
Guido van Rossuma29cc971996-07-30 18:22:07 +000095If the submitted form data contains more than one field with the same
96name, the object retrieved by \code{form[key]} is not a \code{(Mini)FieldStorage}
97instance but a list of such instances. If you expect this possibility
98(i.e., when your HTML form comtains multiple fields with the same
99name), use the \code{type()} function to determine whether you have a single
100instance or a list of instances. For example, here's code that
101concatenates any number of username fields, separated by commas:
Guido van Rossum470be141995-03-17 16:07:09 +0000102
Guido van Rossuma29cc971996-07-30 18:22:07 +0000103\begin{verbatim}
104 username = form["username"]
105 if type(username) is type([]):
106 # Multiple username fields specified
107 usernames = ""
108 for item in username:
109 if usernames:
110 # Next item -- insert comma
111 usernames = usernames + "," + item.value
112 else:
113 # First item -- don't insert comma
114 usernames = item.value
115 else:
116 # Single username field specified
117 usernames = username.value
118\end{verbatim}
119
120If a field represents an uploaded file, the value attribute reads the
121entire file in memory as a string. This may not be what you want. You can
122test for an uploaded file by testing either the filename attribute or the
123file attribute. You can then read the data at leasure from the file
124attribute:
125
126\begin{verbatim}
127 fileitem = form["userfile"]
128 if fileitem.file:
129 # It's an uploaded file; count lines
130 linecount = 0
131 while 1:
132 line = fileitem.file.readline()
133 if not line: break
134 linecount = linecount + 1
135\end{verbatim}
136
137The file upload draft standard entertains the possibility of uploading
138multiple files from one field (using a recursive \code{multipart/*}
139encoding). When this occurs, the item will be a dictionary-like
140FieldStorage item. This can be determined by testing its type
141attribute, which should have the value \code{multipart/form-data} (or
142perhaps another string beginning with \code{multipart/} It this case, it
143can be iterated over recursively just like the top-level form object.
144
145When a form is submitted in the ``old'' format (as the query string or as a
146single data part of type \code{application/x-www-form-urlencoded}), the items
147will actually be instances of the class \code{MiniFieldStorage}. In this case,
148the list, file and filename attributes are always \code{None}.
149
150
151\subsection{Old classes}
152
153These classes, present in earlier versions of the \code{cgi} module, are still
154supported for backward compatibility. New applications should use the
155
156\code{SvFormContentDict}: single value form content as dictionary; assumes each
157field name occurs in the form only once.
158
159\code{FormContentDict}: multiple value form content as dictionary (the form
160items are lists of values). Useful if your form contains multiple
161fields with the same name.
162
163Other classes (\code{FormContent}, \code{InterpFormContentDict}) are present for
164backwards compatibility with really old applications only. If you still
165use these and would be inconvenienced when they disappeared from a next
166version of this module, drop me a note.
167
168
169\subsection{Functions}
170
171These are useful if you want more control, or if you want to employ
172some of the algorithms implemented in this module in other
173circumstances.
174
175\begin{funcdesc}{parse}{fp}: Parse a query in the environment or from a file (default \code{sys.stdin}).
176\end{funcdesc}
177
178\begin{funcdesc}{parse_qs}{qs}: parse a query string given as a string argument (data of type
179\code{application/x-www-form-urlencoded}).
180\end{funcdesc}
181
182\begin{funcdesc}{parse_multipart}{fp\, pdict}: parse input of type \code{multipart/form-data} (for
183file uploads). Arguments are \code{fp} for the input file and
184 \code{pdict} for the dictionary containing other parameters of \code{content-type} header
185
186 Returns a dictionary just like \code{parse_qs()}: keys are the field names, each
187 value is a list of values for that field. This is easy to use but not
188 much good if you are expecting megabytes to be uploaded -- in that case,
189 use the \code{FieldStorage} class instead which is much more flexible. Note
190 that \code{content-type} is the raw, unparsed contents of the \code{content-type}
191 header.
192
193 Note that this does not parse nested multipart parts -- use \code{FieldStorage} for
194 that.
195\end{funcdesc}
196
197\begin{funcdesc}{parse_header}{string}: parse a header like \code{Content-type} into a main
198content-type and a dictionary of parameters.
199\end{funcdesc}
200
201\begin{funcdesc}{test}{}: robust test CGI script, usable as main program.
202 Writes minimal HTTP headers and formats all information provided to
203 the script in HTML form.
204\end{funcdesc}
205
206\begin{funcdesc}{print_environ}{}: format the shell environment in HTML.
207\end{funcdesc}
208
209\begin{funcdesc}{print_form}{form}: format a form in HTML.
210\end{funcdesc}
211
212\begin{funcdesc}{print_directory}{}: format the current directory in HTML.
213\end{funcdesc}
214
215\begin{funcdesc}{print_environ_usage}{}: print a list of useful (used by CGI) environment variables in
216HTML.
217\end{funcdesc}
218
219\begin{funcdesc}{escape}{}: convert the characters ``\code{\&}'', ``\code{<}'' and ``\code{>}'' to HTML-safe
220sequences. Use this if you need to display text that might contain
221such characters in HTML. To translate URLs for inclusion in the HREF
222attribute of an \code{<A>} tag, use \code{urllib.quote()}.
223\end{funcdesc}
224
225
226\subsection{Caring about security}
227
228There's one important rule: if you invoke an external program (e.g.
229via the \code{os.system()} or \code{os.popen()} functions), make very sure you don't
230pass arbitrary strings received from the client to the shell. This is
231a well-known security hole whereby clever hackers anywhere on the web
232can exploit a gullible CGI script to invoke arbitrary shell commands.
233Even parts of the URL or field names cannot be trusted, since the
234request doesn't have to come from your form!
235
236To be on the safe side, if you must pass a string gotten from a form
237to a shell command, you should make sure the string contains only
238alphanumeric characters, dashes, underscores, and periods.
239
240
241\subsection{Installing your CGI script on a Unix system}
242
243Read the documentation for your HTTP server and check with your local
244system administrator to find the directory where CGI scripts should be
245installed; usually this is in a directory \code{cgi-bin} in the server tree.
246
247Make sure that your script is readable and executable by ``others''; the
248Unix file mode should be 755 (use \code{chmod 755 filename}). Make sure
249that the first line of the script contains \code{\#!} starting in column 1
250followed by the pathname of the Python interpreter, for instance:
251
252\begin{verbatim}
253 #!/usr/local/bin/python
254\end{verbatim}
255
256Make sure the Python interpreter exists and is executable by ``others''.
257
258Make sure that any files your script needs to read or write are
259readable or writable, respectively, by ``others'' -- their mode should
260be 644 for readable and 666 for writable. This is because, for
261security reasons, the HTTP server executes your script as user
262``nobody'', without any special privileges. It can only read (write,
263execute) files that everybody can read (write, execute). The current
264directory at execution time is also different (it is usually the
265server's cgi-bin directory) and the set of environment variables is
266also different from what you get at login. in particular, don't count
267on the shell's search path for executables (\code{\$PATH}) or the Python
268module search path (\code{\$PYTHONPATH}) to be set to anything interesting.
269
270If you need to load modules from a directory which is not on Python's
271default module search path, you can change the path in your script,
272before importing other modules, e.g.:
273
274\begin{verbatim}
275 import sys
276 sys.path.insert(0, "/usr/home/joe/lib/python")
277 sys.path.insert(0, "/usr/local/lib/python")
278\end{verbatim}
279
280(This way, the directory inserted last will be searched first!)
281
282Instructions for non-Unix systems will vary; check your HTTP server's
283documentation (it will usually have a section on CGI scripts).
284
285
286\subsection{Testing your CGI script}
287
288Unfortunately, a CGI script will generally not run when you try it
289from the command line, and a script that works perfectly from the
290command line may fail mysteriously when run from the server. There's
291one reason why you should still test your script from the command
292line: if it contains a syntax error, the python interpreter won't
293execute it at all, and the HTTP server will most likely send a cryptic
294error to the client.
295
296Assuming your script has no syntax errors, yet it does not work, you
297have no choice but to read the next section:
298
299
300\subsection{Debugging CGI scripts}
301
302First of all, check for trivial installation errors -- reading the
303section above on installing your CGI script carefully can save you a
304lot of time. If you wonder whether you have understood the
305installation procedure correctly, try installing a copy of this module
306file (\code{cgi.py}) as a CGI script. When invoked as a script, the file
307will dump its environment and the contents of the form in HTML form.
308Give it the right mode etc, and send it a request. If it's installed
309in the standard \code{cgi-bin} directory, it should be possible to send it a
310request by entering a URL into your browser of the form:
311
312\begin{verbatim}
313 http://yourhostname/cgi-bin/cgi.py?name=Joe+Blow&addr=At+Home
314\end{verbatim}
315
316If this gives an error of type 404, the server cannot find the script
317-- perhaps you need to install it in a different directory. If it
318gives another error (e.g. 500), there's an installation problem that
319you should fix before trying to go any further. If you get a nicely
320formatted listing of the environment and form content (in this
321example, the fields should be listed as ``addr'' with value ``At Home''
322and ``name'' with value ``Joe Blow''), the \code{cgi.py} script has been
323installed correctly. If you follow the same procedure for your own
324script, you should now be able to debug it.
325
326The next step could be to call the \code{cgi} module's test() function from
327your script: replace its main code with the single statement
328
329\begin{verbatim}
330 cgi.test()
331\end{verbatim}
332
333This should produce the same results as those gotten from installing
334the \code{cgi.py} file itself.
335
336When an ordinary Python script raises an unhandled exception
337(e.g. because of a typo in a module name, a file that can't be opened,
338etc.), the Python interpreter prints a nice traceback and exits.
339While the Python interpreter will still do this when your CGI script
340raises an exception, most likely the traceback will end up in one of
341the HTTP server's log file, or be discarded altogether.
342
343Fortunately, once you have managed to get your script to execute
344*some* code, it is easy to catch exceptions and cause a traceback to
345be printed. The \code{test()} function below in this module is an example.
346Here are the rules:
347
348\begin{enumerate}
349 \item Import the traceback module (before entering the
350 try-except!)
351
352 \item Make sure you finish printing the headers and the blank
353 line early
354
355 \item Assign \code{sys.stderr} to \code{sys.stdout}
356
357 \item Wrap all remaining code in a try-except statement
358
359 \item In the except clause, call \code{traceback.print_exc()}
360\end{enumerate}
361
362For example:
363
364\begin{verbatim}
365 import sys
366 import traceback
367 print "Content-type: text/html"
368 print
369 sys.stderr = sys.stdout
370 try:
371 ...your code here...
372 except:
373 print "\n\n<PRE>"
374 traceback.print_exc()
375\end{verbatim}
376
377Notes: The assignment to \code{sys.stderr} is needed because the traceback
378prints to \code{sys.stderr}. The \code{print "$\backslash$n$\backslash$n<PRE>"} statement is necessary to
379disable the word wrapping in HTML.
380
381If you suspect that there may be a problem in importing the traceback
382module, you can use an even more robust approach (which only uses
383built-in modules):
384
385\begin{verbatim}
386 import sys
387 sys.stderr = sys.stdout
388 print "Content-type: text/plain"
389 print
390 ...your code here...
391\end{verbatim}
392
393This relies on the Python interpreter to print the traceback. The
394content type of the output is set to plain text, which disables all
395HTML processing. If your script works, the raw HTML will be displayed
396by your client. If it raises an exception, most likely after the
397first two lines have been printed, a traceback will be displayed.
398Because no HTML interpretation is going on, the traceback will
399readable.
400
401
402\subsection{Common problems and solutions}
Guido van Rossum470be141995-03-17 16:07:09 +0000403
404\begin{itemize}
Guido van Rossuma29cc971996-07-30 18:22:07 +0000405\item Most HTTP servers buffer the output from CGI scripts until the
406script is completed. This means that it is not possible to display a
407progress report on the client's display while the script is running.
408
409\item Check the installation instructions above.
410
411\item Check the HTTP server's log files. (\code{tail -f logfile} in a separate
412window may be useful!)
413
414\item Always check a script for syntax errors first, by doing something
415like \code{python script.py}.
416
417\item When using any of the debugging techniques, don't forget to add
418\code{import sys} to the top of the script.
419
420\item When invoking external programs, make sure they can be found.
421Usually, this means using absolute path names -- \code{\$PATH} is usually not
422set to a very useful value in a CGI script.
423
424\item When reading or writing external files, make sure they can be read
425or written by every user on the system.
426
427\item Don't try to give a CGI script a set-uid mode. This doesn't work on
428most systems, and is a security liability as well.
Guido van Rossum470be141995-03-17 16:07:09 +0000429\end{itemize}
430