Benjamin Peterson | 90f5ba5 | 2010-03-11 22:53:45 +0000 | [diff] [blame] | 1 | #! /usr/bin/env python3 |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 2 | # -*- coding: iso-8859-1 -*- |
Benjamin Peterson | eaedaec | 2013-12-22 19:45:38 -0600 | [diff] [blame] | 3 | # Originally written by Barry Warsaw <barry@python.org> |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 4 | # |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 5 | # Minimally patched to make it even more xgettext compatible |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 6 | # by Peter Funk <pf@artcom-gmbh.de> |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 7 | # |
| 8 | # 2002-11-22 Jürgen Hermann <jh@web.de> |
| 9 | # Added checks that _() only contains string literals, and |
| 10 | # command line args are resolved to module lists, i.e. you |
| 11 | # can now pass a filename, a module or package name, or a |
| 12 | # directory (including globbing chars, important for Win32). |
| 13 | # Made docstring fit in 80 chars wide displays using pydoc. |
| 14 | # |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 15 | |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 16 | # for selftesting |
| 17 | try: |
| 18 | import fintl |
| 19 | _ = fintl.gettext |
| 20 | except ImportError: |
| 21 | _ = lambda s: s |
| 22 | |
| 23 | __doc__ = _("""pygettext -- Python equivalent of xgettext(1) |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 24 | |
| 25 | Many systems (Solaris, Linux, Gnu) provide extensive tools that ease the |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 26 | internationalization of C programs. Most of these tools are independent of |
| 27 | the programming language and can be used from within Python programs. |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 28 | Martin von Loewis' work[1] helps considerably in this regard. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 29 | |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 30 | There's one problem though; xgettext is the program that scans source code |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 31 | looking for message strings, but it groks only C (or C++). Python |
| 32 | introduces a few wrinkles, such as dual quoting characters, triple quoted |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 33 | strings, and raw strings. xgettext understands none of this. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 34 | |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 35 | Enter pygettext, which uses Python's standard tokenize module to scan |
| 36 | Python source code, generating .pot files identical to what GNU xgettext[2] |
| 37 | generates for C and C++ code. From there, the standard GNU tools can be |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 38 | used. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 39 | |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 40 | A word about marking Python strings as candidates for translation. GNU |
| 41 | xgettext recognizes the following keywords: gettext, dgettext, dcgettext, |
| 42 | and gettext_noop. But those can be a lot of text to include all over your |
| 43 | code. C and C++ have a trick: they use the C preprocessor. Most |
| 44 | internationalized C source includes a #define for gettext() to _() so that |
| 45 | what has to be written in the source is much less. Thus these are both |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 46 | translatable strings: |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 47 | |
| 48 | gettext("Translatable String") |
| 49 | _("Translatable String") |
| 50 | |
| 51 | Python of course has no preprocessor so this doesn't work so well. Thus, |
| 52 | pygettext searches only for _() by default, but see the -k/--keyword flag |
| 53 | below for how to augment this. |
| 54 | |
| 55 | [1] http://www.python.org/workshops/1997-10/proceedings/loewis.html |
| 56 | [2] http://www.gnu.org/software/gettext/gettext.html |
| 57 | |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 58 | NOTE: pygettext attempts to be option and feature compatible with GNU |
| 59 | xgettext where ever possible. However some options are still missing or are |
| 60 | not fully implemented. Also, xgettext's use of command line switches with |
| 61 | option arguments is broken, and in these cases, pygettext just defines |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 62 | additional switches. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 63 | |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 64 | Usage: pygettext [options] inputfile ... |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 65 | |
| 66 | Options: |
| 67 | |
| 68 | -a |
| 69 | --extract-all |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 70 | Extract all strings. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 71 | |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 72 | -d name |
| 73 | --default-domain=name |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 74 | Rename the default output file from messages.pot to name.pot. |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 75 | |
| 76 | -E |
| 77 | --escape |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 78 | Replace non-ASCII characters with octal escape sequences. |
| 79 | |
| 80 | -D |
| 81 | --docstrings |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 82 | Extract module, class, method, and function docstrings. These do |
| 83 | not need to be wrapped in _() markers, and in fact cannot be for |
| 84 | Python to consider them docstrings. (See also the -X option). |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 85 | |
| 86 | -h |
| 87 | --help |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 88 | Print this help message and exit. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 89 | |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 90 | -k word |
| 91 | --keyword=word |
| 92 | Keywords to look for in addition to the default set, which are: |
| 93 | %(DEFAULTKEYWORDS)s |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 94 | |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 95 | You can have multiple -k flags on the command line. |
| 96 | |
| 97 | -K |
| 98 | --no-default-keywords |
| 99 | Disable the default set of keywords (see above). Any keywords |
| 100 | explicitly added with the -k/--keyword option are still recognized. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 101 | |
| 102 | --no-location |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 103 | Do not write filename/lineno location comments. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 104 | |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 105 | -n |
| 106 | --add-location |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 107 | Write filename/lineno location comments indicating where each |
| 108 | extracted string is found in the source. These lines appear before |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 109 | each msgid. The style of comments is controlled by the -S/--style |
| 110 | option. This is the default. |
| 111 | |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 112 | -o filename |
| 113 | --output=filename |
| 114 | Rename the default output file from messages.pot to filename. If |
| 115 | filename is `-' then the output is sent to standard out. |
| 116 | |
| 117 | -p dir |
| 118 | --output-dir=dir |
| 119 | Output files will be placed in directory dir. |
| 120 | |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 121 | -S stylename |
| 122 | --style stylename |
| 123 | Specify which style to use for location comments. Two styles are |
| 124 | supported: |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 125 | |
| 126 | Solaris # File: filename, line: line-number |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 127 | GNU #: filename:line |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 128 | |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 129 | The style name is case insensitive. GNU style is the default. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 130 | |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 131 | -v |
| 132 | --verbose |
| 133 | Print the names of the files being processed. |
| 134 | |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 135 | -V |
| 136 | --version |
| 137 | Print the version of pygettext and exit. |
| 138 | |
| 139 | -w columns |
| 140 | --width=columns |
| 141 | Set width of output to columns. |
| 142 | |
| 143 | -x filename |
| 144 | --exclude-file=filename |
| 145 | Specify a file that contains a list of strings that are not be |
| 146 | extracted from the input files. Each string to be excluded must |
| 147 | appear on a line by itself in the file. |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 148 | |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 149 | -X filename |
| 150 | --no-docstrings=filename |
| 151 | Specify a file that contains a list of files (one per line) that |
| 152 | should not have their docstrings extracted. This is only useful in |
| 153 | conjunction with the -D option above. |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 154 | |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 155 | If `inputfile' is -, standard input is read. |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 156 | """) |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 157 | |
| 158 | import os |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 159 | import imp |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 160 | import sys |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 161 | import glob |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 162 | import time |
| 163 | import getopt |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 164 | import token |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 165 | import tokenize |
| 166 | |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 167 | __version__ = '1.5' |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 168 | |
| 169 | default_keywords = ['_'] |
| 170 | DEFAULTKEYWORDS = ', '.join(default_keywords) |
| 171 | |
| 172 | EMPTYSTRING = '' |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 173 | |
| 174 | |
| 175 | |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 176 | # The normal pot-file header. msgmerge and Emacs's po-mode work better if it's |
| 177 | # there. |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 178 | pot_header = _('''\ |
| 179 | # SOME DESCRIPTIVE TITLE. |
| 180 | # Copyright (C) YEAR ORGANIZATION |
| 181 | # FIRST AUTHOR <EMAIL@ADDRESS>, YEAR. |
| 182 | # |
| 183 | msgid "" |
| 184 | msgstr "" |
| 185 | "Project-Id-Version: PACKAGE VERSION\\n" |
Martin v. Löwis | 0f6b383 | 2001-03-01 22:56:17 +0000 | [diff] [blame] | 186 | "POT-Creation-Date: %(time)s\\n" |
| 187 | "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\\n" |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 188 | "Last-Translator: FULL NAME <EMAIL@ADDRESS>\\n" |
| 189 | "Language-Team: LANGUAGE <LL@li.org>\\n" |
| 190 | "MIME-Version: 1.0\\n" |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 191 | "Content-Type: text/plain; charset=%(charset)s\\n" |
| 192 | "Content-Transfer-Encoding: %(encoding)s\\n" |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 193 | "Generated-By: pygettext.py %(version)s\\n" |
| 194 | |
| 195 | ''') |
| 196 | |
| 197 | |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 198 | def usage(code, msg=''): |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 199 | print(__doc__ % globals(), file=sys.stderr) |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 200 | if msg: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 201 | print(msg, file=sys.stderr) |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 202 | sys.exit(code) |
| 203 | |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 204 | |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 205 | |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 206 | def make_escapes(pass_nonascii): |
| 207 | global escapes, escape |
| 208 | if pass_nonascii: |
| 209 | # Allow non-ascii characters to pass through so that e.g. 'msgid |
Barry Warsaw | 7733e12 | 2000-02-27 14:30:48 +0000 | [diff] [blame] | 210 | # "Höhe"' would result not result in 'msgid "H\366he"'. Otherwise we |
| 211 | # escape any character outside the 32..126 range. |
| 212 | mod = 128 |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 213 | escape = escape_ascii |
Barry Warsaw | 7733e12 | 2000-02-27 14:30:48 +0000 | [diff] [blame] | 214 | else: |
| 215 | mod = 256 |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 216 | escape = escape_nonascii |
| 217 | escapes = [r"\%03o" % i for i in range(mod)] |
| 218 | for i in range(32, 127): |
| 219 | escapes[i] = chr(i) |
| 220 | escapes[ord('\\')] = r'\\' |
| 221 | escapes[ord('\t')] = r'\t' |
| 222 | escapes[ord('\r')] = r'\r' |
| 223 | escapes[ord('\n')] = r'\n' |
| 224 | escapes[ord('\"')] = r'\"' |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 225 | |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 226 | |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 227 | def escape_ascii(s, encoding): |
| 228 | return ''.join(escapes[ord(c)] if ord(c) < 128 else c for c in s) |
| 229 | |
| 230 | def escape_nonascii(s, encoding): |
| 231 | return ''.join(escapes[b] for b in s.encode(encoding)) |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 232 | |
| 233 | |
| 234 | def safe_eval(s): |
| 235 | # unwrap quotes, safely |
| 236 | return eval(s, {'__builtins__':{}}, {}) |
| 237 | |
| 238 | |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 239 | def normalize(s, encoding): |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 240 | # This converts the various Python string types into a format that is |
| 241 | # appropriate for .po files, namely much closer to C style. |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 242 | lines = s.split('\n') |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 243 | if len(lines) == 1: |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 244 | s = '"' + escape(s, encoding) + '"' |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 245 | else: |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 246 | if not lines[-1]: |
| 247 | del lines[-1] |
| 248 | lines[-1] = lines[-1] + '\n' |
| 249 | for i in range(len(lines)): |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 250 | lines[i] = escape(lines[i], encoding) |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 251 | lineterm = '\\n"\n"' |
| 252 | s = '""\n"' + lineterm.join(lines) + '"' |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 253 | return s |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 254 | |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 255 | |
| 256 | def containsAny(str, set): |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 257 | """Check whether 'str' contains ANY of the chars in 'set'""" |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 258 | return 1 in [c in str for c in set] |
| 259 | |
| 260 | |
| 261 | def _visit_pyfiles(list, dirname, names): |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 262 | """Helper for getFilesForName().""" |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 263 | # get extension for python source files |
Georg Brandl | bf82e37 | 2008-05-16 17:02:34 +0000 | [diff] [blame] | 264 | if '_py_ext' not in globals(): |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 265 | global _py_ext |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 266 | _py_ext = [triple[0] for triple in imp.get_suffixes() |
| 267 | if triple[2] == imp.PY_SOURCE][0] |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 268 | |
| 269 | # don't recurse into CVS directories |
| 270 | if 'CVS' in names: |
| 271 | names.remove('CVS') |
| 272 | |
| 273 | # add all *.py files to list |
| 274 | list.extend( |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 275 | [os.path.join(dirname, file) for file in names |
| 276 | if os.path.splitext(file)[1] == _py_ext] |
| 277 | ) |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 278 | |
| 279 | |
| 280 | def _get_modpkg_path(dotted_name, pathlist=None): |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 281 | """Get the filesystem path for a module or a package. |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 282 | |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 283 | Return the file system path to a file for a module, and to a directory for |
| 284 | a package. Return None if the name is not found, or is a builtin or |
| 285 | extension module. |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 286 | """ |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 287 | # split off top-most name |
| 288 | parts = dotted_name.split('.', 1) |
| 289 | |
| 290 | if len(parts) > 1: |
| 291 | # we have a dotted path, import top-level package |
| 292 | try: |
| 293 | file, pathname, description = imp.find_module(parts[0], pathlist) |
| 294 | if file: file.close() |
| 295 | except ImportError: |
| 296 | return None |
| 297 | |
| 298 | # check if it's indeed a package |
| 299 | if description[2] == imp.PKG_DIRECTORY: |
| 300 | # recursively handle the remaining name parts |
| 301 | pathname = _get_modpkg_path(parts[1], [pathname]) |
| 302 | else: |
| 303 | pathname = None |
| 304 | else: |
| 305 | # plain name |
| 306 | try: |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 307 | file, pathname, description = imp.find_module( |
| 308 | dotted_name, pathlist) |
| 309 | if file: |
| 310 | file.close() |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 311 | if description[2] not in [imp.PY_SOURCE, imp.PKG_DIRECTORY]: |
| 312 | pathname = None |
| 313 | except ImportError: |
| 314 | pathname = None |
| 315 | |
| 316 | return pathname |
| 317 | |
| 318 | |
| 319 | def getFilesForName(name): |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 320 | """Get a list of module files for a filename, a module or package name, |
| 321 | or a directory. |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 322 | """ |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 323 | if not os.path.exists(name): |
| 324 | # check for glob chars |
| 325 | if containsAny(name, "*?[]"): |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 326 | files = glob.glob(name) |
| 327 | list = [] |
| 328 | for file in files: |
| 329 | list.extend(getFilesForName(file)) |
| 330 | return list |
| 331 | |
| 332 | # try to find module or package |
| 333 | name = _get_modpkg_path(name) |
| 334 | if not name: |
| 335 | return [] |
| 336 | |
| 337 | if os.path.isdir(name): |
| 338 | # find all python files in directory |
| 339 | list = [] |
Alexandre Vassalotti | 4e6531e | 2008-05-09 20:00:17 +0000 | [diff] [blame] | 340 | os.walk(name, _visit_pyfiles, list) |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 341 | return list |
| 342 | elif os.path.exists(name): |
| 343 | # a single file |
| 344 | return [name] |
| 345 | |
| 346 | return [] |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 347 | |
| 348 | |
| 349 | class TokenEater: |
| 350 | def __init__(self, options): |
| 351 | self.__options = options |
| 352 | self.__messages = {} |
| 353 | self.__state = self.__waiting |
| 354 | self.__data = [] |
| 355 | self.__lineno = -1 |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 356 | self.__freshmodule = 1 |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 357 | self.__curfile = None |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 358 | |
| 359 | def __call__(self, ttype, tstring, stup, etup, line): |
| 360 | # dispatch |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 361 | ## import token |
| 362 | ## print >> sys.stderr, 'ttype:', token.tok_name[ttype], \ |
| 363 | ## 'tstring:', tstring |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 364 | self.__state(ttype, tstring, stup[0]) |
| 365 | |
| 366 | def __waiting(self, ttype, tstring, lineno): |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 367 | opts = self.__options |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 368 | # Do docstring extractions, if enabled |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 369 | if opts.docstrings and not opts.nodocstrings.get(self.__curfile): |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 370 | # module docstring? |
| 371 | if self.__freshmodule: |
| 372 | if ttype == tokenize.STRING: |
Barry Warsaw | 16b62c1 | 2001-05-21 19:51:26 +0000 | [diff] [blame] | 373 | self.__addentry(safe_eval(tstring), lineno, isdocstring=1) |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 374 | self.__freshmodule = 0 |
| 375 | elif ttype not in (tokenize.COMMENT, tokenize.NL): |
| 376 | self.__freshmodule = 0 |
| 377 | return |
| 378 | # class docstring? |
| 379 | if ttype == tokenize.NAME and tstring in ('class', 'def'): |
| 380 | self.__state = self.__suiteseen |
| 381 | return |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 382 | if ttype == tokenize.NAME and tstring in opts.keywords: |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 383 | self.__state = self.__keywordseen |
| 384 | |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 385 | def __suiteseen(self, ttype, tstring, lineno): |
| 386 | # ignore anything until we see the colon |
| 387 | if ttype == tokenize.OP and tstring == ':': |
| 388 | self.__state = self.__suitedocstring |
| 389 | |
| 390 | def __suitedocstring(self, ttype, tstring, lineno): |
| 391 | # ignore any intervening noise |
| 392 | if ttype == tokenize.STRING: |
Barry Warsaw | 16b62c1 | 2001-05-21 19:51:26 +0000 | [diff] [blame] | 393 | self.__addentry(safe_eval(tstring), lineno, isdocstring=1) |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 394 | self.__state = self.__waiting |
| 395 | elif ttype not in (tokenize.NEWLINE, tokenize.INDENT, |
| 396 | tokenize.COMMENT): |
| 397 | # there was no class docstring |
| 398 | self.__state = self.__waiting |
| 399 | |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 400 | def __keywordseen(self, ttype, tstring, lineno): |
| 401 | if ttype == tokenize.OP and tstring == '(': |
| 402 | self.__data = [] |
| 403 | self.__lineno = lineno |
| 404 | self.__state = self.__openseen |
| 405 | else: |
| 406 | self.__state = self.__waiting |
| 407 | |
| 408 | def __openseen(self, ttype, tstring, lineno): |
| 409 | if ttype == tokenize.OP and tstring == ')': |
| 410 | # We've seen the last of the translatable strings. Record the |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 411 | # line number of the first line of the strings and update the list |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 412 | # of messages seen. Reset state for the next batch. If there |
| 413 | # were no strings inside _(), then just ignore this entry. |
| 414 | if self.__data: |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 415 | self.__addentry(EMPTYSTRING.join(self.__data)) |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 416 | self.__state = self.__waiting |
| 417 | elif ttype == tokenize.STRING: |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 418 | self.__data.append(safe_eval(tstring)) |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 419 | elif ttype not in [tokenize.COMMENT, token.INDENT, token.DEDENT, |
| 420 | token.NEWLINE, tokenize.NL]: |
| 421 | # warn if we see anything else than STRING or whitespace |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 422 | print(_( |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 423 | '*** %(file)s:%(lineno)s: Seen unexpected token "%(token)s"' |
| 424 | ) % { |
| 425 | 'token': tstring, |
| 426 | 'file': self.__curfile, |
| 427 | 'lineno': self.__lineno |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 428 | }, file=sys.stderr) |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 429 | self.__state = self.__waiting |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 430 | |
Barry Warsaw | 16b62c1 | 2001-05-21 19:51:26 +0000 | [diff] [blame] | 431 | def __addentry(self, msg, lineno=None, isdocstring=0): |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 432 | if lineno is None: |
| 433 | lineno = self.__lineno |
| 434 | if not msg in self.__options.toexclude: |
| 435 | entry = (self.__curfile, lineno) |
Barry Warsaw | 16b62c1 | 2001-05-21 19:51:26 +0000 | [diff] [blame] | 436 | self.__messages.setdefault(msg, {})[entry] = isdocstring |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 437 | |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 438 | def set_filename(self, filename): |
| 439 | self.__curfile = filename |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 440 | self.__freshmodule = 1 |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 441 | |
| 442 | def write(self, fp): |
| 443 | options = self.__options |
R David Murray | 2b78129 | 2015-04-16 12:15:09 -0400 | [diff] [blame] | 444 | timestamp = time.strftime('%Y-%m-%d %H:%M%z') |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 445 | encoding = fp.encoding if fp.encoding else 'UTF-8' |
| 446 | print(pot_header % {'time': timestamp, 'version': __version__, |
| 447 | 'charset': encoding, |
| 448 | 'encoding': '8bit'}, file=fp) |
Barry Warsaw | 128c77d | 2001-05-23 16:59:45 +0000 | [diff] [blame] | 449 | # Sort the entries. First sort each particular entry's keys, then |
| 450 | # sort all the entries by their first item. |
| 451 | reverse = {} |
Fred Drake | 33e2c3e | 2000-10-26 03:49:15 +0000 | [diff] [blame] | 452 | for k, v in self.__messages.items(): |
Guido van Rossum | f7bd964 | 2008-01-15 17:41:38 +0000 | [diff] [blame] | 453 | keys = sorted(v.keys()) |
Barry Warsaw | 50cf706 | 2001-05-24 23:06:13 +0000 | [diff] [blame] | 454 | reverse.setdefault(tuple(keys), []).append((k, v)) |
Guido van Rossum | f7bd964 | 2008-01-15 17:41:38 +0000 | [diff] [blame] | 455 | rkeys = sorted(reverse.keys()) |
Barry Warsaw | 128c77d | 2001-05-23 16:59:45 +0000 | [diff] [blame] | 456 | for rkey in rkeys: |
Barry Warsaw | 50cf706 | 2001-05-24 23:06:13 +0000 | [diff] [blame] | 457 | rentries = reverse[rkey] |
| 458 | rentries.sort() |
| 459 | for k, v in rentries: |
| 460 | # If the entry was gleaned out of a docstring, then add a |
| 461 | # comment stating so. This is to aid translators who may wish |
| 462 | # to skip translating some unimportant docstrings. |
Guido van Rossum | 89da5d7 | 2006-08-22 00:21:25 +0000 | [diff] [blame] | 463 | isdocstring = any(v.values()) |
Barry Warsaw | 50cf706 | 2001-05-24 23:06:13 +0000 | [diff] [blame] | 464 | # k is the message string, v is a dictionary-set of (filename, |
| 465 | # lineno) tuples. We want to sort the entries in v first by |
| 466 | # file name and then by line number. |
Guido van Rossum | f7bd964 | 2008-01-15 17:41:38 +0000 | [diff] [blame] | 467 | v = sorted(v.keys()) |
Barry Warsaw | 50cf706 | 2001-05-24 23:06:13 +0000 | [diff] [blame] | 468 | if not options.writelocations: |
| 469 | pass |
| 470 | # location comments are different b/w Solaris and GNU: |
| 471 | elif options.locationstyle == options.SOLARIS: |
| 472 | for filename, lineno in v: |
| 473 | d = {'filename': filename, 'lineno': lineno} |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 474 | print(_( |
| 475 | '# File: %(filename)s, line: %(lineno)d') % d, file=fp) |
Barry Warsaw | 50cf706 | 2001-05-24 23:06:13 +0000 | [diff] [blame] | 476 | elif options.locationstyle == options.GNU: |
| 477 | # fit as many locations on one line, as long as the |
| 478 | # resulting line length doesn't exceeds 'options.width' |
| 479 | locline = '#:' |
| 480 | for filename, lineno in v: |
| 481 | d = {'filename': filename, 'lineno': lineno} |
| 482 | s = _(' %(filename)s:%(lineno)d') % d |
| 483 | if len(locline) + len(s) <= options.width: |
| 484 | locline = locline + s |
| 485 | else: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 486 | print(locline, file=fp) |
Barry Warsaw | 50cf706 | 2001-05-24 23:06:13 +0000 | [diff] [blame] | 487 | locline = "#:" + s |
| 488 | if len(locline) > 2: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 489 | print(locline, file=fp) |
Barry Warsaw | 5c94ce5 | 2001-06-20 19:41:40 +0000 | [diff] [blame] | 490 | if isdocstring: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 491 | print('#, docstring', file=fp) |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 492 | print('msgid', normalize(k, encoding), file=fp) |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 493 | print('msgstr ""\n', file=fp) |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 494 | |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 495 | |
| 496 | |
| 497 | def main(): |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 498 | global default_keywords |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 499 | try: |
| 500 | opts, args = getopt.getopt( |
| 501 | sys.argv[1:], |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 502 | 'ad:DEhk:Kno:p:S:Vvw:x:X:', |
Barry Warsaw | 2b63969 | 2001-05-21 19:58:23 +0000 | [diff] [blame] | 503 | ['extract-all', 'default-domain=', 'escape', 'help', |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 504 | 'keyword=', 'no-default-keywords', |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 505 | 'add-location', 'no-location', 'output=', 'output-dir=', |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 506 | 'style=', 'verbose', 'version', 'width=', 'exclude-file=', |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 507 | 'docstrings', 'no-docstrings', |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 508 | ]) |
Guido van Rossum | b940e11 | 2007-01-10 16:19:56 +0000 | [diff] [blame] | 509 | except getopt.error as msg: |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 510 | usage(1, msg) |
| 511 | |
| 512 | # for holding option values |
| 513 | class Options: |
| 514 | # constants |
| 515 | GNU = 1 |
| 516 | SOLARIS = 2 |
| 517 | # defaults |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 518 | extractall = 0 # FIXME: currently this option has no effect at all. |
| 519 | escape = 0 |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 520 | keywords = [] |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 521 | outpath = '' |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 522 | outfile = 'messages.pot' |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 523 | writelocations = 1 |
| 524 | locationstyle = GNU |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 525 | verbose = 0 |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 526 | width = 78 |
| 527 | excludefilename = '' |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 528 | docstrings = 0 |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 529 | nodocstrings = {} |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 530 | |
| 531 | options = Options() |
| 532 | locations = {'gnu' : options.GNU, |
| 533 | 'solaris' : options.SOLARIS, |
| 534 | } |
| 535 | |
| 536 | # parse options |
| 537 | for opt, arg in opts: |
| 538 | if opt in ('-h', '--help'): |
| 539 | usage(0) |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 540 | elif opt in ('-a', '--extract-all'): |
| 541 | options.extractall = 1 |
| 542 | elif opt in ('-d', '--default-domain'): |
| 543 | options.outfile = arg + '.pot' |
| 544 | elif opt in ('-E', '--escape'): |
| 545 | options.escape = 1 |
Barry Warsaw | 08a8a35 | 2000-10-27 04:56:28 +0000 | [diff] [blame] | 546 | elif opt in ('-D', '--docstrings'): |
| 547 | options.docstrings = 1 |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 548 | elif opt in ('-k', '--keyword'): |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 549 | options.keywords.append(arg) |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 550 | elif opt in ('-K', '--no-default-keywords'): |
| 551 | default_keywords = [] |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 552 | elif opt in ('-n', '--add-location'): |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 553 | options.writelocations = 1 |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 554 | elif opt in ('--no-location',): |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 555 | options.writelocations = 0 |
| 556 | elif opt in ('-S', '--style'): |
| 557 | options.locationstyle = locations.get(arg.lower()) |
| 558 | if options.locationstyle is None: |
| 559 | usage(1, _('Invalid value for --style: %s') % arg) |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 560 | elif opt in ('-o', '--output'): |
| 561 | options.outfile = arg |
| 562 | elif opt in ('-p', '--output-dir'): |
| 563 | options.outpath = arg |
Barry Warsaw | 5dbf526 | 1999-11-03 18:47:52 +0000 | [diff] [blame] | 564 | elif opt in ('-v', '--verbose'): |
| 565 | options.verbose = 1 |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 566 | elif opt in ('-V', '--version'): |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 567 | print(_('pygettext.py (xgettext for Python) %s') % __version__) |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 568 | sys.exit(0) |
| 569 | elif opt in ('-w', '--width'): |
| 570 | try: |
| 571 | options.width = int(arg) |
| 572 | except ValueError: |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 573 | usage(1, _('--width argument must be an integer: %s') % arg) |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 574 | elif opt in ('-x', '--exclude-file'): |
| 575 | options.excludefilename = arg |
Barry Warsaw | 63ce5af | 2001-07-27 16:47:18 +0000 | [diff] [blame] | 576 | elif opt in ('-X', '--no-docstrings'): |
| 577 | fp = open(arg) |
| 578 | try: |
| 579 | while 1: |
| 580 | line = fp.readline() |
| 581 | if not line: |
| 582 | break |
| 583 | options.nodocstrings[line[:-1]] = 1 |
| 584 | finally: |
| 585 | fp.close() |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 586 | |
| 587 | # calculate escapes |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 588 | make_escapes(not options.escape) |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 589 | |
| 590 | # calculate all keywords |
| 591 | options.keywords.extend(default_keywords) |
| 592 | |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 593 | # initialize list of strings to exclude |
| 594 | if options.excludefilename: |
| 595 | try: |
| 596 | fp = open(options.excludefilename) |
| 597 | options.toexclude = fp.readlines() |
| 598 | fp.close() |
| 599 | except IOError: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 600 | print(_( |
| 601 | "Can't read --exclude-file: %s") % options.excludefilename, file=sys.stderr) |
Barry Warsaw | c8f0892 | 2000-02-26 20:56:47 +0000 | [diff] [blame] | 602 | sys.exit(1) |
| 603 | else: |
| 604 | options.toexclude = [] |
| 605 | |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 606 | # resolve args to module lists |
| 607 | expanded = [] |
| 608 | for arg in args: |
| 609 | if arg == '-': |
| 610 | expanded.append(arg) |
| 611 | else: |
| 612 | expanded.extend(getFilesForName(arg)) |
| 613 | args = expanded |
| 614 | |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 615 | # slurp through all the files |
| 616 | eater = TokenEater(options) |
| 617 | for filename in args: |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 618 | if filename == '-': |
| 619 | if options.verbose: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 620 | print(_('Reading standard input')) |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 621 | fp = sys.stdin.buffer |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 622 | closep = 0 |
| 623 | else: |
| 624 | if options.verbose: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 625 | print(_('Working on %s') % filename) |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 626 | fp = open(filename, 'rb') |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 627 | closep = 1 |
| 628 | try: |
| 629 | eater.set_filename(filename) |
Barry Warsaw | 75ee8f5 | 2001-02-26 04:46:53 +0000 | [diff] [blame] | 630 | try: |
Serhiy Storchaka | b6ed173 | 2013-02-09 22:37:22 +0200 | [diff] [blame] | 631 | tokens = tokenize.tokenize(fp.readline) |
Trent Nelson | 428de65 | 2008-03-18 22:41:35 +0000 | [diff] [blame] | 632 | for _token in tokens: |
| 633 | eater(*_token) |
Guido van Rossum | b940e11 | 2007-01-10 16:19:56 +0000 | [diff] [blame] | 634 | except tokenize.TokenError as e: |
Collin Winter | 6afaeb7 | 2007-08-03 17:06:41 +0000 | [diff] [blame] | 635 | print('%s: %s, line %d, column %d' % ( |
Georg Brandl | 6464d47 | 2007-10-22 16:16:13 +0000 | [diff] [blame] | 636 | e.args[0], filename, e.args[1][0], e.args[1][1]), |
| 637 | file=sys.stderr) |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 638 | finally: |
| 639 | if closep: |
| 640 | fp.close() |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 641 | |
Barry Warsaw | a17e0f1 | 2000-03-08 15:18:35 +0000 | [diff] [blame] | 642 | # write the output |
| 643 | if options.outfile == '-': |
| 644 | fp = sys.stdout |
| 645 | closep = 0 |
| 646 | else: |
| 647 | if options.outpath: |
| 648 | options.outfile = os.path.join(options.outpath, options.outfile) |
| 649 | fp = open(options.outfile, 'w') |
| 650 | closep = 1 |
| 651 | try: |
| 652 | eater.write(fp) |
| 653 | finally: |
| 654 | if closep: |
| 655 | fp.close() |
Barry Warsaw | e27db5a | 1999-08-13 20:59:48 +0000 | [diff] [blame] | 656 | |
| 657 | |
| 658 | if __name__ == '__main__': |
| 659 | main() |
Barry Warsaw | 75a6e67 | 2000-05-02 19:28:30 +0000 | [diff] [blame] | 660 | # some more test strings |
Barry Warsaw | e04ee70 | 2003-04-16 18:08:23 +0000 | [diff] [blame] | 661 | # this one creates a warning |
| 662 | _('*** Seen unexpected token "%(token)s"') % {'token': 'test'} |
Martin v. Löwis | 0d1fdea | 2002-11-22 08:36:54 +0000 | [diff] [blame] | 663 | _('more' 'than' 'one' 'string') |