blob: 63a4a70739251361dcbd74008c022d7ba86bcd2a [file] [log] [blame]
John Thompsond977c1e2013-03-27 18:34:38 +00001//===- extra/modularize/Modularize.cpp - Check modularized headers --------===//
John Thompson4f8ba652013-03-12 02:07:30 +00002//
3// The LLVM Compiler Infrastructure
4//
5// This file is distributed under the University of Illinois Open Source
6// License. See LICENSE.TXT for details.
7//
8//===----------------------------------------------------------------------===//
9//
10// This file implements a tool that checks whether a set of headers provides
11// the consistent definitions required to use modules. For example, it detects
12// whether the same entity (say, a NULL macro or size_t typedef) is defined in
13// multiple headers or whether a header produces different definitions under
14// different circumstances. These conditions cause modules built from the
John Thompsonf5db45b2013-03-27 01:02:46 +000015// headers to behave poorly, and should be fixed before introducing a module
John Thompson4f8ba652013-03-12 02:07:30 +000016// map.
17//
18// Modularize takes as argument a file name for a file containing the
19// newline-separated list of headers to check with respect to each other.
John Thompsonf5db45b2013-03-27 01:02:46 +000020// Lines beginning with '#' and empty lines are ignored.
John Thompson74751802013-09-03 18:48:43 +000021// Header file names followed by a colon and other space-separated
22// file names will include those extra files as dependencies.
23// The file names can be relative or full paths, but must be on the
24// same line.
25//
John Thompson4f8ba652013-03-12 02:07:30 +000026// Modularize also accepts regular front-end arguments.
27//
John Thompsonf5db45b2013-03-27 01:02:46 +000028// Usage: modularize [-prefix (optional header path prefix)]
John Thompsona2de1082013-03-26 01:17:48 +000029// (include-files_list) [(front-end-options) ...]
30//
John Thompsona44f85a2013-04-15 22:32:28 +000031// Note that unless a "-prefix (header path)" option is specified,
John Thompsona2de1082013-03-26 01:17:48 +000032// non-absolute file paths in the header list file will be relative
33// to the header list file directory. Use -prefix to specify a different
34// directory.
John Thompson4f8ba652013-03-12 02:07:30 +000035//
John Thompsonfd8ca382013-03-27 19:31:22 +000036// Note that by default, the underlying Clang front end assumes .h files
37// contain C source. If your .h files in the file list contain C++ source,
John Thompsonea6c8db2013-03-27 21:23:21 +000038// you should append the following to your command lines: -x c++
John Thompsonfd8ca382013-03-27 19:31:22 +000039//
John Thompson4f8ba652013-03-12 02:07:30 +000040// Modularize will do normal parsing, reporting normal errors and warnings,
41// but will also report special error messages like the following:
42//
John Thompson7c6e79f32013-07-29 19:07:00 +000043// error: '(symbol)' defined at multiple locations:
44// (file):(row):(column)
45// (file):(row):(column)
John Thompson4f8ba652013-03-12 02:07:30 +000046//
John Thompsondc118272013-07-29 21:59:41 +000047// error: header '(file)' has different contents depending on how it was
John Thompson7c6e79f32013-07-29 19:07:00 +000048// included
John Thompson4f8ba652013-03-12 02:07:30 +000049//
50// The latter might be followed by messages like the following:
51//
John Thompson7c6e79f32013-07-29 19:07:00 +000052// note: '(symbol)' in (file) at (row):(column) not always provided
John Thompson4f8ba652013-03-12 02:07:30 +000053//
John Thompson7c6e79f32013-07-29 19:07:00 +000054// Checks will also be performed for macro expansions, defined(macro)
55// expressions, and preprocessor conditional directives that evaluate
56// inconsistently, and can produce error messages like the following:
57//
Nico Weber8e20be22013-08-12 11:43:36 +000058// (...)/SubHeader.h:11:5:
59// #if SYMBOL == 1
60// ^
61// error: Macro instance 'SYMBOL' has different values in this header,
62// depending on how it was included.
63// 'SYMBOL' expanded to: '1' with respect to these inclusion paths:
64// (...)/Header1.h
65// (...)/SubHeader.h
66// (...)/SubHeader.h:3:9:
67// #define SYMBOL 1
68// ^
69// Macro defined here.
70// 'SYMBOL' expanded to: '2' with respect to these inclusion paths:
71// (...)/Header2.h
72// (...)/SubHeader.h
73// (...)/SubHeader.h:7:9:
74// #define SYMBOL 2
75// ^
76// Macro defined here.
77//
John Thompson4fa9c2c2013-08-09 00:19:03 +000078// See PreprocessorTracker.cpp for additional details.
John Thompsoncc2e2912013-09-03 18:44:11 +000079//
John Thompson74751802013-09-03 18:48:43 +000080// Current problems:
John Thompsonce601e22013-03-14 01:41:29 +000081//
John Thompson74751802013-09-03 18:48:43 +000082// Modularize has problems with C++:
John Thompsonce601e22013-03-14 01:41:29 +000083//
John Thompson74751802013-09-03 18:48:43 +000084// 1. Modularize doesn't distinguish class of the same name in
85// different namespaces. The result is erroneous duplicate definition errors.
John Thompsonce601e22013-03-14 01:41:29 +000086//
John Thompson74751802013-09-03 18:48:43 +000087// 2. Modularize doesn't distinguish between a regular class and a template
88// class of the same name.
John Thompson181ea2e2013-08-08 00:00:10 +000089//
John Thompson74751802013-09-03 18:48:43 +000090// Other problems:
John Thompsonce601e22013-03-14 01:41:29 +000091//
John Thompson74751802013-09-03 18:48:43 +000092// 3. There seem to be a lot of spurious "not always provided" messages,
93// and many duplicates of these.
John Thompsonce601e22013-03-14 01:41:29 +000094//
John Thompson74751802013-09-03 18:48:43 +000095// 4. There are some legitimate uses of preprocessor macros that
John Thompson9e8d6722013-08-26 15:55:47 +000096// modularize will flag as errors, such as repeatedly #include'ing
97// a file and using interleaving defined/undefined macros
98// to change declarations in the included file. Is there a way
99// to address this? Maybe have modularize accept a list of macros
100// to ignore. Otherwise you can just exclude the file, after checking
101// for legitimate errors.
John Thompson4fa9c2c2013-08-09 00:19:03 +0000102//
John Thompson74751802013-09-03 18:48:43 +0000103// Future directions:
104//
105// Basically, we want to add new checks for whatever we can check with respect
106// to checking headers for module'ability.
107//
108// Some ideas:
109//
110// 1. Fix the C++ and other problems.
111//
112// 2. Add options to disable any of the checks, in case
113// there is some problem with them, or the messages get too verbose.
114//
115// 3. Try to figure out the preprocessor conditional directives that
116// contribute to problems and tie them to the inconsistent definitions.
117//
118// 4. Check for correct and consistent usage of extern "C" {} and other
119// directives. Warn about #include inside extern "C" {}.
120//
121// 5. What else?
John Thompsonce601e22013-03-14 01:41:29 +0000122//
123// General clean-up and refactoring:
124//
125// 1. The Location class seems to be something that we might
126// want to design to be applicable to a wider range of tools, and stick it
127// somewhere into Tooling/ in mainline
128//
John Thompson4f8ba652013-03-12 02:07:30 +0000129//===----------------------------------------------------------------------===//
John Thompsonf5db45b2013-03-27 01:02:46 +0000130
John Thompsond977c1e2013-03-27 18:34:38 +0000131#include "clang/AST/ASTConsumer.h"
132#include "clang/AST/ASTContext.h"
133#include "clang/AST/RecursiveASTVisitor.h"
134#include "clang/Basic/SourceManager.h"
John Thompson74751802013-09-03 18:48:43 +0000135#include "clang/Driver/Options.h"
John Thompsond977c1e2013-03-27 18:34:38 +0000136#include "clang/Frontend/CompilerInstance.h"
137#include "clang/Frontend/FrontendActions.h"
138#include "clang/Lex/Preprocessor.h"
139#include "clang/Tooling/CompilationDatabase.h"
140#include "clang/Tooling/Tooling.h"
141#include "llvm/ADT/OwningPtr.h"
142#include "llvm/ADT/StringRef.h"
John Thompson74751802013-09-03 18:48:43 +0000143#include "llvm/ADT/StringMap.h"
John Thompson4f8ba652013-03-12 02:07:30 +0000144#include "llvm/Config/config.h"
John Thompson74751802013-09-03 18:48:43 +0000145#include "llvm/Option/Arg.h"
146#include "llvm/Option/ArgList.h"
147#include "llvm/Option/OptTable.h"
148#include "llvm/Option/Option.h"
John Thompsona2de1082013-03-26 01:17:48 +0000149#include "llvm/Support/CommandLine.h"
John Thompson4f8ba652013-03-12 02:07:30 +0000150#include "llvm/Support/FileSystem.h"
John Thompsonf5db45b2013-03-27 01:02:46 +0000151#include "llvm/Support/MemoryBuffer.h"
John Thompsona2de1082013-03-26 01:17:48 +0000152#include "llvm/Support/Path.h"
John Thompson4f8ba652013-03-12 02:07:30 +0000153#include <algorithm>
John Thompsond977c1e2013-03-27 18:34:38 +0000154#include <fstream>
John Thompson4f8ba652013-03-12 02:07:30 +0000155#include <iterator>
John Thompsond977c1e2013-03-27 18:34:38 +0000156#include <string>
157#include <vector>
John Thompson94faa4d2013-07-26 23:56:42 +0000158#include "PreprocessorTracker.h"
John Thompson4f8ba652013-03-12 02:07:30 +0000159
John Thompson4f8ba652013-03-12 02:07:30 +0000160using namespace clang;
John Thompson74751802013-09-03 18:48:43 +0000161using namespace clang::driver;
162using namespace clang::driver::options;
163using namespace clang::tooling;
John Thompsona2de1082013-03-26 01:17:48 +0000164using namespace llvm;
John Thompson74751802013-09-03 18:48:43 +0000165using namespace llvm::opt;
John Thompson94faa4d2013-07-26 23:56:42 +0000166using namespace Modularize;
John Thompson4f8ba652013-03-12 02:07:30 +0000167
John Thompsonea6c8db2013-03-27 21:23:21 +0000168// Option to specify a file name for a list of header files to check.
John Thompsonb809dfc2013-07-19 14:19:31 +0000169cl::opt<std::string>
170ListFileName(cl::Positional,
171 cl::desc("<name of file containing list of headers to check>"));
John Thompsonea6c8db2013-03-27 21:23:21 +0000172
173// Collect all other arguments, which will be passed to the front end.
John Thompson161381e2013-06-27 18:52:23 +0000174cl::list<std::string>
John Thompsonb809dfc2013-07-19 14:19:31 +0000175CC1Arguments(cl::ConsumeAfter,
176 cl::desc("<arguments to be passed to front end>..."));
John Thompsonea6c8db2013-03-27 21:23:21 +0000177
178// Option to specify a prefix to be prepended to the header names.
179cl::opt<std::string> HeaderPrefix(
180 "prefix", cl::init(""),
181 cl::desc(
182 "Prepend header file paths with this prefix."
183 " If not specified,"
184 " the files are considered to be relative to the header list file."));
185
John Thompson74751802013-09-03 18:48:43 +0000186typedef SmallVector<std::string, 4> DependentsVector;
187typedef StringMap<DependentsVector> DependencyMap;
John Thompsonea6c8db2013-03-27 21:23:21 +0000188
John Thompson74751802013-09-03 18:48:43 +0000189// Read the header list file and collect the header file names and
190// optional dependencies.
191error_code getHeaderFileNames(SmallVectorImpl<std::string> &HeaderFileNames,
192 DependencyMap &Dependencies,
193 StringRef ListFileName, StringRef HeaderPrefix) {
John Thompsonea6c8db2013-03-27 21:23:21 +0000194 // By default, use the path component of the list file name.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000195 SmallString<256> HeaderDirectory(ListFileName);
196 sys::path::remove_filename(HeaderDirectory);
John Thompson74751802013-09-03 18:48:43 +0000197 SmallString<256> CurrentDirectory;
198 sys::fs::current_path(CurrentDirectory);
John Thompsonea6c8db2013-03-27 21:23:21 +0000199
200 // Get the prefix if we have one.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000201 if (HeaderPrefix.size() != 0)
202 HeaderDirectory = HeaderPrefix;
John Thompsonea6c8db2013-03-27 21:23:21 +0000203
204 // Read the header list file into a buffer.
205 OwningPtr<MemoryBuffer> listBuffer;
John Thompsonbb0a3b02013-08-09 13:52:09 +0000206 if (error_code ec = MemoryBuffer::getFile(ListFileName, listBuffer)) {
John Thompsonea6c8db2013-03-27 21:23:21 +0000207 return ec;
208 }
209
210 // Parse the header list into strings.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000211 SmallVector<StringRef, 32> Strings;
212 listBuffer->getBuffer().split(Strings, "\n", -1, false);
John Thompsonea6c8db2013-03-27 21:23:21 +0000213
214 // Collect the header file names from the string list.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000215 for (SmallVectorImpl<StringRef>::iterator I = Strings.begin(),
216 E = Strings.end();
John Thompsonea6c8db2013-03-27 21:23:21 +0000217 I != E; ++I) {
John Thompson74751802013-09-03 18:48:43 +0000218 StringRef Line = I->trim();
John Thompsonea6c8db2013-03-27 21:23:21 +0000219 // Ignore comments and empty lines.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000220 if (Line.empty() || (Line[0] == '#'))
John Thompsonea6c8db2013-03-27 21:23:21 +0000221 continue;
John Thompson74751802013-09-03 18:48:43 +0000222 std::pair<StringRef, StringRef> TargetAndDependents = Line.split(':');
John Thompsonbb0a3b02013-08-09 13:52:09 +0000223 SmallString<256> HeaderFileName;
John Thompsonea6c8db2013-03-27 21:23:21 +0000224 // Prepend header file name prefix if it's not absolute.
John Thompson74751802013-09-03 18:48:43 +0000225 if (sys::path::is_absolute(TargetAndDependents.first))
226 llvm::sys::path::native(TargetAndDependents.first, HeaderFileName);
John Thompsonea6c8db2013-03-27 21:23:21 +0000227 else {
John Thompson74751802013-09-03 18:48:43 +0000228 if (HeaderDirectory.size() != 0)
229 HeaderFileName = HeaderDirectory;
230 else
231 HeaderFileName = CurrentDirectory;
232 sys::path::append(HeaderFileName, TargetAndDependents.first);
233 llvm::sys::path::native(HeaderFileName.str(), HeaderFileName);
John Thompsonea6c8db2013-03-27 21:23:21 +0000234 }
John Thompson74751802013-09-03 18:48:43 +0000235 // Handle optional dependencies.
236 DependentsVector Dependents;
237 SmallVector<StringRef, 4> DependentsList;
238 TargetAndDependents.second.split(DependentsList, " ", -1, false);
239 int Count = DependentsList.size();
240 for (int Index = 0; Index < Count; ++Index) {
241 SmallString<256> Dependent;
242 if (sys::path::is_absolute(DependentsList[Index]))
243 Dependent = DependentsList[Index];
244 else {
245 if (HeaderDirectory.size() != 0)
246 Dependent = HeaderDirectory;
247 else
248 Dependent = CurrentDirectory;
249 sys::path::append(Dependent, DependentsList[Index]);
250 }
251 llvm::sys::path::native(Dependent.str(), Dependent);
252 Dependents.push_back(Dependent.str());
253 }
254 // Save the resulting header file path and dependencies.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000255 HeaderFileNames.push_back(HeaderFileName.str());
John Thompson74751802013-09-03 18:48:43 +0000256 Dependencies[HeaderFileName.str()] = Dependents;
John Thompsonea6c8db2013-03-27 21:23:21 +0000257 }
258
259 return error_code::success();
260}
261
John Thompson74751802013-09-03 18:48:43 +0000262// Helper function for finding the input file in an arguments list.
263llvm::StringRef findInputFile(const CommandLineArguments &CLArgs) {
264 OwningPtr<OptTable> Opts(createDriverOptTable());
265 const unsigned IncludedFlagsBitmask = options::CC1Option;
266 unsigned MissingArgIndex, MissingArgCount;
267 SmallVector<const char *, 256> Argv;
268 for (CommandLineArguments::const_iterator I = CLArgs.begin(),
269 E = CLArgs.end();
270 I != E; ++I)
271 Argv.push_back(I->c_str());
272 OwningPtr<InputArgList> Args(
273 Opts->ParseArgs(Argv.data(), Argv.data() + Argv.size(), MissingArgIndex,
274 MissingArgCount, IncludedFlagsBitmask));
275 std::vector<std::string> Inputs = Args->getAllArgValues(OPT_INPUT);
276 return Inputs.back();
277}
278
279// We provide this derivation to add in "-include (file)" arguments for header
280// dependencies.
281class AddDependenciesAdjuster : public ArgumentsAdjuster {
282public:
283 AddDependenciesAdjuster(DependencyMap &Dependencies)
284 : Dependencies(Dependencies) {}
285
286private:
287 // Callback for adjusting commandline arguments.
288 CommandLineArguments Adjust(const CommandLineArguments &Args) {
289 llvm::StringRef InputFile = findInputFile(Args);
290 DependentsVector &FileDependents = Dependencies[InputFile];
291 int Count = FileDependents.size();
292 if (Count == 0)
293 return Args;
294 CommandLineArguments NewArgs(Args);
295 for (int Index = 0; Index < Count; ++Index) {
296 NewArgs.push_back("-include");
297 std::string File(std::string("\"") + FileDependents[Index] +
298 std::string("\""));
299 NewArgs.push_back(FileDependents[Index]);
300 }
301 return NewArgs;
302 }
303 DependencyMap &Dependencies;
304};
305
John Thompsonce601e22013-03-14 01:41:29 +0000306// FIXME: The Location class seems to be something that we might
307// want to design to be applicable to a wider range of tools, and stick it
308// somewhere into Tooling/ in mainline
John Thompson4f8ba652013-03-12 02:07:30 +0000309struct Location {
310 const FileEntry *File;
311 unsigned Line, Column;
John Thompsonf5db45b2013-03-27 01:02:46 +0000312
313 Location() : File(), Line(), Column() {}
314
John Thompson4f8ba652013-03-12 02:07:30 +0000315 Location(SourceManager &SM, SourceLocation Loc) : File(), Line(), Column() {
316 Loc = SM.getExpansionLoc(Loc);
317 if (Loc.isInvalid())
318 return;
John Thompsonf5db45b2013-03-27 01:02:46 +0000319
John Thompson4f8ba652013-03-12 02:07:30 +0000320 std::pair<FileID, unsigned> Decomposed = SM.getDecomposedLoc(Loc);
321 File = SM.getFileEntryForID(Decomposed.first);
322 if (!File)
323 return;
John Thompsonf5db45b2013-03-27 01:02:46 +0000324
John Thompson4f8ba652013-03-12 02:07:30 +0000325 Line = SM.getLineNumber(Decomposed.first, Decomposed.second);
326 Column = SM.getColumnNumber(Decomposed.first, Decomposed.second);
327 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000328
John Thompson4f8ba652013-03-12 02:07:30 +0000329 operator bool() const { return File != 0; }
John Thompsonf5db45b2013-03-27 01:02:46 +0000330
John Thompson4f8ba652013-03-12 02:07:30 +0000331 friend bool operator==(const Location &X, const Location &Y) {
332 return X.File == Y.File && X.Line == Y.Line && X.Column == Y.Column;
333 }
334
335 friend bool operator!=(const Location &X, const Location &Y) {
336 return !(X == Y);
337 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000338
John Thompson4f8ba652013-03-12 02:07:30 +0000339 friend bool operator<(const Location &X, const Location &Y) {
340 if (X.File != Y.File)
341 return X.File < Y.File;
342 if (X.Line != Y.Line)
343 return X.Line < Y.Line;
344 return X.Column < Y.Column;
345 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000346 friend bool operator>(const Location &X, const Location &Y) { return Y < X; }
John Thompson4f8ba652013-03-12 02:07:30 +0000347 friend bool operator<=(const Location &X, const Location &Y) {
348 return !(Y < X);
349 }
350 friend bool operator>=(const Location &X, const Location &Y) {
351 return !(X < Y);
352 }
John Thompson4f8ba652013-03-12 02:07:30 +0000353};
354
John Thompson4f8ba652013-03-12 02:07:30 +0000355struct Entry {
John Thompson52d98862013-03-28 18:38:43 +0000356 enum EntryKind {
357 EK_Tag,
358 EK_Value,
359 EK_Macro,
360
361 EK_NumberOfKinds
John Thompson4f8ba652013-03-12 02:07:30 +0000362 } Kind;
John Thompsonf5db45b2013-03-27 01:02:46 +0000363
John Thompson4f8ba652013-03-12 02:07:30 +0000364 Location Loc;
John Thompson4e4d9b32013-03-28 01:20:19 +0000365
366 StringRef getKindName() { return getKindName(Kind); }
John Thompson52d98862013-03-28 18:38:43 +0000367 static StringRef getKindName(EntryKind kind);
John Thompson4f8ba652013-03-12 02:07:30 +0000368};
369
John Thompson4e4d9b32013-03-28 01:20:19 +0000370// Return a string representing the given kind.
John Thompson52d98862013-03-28 18:38:43 +0000371StringRef Entry::getKindName(Entry::EntryKind kind) {
John Thompson4e4d9b32013-03-28 01:20:19 +0000372 switch (kind) {
John Thompson52d98862013-03-28 18:38:43 +0000373 case EK_Tag:
John Thompson4e4d9b32013-03-28 01:20:19 +0000374 return "tag";
John Thompson52d98862013-03-28 18:38:43 +0000375 case EK_Value:
John Thompson4e4d9b32013-03-28 01:20:19 +0000376 return "value";
John Thompson52d98862013-03-28 18:38:43 +0000377 case EK_Macro:
John Thompson4e4d9b32013-03-28 01:20:19 +0000378 return "macro";
John Thompson52d98862013-03-28 18:38:43 +0000379 case EK_NumberOfKinds:
John Thompson4e4d9b32013-03-28 01:20:19 +0000380 break;
John Thompson4e4d9b32013-03-28 01:20:19 +0000381 }
David Blaikiec66c07d2013-03-28 02:30:37 +0000382 llvm_unreachable("invalid Entry kind");
John Thompson4e4d9b32013-03-28 01:20:19 +0000383}
384
John Thompson4f8ba652013-03-12 02:07:30 +0000385struct HeaderEntry {
386 std::string Name;
387 Location Loc;
John Thompsonf5db45b2013-03-27 01:02:46 +0000388
John Thompson4f8ba652013-03-12 02:07:30 +0000389 friend bool operator==(const HeaderEntry &X, const HeaderEntry &Y) {
390 return X.Loc == Y.Loc && X.Name == Y.Name;
391 }
392 friend bool operator!=(const HeaderEntry &X, const HeaderEntry &Y) {
393 return !(X == Y);
394 }
395 friend bool operator<(const HeaderEntry &X, const HeaderEntry &Y) {
396 return X.Loc < Y.Loc || (X.Loc == Y.Loc && X.Name < Y.Name);
397 }
398 friend bool operator>(const HeaderEntry &X, const HeaderEntry &Y) {
399 return Y < X;
400 }
401 friend bool operator<=(const HeaderEntry &X, const HeaderEntry &Y) {
402 return !(Y < X);
403 }
404 friend bool operator>=(const HeaderEntry &X, const HeaderEntry &Y) {
405 return !(X < Y);
406 }
407};
408
409typedef std::vector<HeaderEntry> HeaderContents;
410
John Thompsonf5db45b2013-03-27 01:02:46 +0000411class EntityMap : public StringMap<SmallVector<Entry, 2> > {
John Thompson4f8ba652013-03-12 02:07:30 +0000412public:
John Thompsonf5db45b2013-03-27 01:02:46 +0000413 DenseMap<const FileEntry *, HeaderContents> HeaderContentMismatches;
414
John Thompson52d98862013-03-28 18:38:43 +0000415 void add(const std::string &Name, enum Entry::EntryKind Kind, Location Loc) {
John Thompson4f8ba652013-03-12 02:07:30 +0000416 // Record this entity in its header.
417 HeaderEntry HE = { Name, Loc };
418 CurHeaderContents[Loc.File].push_back(HE);
John Thompsonf5db45b2013-03-27 01:02:46 +0000419
John Thompson4f8ba652013-03-12 02:07:30 +0000420 // Check whether we've seen this entry before.
John Thompsonf5db45b2013-03-27 01:02:46 +0000421 SmallVector<Entry, 2> &Entries = (*this)[Name];
John Thompson4f8ba652013-03-12 02:07:30 +0000422 for (unsigned I = 0, N = Entries.size(); I != N; ++I) {
423 if (Entries[I].Kind == Kind && Entries[I].Loc == Loc)
424 return;
425 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000426
John Thompson4f8ba652013-03-12 02:07:30 +0000427 // We have not seen this entry before; record it.
428 Entry E = { Kind, Loc };
429 Entries.push_back(E);
430 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000431
John Thompson4f8ba652013-03-12 02:07:30 +0000432 void mergeCurHeaderContents() {
John Thompsonf5db45b2013-03-27 01:02:46 +0000433 for (DenseMap<const FileEntry *, HeaderContents>::iterator
434 H = CurHeaderContents.begin(),
435 HEnd = CurHeaderContents.end();
John Thompson4f8ba652013-03-12 02:07:30 +0000436 H != HEnd; ++H) {
437 // Sort contents.
438 std::sort(H->second.begin(), H->second.end());
439
440 // Check whether we've seen this header before.
John Thompsonf5db45b2013-03-27 01:02:46 +0000441 DenseMap<const FileEntry *, HeaderContents>::iterator KnownH =
442 AllHeaderContents.find(H->first);
John Thompson4f8ba652013-03-12 02:07:30 +0000443 if (KnownH == AllHeaderContents.end()) {
444 // We haven't seen this header before; record its contents.
445 AllHeaderContents.insert(*H);
446 continue;
447 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000448
John Thompson4f8ba652013-03-12 02:07:30 +0000449 // If the header contents are the same, we're done.
450 if (H->second == KnownH->second)
451 continue;
John Thompsonf5db45b2013-03-27 01:02:46 +0000452
John Thompson4f8ba652013-03-12 02:07:30 +0000453 // Determine what changed.
John Thompsonf5db45b2013-03-27 01:02:46 +0000454 std::set_symmetric_difference(
455 H->second.begin(), H->second.end(), KnownH->second.begin(),
456 KnownH->second.end(),
457 std::back_inserter(HeaderContentMismatches[H->first]));
John Thompson4f8ba652013-03-12 02:07:30 +0000458 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000459
John Thompson4f8ba652013-03-12 02:07:30 +0000460 CurHeaderContents.clear();
461 }
John Thompson161381e2013-06-27 18:52:23 +0000462
John Thompson1f67ccb2013-03-12 18:51:47 +0000463private:
John Thompsonf5db45b2013-03-27 01:02:46 +0000464 DenseMap<const FileEntry *, HeaderContents> CurHeaderContents;
465 DenseMap<const FileEntry *, HeaderContents> AllHeaderContents;
John Thompson4f8ba652013-03-12 02:07:30 +0000466};
467
John Thompson161381e2013-06-27 18:52:23 +0000468class CollectEntitiesVisitor
469 : public RecursiveASTVisitor<CollectEntitiesVisitor> {
John Thompson4f8ba652013-03-12 02:07:30 +0000470public:
471 CollectEntitiesVisitor(SourceManager &SM, EntityMap &Entities)
John Thompsonf5db45b2013-03-27 01:02:46 +0000472 : SM(SM), Entities(Entities) {}
473
John Thompson4f8ba652013-03-12 02:07:30 +0000474 bool TraverseStmt(Stmt *S) { return true; }
475 bool TraverseType(QualType T) { return true; }
476 bool TraverseTypeLoc(TypeLoc TL) { return true; }
477 bool TraverseNestedNameSpecifier(NestedNameSpecifier *NNS) { return true; }
John Thompsonf5db45b2013-03-27 01:02:46 +0000478 bool TraverseNestedNameSpecifierLoc(NestedNameSpecifierLoc NNS) {
479 return true;
480 }
481 bool TraverseDeclarationNameInfo(DeclarationNameInfo NameInfo) {
482 return true;
483 }
John Thompson4f8ba652013-03-12 02:07:30 +0000484 bool TraverseTemplateName(TemplateName Template) { return true; }
485 bool TraverseTemplateArgument(const TemplateArgument &Arg) { return true; }
John Thompsonf5db45b2013-03-27 01:02:46 +0000486 bool TraverseTemplateArgumentLoc(const TemplateArgumentLoc &ArgLoc) {
487 return true;
488 }
John Thompson4f8ba652013-03-12 02:07:30 +0000489 bool TraverseTemplateArguments(const TemplateArgument *Args,
John Thompsonf5db45b2013-03-27 01:02:46 +0000490 unsigned NumArgs) {
491 return true;
492 }
John Thompson4f8ba652013-03-12 02:07:30 +0000493 bool TraverseConstructorInitializer(CXXCtorInitializer *Init) { return true; }
494 bool TraverseLambdaCapture(LambdaExpr::Capture C) { return true; }
John Thompsonf5db45b2013-03-27 01:02:46 +0000495
John Thompson4f8ba652013-03-12 02:07:30 +0000496 bool VisitNamedDecl(NamedDecl *ND) {
497 // We only care about file-context variables.
498 if (!ND->getDeclContext()->isFileContext())
499 return true;
John Thompsonf5db45b2013-03-27 01:02:46 +0000500
John Thompson4f8ba652013-03-12 02:07:30 +0000501 // Skip declarations that tend to be properly multiply-declared.
502 if (isa<NamespaceDecl>(ND) || isa<UsingDirectiveDecl>(ND) ||
John Thompsonf5db45b2013-03-27 01:02:46 +0000503 isa<NamespaceAliasDecl>(ND) ||
504 isa<ClassTemplateSpecializationDecl>(ND) || isa<UsingDecl>(ND) ||
John Thompson8e01c062013-08-26 15:17:23 +0000505 isa<ClassTemplateDecl>(ND) || isa<TemplateTypeParmDecl>(ND) ||
John Thompsoncc2e2912013-09-03 18:44:11 +0000506 isa<TypeAliasTemplateDecl>(ND) || isa<UsingShadowDecl>(ND) ||
507 isa<FunctionDecl>(ND) || isa<FunctionTemplateDecl>(ND) ||
John Thompson4f8ba652013-03-12 02:07:30 +0000508 (isa<TagDecl>(ND) &&
509 !cast<TagDecl>(ND)->isThisDeclarationADefinition()))
510 return true;
John Thompsonf5db45b2013-03-27 01:02:46 +0000511
John Thompson8e01c062013-08-26 15:17:23 +0000512 // Skip anonymous declarations.
513 if (!ND->getDeclName())
514 return true;
515
516 // Get the qualified name.
John Thompsoncc2e2912013-09-03 18:44:11 +0000517 std::string Name;
518 llvm::raw_string_ostream OS(Name);
519 ND->printQualifiedName(OS);
520 OS.flush();
John Thompson4f8ba652013-03-12 02:07:30 +0000521 if (Name.empty())
522 return true;
John Thompsonf5db45b2013-03-27 01:02:46 +0000523
John Thompson4f8ba652013-03-12 02:07:30 +0000524 Location Loc(SM, ND->getLocation());
525 if (!Loc)
526 return true;
John Thompsonf5db45b2013-03-27 01:02:46 +0000527
John Thompson52d98862013-03-28 18:38:43 +0000528 Entities.add(Name, isa<TagDecl>(ND) ? Entry::EK_Tag : Entry::EK_Value, Loc);
John Thompson4f8ba652013-03-12 02:07:30 +0000529 return true;
530 }
John Thompson161381e2013-06-27 18:52:23 +0000531
John Thompson1f67ccb2013-03-12 18:51:47 +0000532private:
533 SourceManager &SM;
534 EntityMap &Entities;
John Thompson4f8ba652013-03-12 02:07:30 +0000535};
536
537class CollectEntitiesConsumer : public ASTConsumer {
John Thompson4f8ba652013-03-12 02:07:30 +0000538public:
John Thompson94faa4d2013-07-26 23:56:42 +0000539 CollectEntitiesConsumer(EntityMap &Entities,
540 PreprocessorTracker &preprocessorTracker,
541 Preprocessor &PP, StringRef InFile)
542 : Entities(Entities), PPTracker(preprocessorTracker), PP(PP) {
543 PPTracker.handlePreprocessorEntry(PP, InFile);
544 }
545
546 ~CollectEntitiesConsumer() { PPTracker.handlePreprocessorExit(); }
John Thompsonf5db45b2013-03-27 01:02:46 +0000547
John Thompson4f8ba652013-03-12 02:07:30 +0000548 virtual void HandleTranslationUnit(ASTContext &Ctx) {
549 SourceManager &SM = Ctx.getSourceManager();
John Thompsonf5db45b2013-03-27 01:02:46 +0000550
John Thompson4f8ba652013-03-12 02:07:30 +0000551 // Collect declared entities.
552 CollectEntitiesVisitor(SM, Entities)
John Thompsonf5db45b2013-03-27 01:02:46 +0000553 .TraverseDecl(Ctx.getTranslationUnitDecl());
554
John Thompson4f8ba652013-03-12 02:07:30 +0000555 // Collect macro definitions.
556 for (Preprocessor::macro_iterator M = PP.macro_begin(),
John Thompsonf5db45b2013-03-27 01:02:46 +0000557 MEnd = PP.macro_end();
John Thompson4f8ba652013-03-12 02:07:30 +0000558 M != MEnd; ++M) {
559 Location Loc(SM, M->second->getLocation());
560 if (!Loc)
561 continue;
562
John Thompson52d98862013-03-28 18:38:43 +0000563 Entities.add(M->first->getName().str(), Entry::EK_Macro, Loc);
John Thompson4f8ba652013-03-12 02:07:30 +0000564 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000565
John Thompson4f8ba652013-03-12 02:07:30 +0000566 // Merge header contents.
567 Entities.mergeCurHeaderContents();
568 }
John Thompson161381e2013-06-27 18:52:23 +0000569
John Thompson1f67ccb2013-03-12 18:51:47 +0000570private:
571 EntityMap &Entities;
John Thompson94faa4d2013-07-26 23:56:42 +0000572 PreprocessorTracker &PPTracker;
John Thompson1f67ccb2013-03-12 18:51:47 +0000573 Preprocessor &PP;
John Thompson4f8ba652013-03-12 02:07:30 +0000574};
575
576class CollectEntitiesAction : public SyntaxOnlyAction {
John Thompson1f67ccb2013-03-12 18:51:47 +0000577public:
John Thompson94faa4d2013-07-26 23:56:42 +0000578 CollectEntitiesAction(EntityMap &Entities,
579 PreprocessorTracker &preprocessorTracker)
580 : Entities(Entities), PPTracker(preprocessorTracker) {}
John Thompson161381e2013-06-27 18:52:23 +0000581
John Thompson4f8ba652013-03-12 02:07:30 +0000582protected:
John Thompson161381e2013-06-27 18:52:23 +0000583 virtual clang::ASTConsumer *CreateASTConsumer(CompilerInstance &CI,
584 StringRef InFile) {
John Thompson94faa4d2013-07-26 23:56:42 +0000585 return new CollectEntitiesConsumer(Entities, PPTracker,
586 CI.getPreprocessor(), InFile);
John Thompson4f8ba652013-03-12 02:07:30 +0000587 }
John Thompson161381e2013-06-27 18:52:23 +0000588
John Thompson1f67ccb2013-03-12 18:51:47 +0000589private:
John Thompsonf5db45b2013-03-27 01:02:46 +0000590 EntityMap &Entities;
John Thompson94faa4d2013-07-26 23:56:42 +0000591 PreprocessorTracker &PPTracker;
John Thompson4f8ba652013-03-12 02:07:30 +0000592};
593
594class ModularizeFrontendActionFactory : public FrontendActionFactory {
John Thompson4f8ba652013-03-12 02:07:30 +0000595public:
John Thompson94faa4d2013-07-26 23:56:42 +0000596 ModularizeFrontendActionFactory(EntityMap &Entities,
597 PreprocessorTracker &preprocessorTracker)
598 : Entities(Entities), PPTracker(preprocessorTracker) {}
John Thompson4f8ba652013-03-12 02:07:30 +0000599
600 virtual CollectEntitiesAction *create() {
John Thompson94faa4d2013-07-26 23:56:42 +0000601 return new CollectEntitiesAction(Entities, PPTracker);
John Thompson4f8ba652013-03-12 02:07:30 +0000602 }
John Thompson161381e2013-06-27 18:52:23 +0000603
John Thompson1f67ccb2013-03-12 18:51:47 +0000604private:
John Thompsonf5db45b2013-03-27 01:02:46 +0000605 EntityMap &Entities;
John Thompson94faa4d2013-07-26 23:56:42 +0000606 PreprocessorTracker &PPTracker;
John Thompson4f8ba652013-03-12 02:07:30 +0000607};
608
John Thompsonbb0a3b02013-08-09 13:52:09 +0000609int main(int Argc, const char **Argv) {
John Thompsona2de1082013-03-26 01:17:48 +0000610
611 // This causes options to be parsed.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000612 cl::ParseCommandLineOptions(Argc, Argv, "modularize.\n");
John Thompsona2de1082013-03-26 01:17:48 +0000613
614 // No go if we have no header list file.
615 if (ListFileName.size() == 0) {
616 cl::PrintHelpMessage();
John Thompsonea6c8db2013-03-27 21:23:21 +0000617 return 1;
John Thompson4f8ba652013-03-12 02:07:30 +0000618 }
John Thompsona2de1082013-03-26 01:17:48 +0000619
John Thompson74751802013-09-03 18:48:43 +0000620 // Get header file names and dependencies.
John Thompsonf5db45b2013-03-27 01:02:46 +0000621 SmallVector<std::string, 32> Headers;
John Thompson74751802013-09-03 18:48:43 +0000622 DependencyMap Dependencies;
623 if (error_code EC = getHeaderFileNames(Headers, Dependencies, ListFileName,
624 HeaderPrefix)) {
John Thompsonbb0a3b02013-08-09 13:52:09 +0000625 errs() << Argv[0] << ": error: Unable to get header list '" << ListFileName
626 << "': " << EC.message() << '\n';
John Thompsonea6c8db2013-03-27 21:23:21 +0000627 return 1;
John Thompson4f8ba652013-03-12 02:07:30 +0000628 }
John Thompsona2de1082013-03-26 01:17:48 +0000629
John Thompson4f8ba652013-03-12 02:07:30 +0000630 // Create the compilation database.
John Thompsona2de1082013-03-26 01:17:48 +0000631 SmallString<256> PathBuf;
John Thompsonf5db45b2013-03-27 01:02:46 +0000632 sys::fs::current_path(PathBuf);
633 OwningPtr<CompilationDatabase> Compilations;
634 Compilations.reset(
635 new FixedCompilationDatabase(Twine(PathBuf), CC1Arguments));
John Thompsona2de1082013-03-26 01:17:48 +0000636
John Thompson94faa4d2013-07-26 23:56:42 +0000637 // Create preprocessor tracker, to watch for macro and conditional problems.
638 OwningPtr<PreprocessorTracker> PPTracker(PreprocessorTracker::create());
639
John Thompson4f8ba652013-03-12 02:07:30 +0000640 // Parse all of the headers, detecting duplicates.
641 EntityMap Entities;
642 ClangTool Tool(*Compilations, Headers);
John Thompson74751802013-09-03 18:48:43 +0000643 Tool.appendArgumentsAdjuster(new AddDependenciesAdjuster(Dependencies));
John Thompson94faa4d2013-07-26 23:56:42 +0000644 int HadErrors =
645 Tool.run(new ModularizeFrontendActionFactory(Entities, *PPTracker));
John Thompsonce601e22013-03-14 01:41:29 +0000646
John Thompson4e4d9b32013-03-28 01:20:19 +0000647 // Create a place to save duplicate entity locations, separate bins per kind.
648 typedef SmallVector<Location, 8> LocationArray;
John Thompson52d98862013-03-28 18:38:43 +0000649 typedef SmallVector<LocationArray, Entry::EK_NumberOfKinds> EntryBinArray;
John Thompson4e4d9b32013-03-28 01:20:19 +0000650 EntryBinArray EntryBins;
John Thompsonbb0a3b02013-08-09 13:52:09 +0000651 int KindIndex;
652 for (KindIndex = 0; KindIndex < Entry::EK_NumberOfKinds; ++KindIndex) {
653 LocationArray Array;
654 EntryBins.push_back(Array);
Michael Gottesman4b249212013-03-28 06:07:15 +0000655 }
John Thompson4e4d9b32013-03-28 01:20:19 +0000656
John Thompson4f8ba652013-03-12 02:07:30 +0000657 // Check for the same entity being defined in multiple places.
658 for (EntityMap::iterator E = Entities.begin(), EEnd = Entities.end();
659 E != EEnd; ++E) {
John Thompson4e4d9b32013-03-28 01:20:19 +0000660 // If only one occurance, exit early.
661 if (E->second.size() == 1)
662 continue;
663 // Clear entity locations.
664 for (EntryBinArray::iterator CI = EntryBins.begin(), CE = EntryBins.end();
665 CI != CE; ++CI) {
John Thompson52d98862013-03-28 18:38:43 +0000666 CI->clear();
John Thompson4e4d9b32013-03-28 01:20:19 +0000667 }
668 // Walk the entities of a single name, collecting the locations,
669 // separated into separate bins.
John Thompson4f8ba652013-03-12 02:07:30 +0000670 for (unsigned I = 0, N = E->second.size(); I != N; ++I) {
John Thompson52d98862013-03-28 18:38:43 +0000671 EntryBins[E->second[I].Kind].push_back(E->second[I].Loc);
John Thompson4e4d9b32013-03-28 01:20:19 +0000672 }
673 // Report any duplicate entity definition errors.
John Thompsonbb0a3b02013-08-09 13:52:09 +0000674 int KindIndex = 0;
John Thompson4e4d9b32013-03-28 01:20:19 +0000675 for (EntryBinArray::iterator DI = EntryBins.begin(), DE = EntryBins.end();
John Thompsonbb0a3b02013-08-09 13:52:09 +0000676 DI != DE; ++DI, ++KindIndex) {
677 int ECount = DI->size();
John Thompson4e4d9b32013-03-28 01:20:19 +0000678 // If only 1 occurance, skip;
John Thompsonbb0a3b02013-08-09 13:52:09 +0000679 if (ECount <= 1)
John Thompson4f8ba652013-03-12 02:07:30 +0000680 continue;
John Thompson52d98862013-03-28 18:38:43 +0000681 LocationArray::iterator FI = DI->begin();
John Thompsonbb0a3b02013-08-09 13:52:09 +0000682 StringRef kindName = Entry::getKindName((Entry::EntryKind)KindIndex);
John Thompson4e4d9b32013-03-28 01:20:19 +0000683 errs() << "error: " << kindName << " '" << E->first()
684 << "' defined at multiple locations:\n";
John Thompson52d98862013-03-28 18:38:43 +0000685 for (LocationArray::iterator FE = DI->end(); FI != FE; ++FI) {
John Thompson4e4d9b32013-03-28 01:20:19 +0000686 errs() << " " << FI->File->getName() << ":" << FI->Line << ":"
687 << FI->Column << "\n";
John Thompson4f8ba652013-03-12 02:07:30 +0000688 }
John Thompson4f8ba652013-03-12 02:07:30 +0000689 HadErrors = 1;
690 }
691 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000692
John Thompson94faa4d2013-07-26 23:56:42 +0000693 // Complain about macro instance in header files that differ based on how
694 // they are included.
695 if (PPTracker->reportInconsistentMacros(errs()))
696 HadErrors = 1;
697
698 // Complain about preprocessor conditional directives in header files that
699 // differ based on how they are included.
700 if (PPTracker->reportInconsistentConditionals(errs()))
701 HadErrors = 1;
702
John Thompson4f8ba652013-03-12 02:07:30 +0000703 // Complain about any headers that have contents that differ based on how
704 // they are included.
John Thompsonce601e22013-03-14 01:41:29 +0000705 // FIXME: Could we provide information about which preprocessor conditionals
706 // are involved?
John Thompsonf5db45b2013-03-27 01:02:46 +0000707 for (DenseMap<const FileEntry *, HeaderContents>::iterator
708 H = Entities.HeaderContentMismatches.begin(),
709 HEnd = Entities.HeaderContentMismatches.end();
John Thompson4f8ba652013-03-12 02:07:30 +0000710 H != HEnd; ++H) {
711 if (H->second.empty()) {
John Thompsonf5db45b2013-03-27 01:02:46 +0000712 errs() << "internal error: phantom header content mismatch\n";
John Thompson4f8ba652013-03-12 02:07:30 +0000713 continue;
714 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000715
John Thompson4f8ba652013-03-12 02:07:30 +0000716 HadErrors = 1;
John Thompsonf5db45b2013-03-27 01:02:46 +0000717 errs() << "error: header '" << H->first->getName()
John Thompson94faa4d2013-07-26 23:56:42 +0000718 << "' has different contents depending on how it was included.\n";
John Thompson4f8ba652013-03-12 02:07:30 +0000719 for (unsigned I = 0, N = H->second.size(); I != N; ++I) {
John Thompson161381e2013-06-27 18:52:23 +0000720 errs() << "note: '" << H->second[I].Name << "' in "
721 << H->second[I].Loc.File->getName() << " at "
722 << H->second[I].Loc.Line << ":" << H->second[I].Loc.Column
723 << " not always provided\n";
John Thompson4f8ba652013-03-12 02:07:30 +0000724 }
725 }
John Thompsonf5db45b2013-03-27 01:02:46 +0000726
John Thompson4f8ba652013-03-12 02:07:30 +0000727 return HadErrors;
728}