[lldb] Decouple importing the std C++ module from the way the program is compiled
Summary:
At the moment, when trying to import the `std` module in LLDB, we look at the imported modules used in the compiled program
and try to infer the Clang configuration we need from the DWARF module-import. That was the initial idea but turned out to
cause a few problems or inconveniences:
* It requires that users compile their programs with C++ modules. Given how experimental C++ modules are makes this feature inaccessible
for many users. Also it means that people can't just get the benefits of this feature for free when we activate it by default
(and we can't just close all the associated bug reports).
* Relying on DWARF's imported module tags (that are only emitted by default on macOS) means this can only be used when using DWARF (and with -glldb on Linux).
* We essentially hardcoded the C standard library paths on some platforms (Linux) or just couldn't support this feature on other platforms (macOS).
This patch drops the whole idea of looking at the imported module DWARF tags and instead just uses the support files of the compilation unit.
If we look at the support files and see file paths that indicate where the C standard library and libc++ are, we can just create the module
configuration this information. This fixes all the problems above which means we can enable all the tests now on Linux, macOS and with other debug information
than what we currently had. The only debug information specific code is now the iteration over external type module when -gmodules is used (as `std` and also the
`Darwin` module are their own external type module with their own files).
The meat of this patch is the CppModuleConfiguration which looks at the file paths from the compilation unit and then figures out the include paths
based on those paths. It's quite conservative in that it only enables modules if we find a single C library and single libc++ library. It's still missing some
test mode where we try to compile an expression before we actually activate the config for the user (which probably also needs some caching mechanism),
but for now it works and makes the feature usable.
Reviewers: aprantl, shafik, jdoerfert
Reviewed By: aprantl
Subscribers: mgorny, abidh, JDevlieghere, lldb-commits
Tags: #c_modules_in_lldb, #lldb
Differential Revision: https://reviews.llvm.org/D67760
llvm-svn: 372716
diff --git a/lldb/source/Plugins/ExpressionParser/Clang/CMakeLists.txt b/lldb/source/Plugins/ExpressionParser/Clang/CMakeLists.txt
index 99d50c4..b114ce77 100644
--- a/lldb/source/Plugins/ExpressionParser/Clang/CMakeLists.txt
+++ b/lldb/source/Plugins/ExpressionParser/Clang/CMakeLists.txt
@@ -19,6 +19,7 @@
ClangPersistentVariables.cpp
ClangUserExpression.cpp
ClangUtilityFunction.cpp
+ CppModuleConfiguration.cpp
IRForTarget.cpp
IRDynamicChecks.cpp
diff --git a/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp b/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp
index cb0e985..593348f 100644
--- a/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp
+++ b/lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp
@@ -241,38 +241,23 @@
search_opts.ModuleCachePath = module_cache.str();
LLDB_LOG(log, "Using module cache path: {0}", module_cache.c_str());
- FileSpec clang_resource_dir = GetClangResourceDir();
- std::string resource_dir = clang_resource_dir.GetPath();
- if (FileSystem::Instance().IsDirectory(resource_dir)) {
- search_opts.ResourceDir = resource_dir;
- std::string resource_include = resource_dir + "/include";
- search_opts.AddPath(resource_include, frontend::System, false, true);
-
- LLDB_LOG(log, "Added resource include dir: {0}", resource_include);
- }
+ search_opts.ResourceDir = GetClangResourceDir().GetPath();
search_opts.ImplicitModuleMaps = true;
-
- std::vector<std::string> system_include_directories =
- target_sp->GetPlatform()->GetSystemIncludeDirectories(
- lldb::eLanguageTypeC_plus_plus);
-
- for (const std::string &include_dir : system_include_directories) {
- search_opts.AddPath(include_dir, frontend::System, false, true);
-
- LLDB_LOG(log, "Added system include dir: {0}", include_dir);
- }
}
//===----------------------------------------------------------------------===//
// Implementation of ClangExpressionParser
//===----------------------------------------------------------------------===//
-ClangExpressionParser::ClangExpressionParser(ExecutionContextScope *exe_scope, Expression &expr,
- bool generate_debug_info, std::vector<std::string> include_directories, std::string filename)
+ClangExpressionParser::ClangExpressionParser(
+ ExecutionContextScope *exe_scope, Expression &expr,
+ bool generate_debug_info, std::vector<std::string> include_directories,
+ std::string filename)
: ExpressionParser(exe_scope, expr, generate_debug_info), m_compiler(),
m_pp_callbacks(nullptr),
- m_include_directories(std::move(include_directories)), m_filename(std::move(filename)) {
+ m_include_directories(std::move(include_directories)),
+ m_filename(std::move(filename)) {
Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_EXPRESSIONS));
// We can't compile expressions without a target. So if the exe_scope is
diff --git a/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.cpp b/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.cpp
index ae95c68..813d03c 100644
--- a/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.cpp
+++ b/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.cpp
@@ -25,6 +25,7 @@
#include "ClangExpressionParser.h"
#include "ClangModulesDeclVendor.h"
#include "ClangPersistentVariables.h"
+#include "CppModuleConfiguration.h"
#include "lldb/Core/Debugger.h"
#include "lldb/Core/Module.h"
@@ -433,48 +434,59 @@
}
}
-std::vector<std::string>
-ClangUserExpression::GetModulesToImport(ExecutionContext &exe_ctx) {
+/// Utility method that puts a message into the expression log and
+/// returns an invalid module configuration.
+static CppModuleConfiguration LogConfigError(const std::string &msg) {
Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_EXPRESSIONS));
+ LLDB_LOG(log, "[C++ module config] {0}", msg);
+ return CppModuleConfiguration();
+}
- if (!SupportsCxxModuleImport(Language()))
- return {};
+CppModuleConfiguration GetModuleConfig(lldb::LanguageType language,
+ ExecutionContext &exe_ctx) {
+ // Don't do anything if this is not a C++ module configuration.
+ if (!SupportsCxxModuleImport(language))
+ return LogConfigError("Language doesn't support C++ modules");
Target *target = exe_ctx.GetTargetPtr();
- if (!target || !target->GetEnableImportStdModule())
- return {};
+ if (!target)
+ return LogConfigError("No target");
+
+ if (!target->GetEnableImportStdModule())
+ return LogConfigError("Importing std module not enabled in settings");
StackFrame *frame = exe_ctx.GetFramePtr();
if (!frame)
- return {};
+ return LogConfigError("No frame");
Block *block = frame->GetFrameBlock();
if (!block)
- return {};
+ return LogConfigError("No block");
SymbolContext sc;
block->CalculateSymbolContext(&sc);
if (!sc.comp_unit)
- return {};
+ return LogConfigError("Couldn't calculate symbol context");
- if (log) {
- for (const SourceModule &m : sc.comp_unit->GetImportedModules()) {
- LLDB_LOG(log, "Found module in compile unit: {0:$[.]} - include dir: {1}",
- llvm::make_range(m.path.begin(), m.path.end()), m.search_path);
+ // Build a list of files we need to analyze to build the configuration.
+ FileSpecList files;
+ for (const FileSpec &f : sc.comp_unit->GetSupportFiles())
+ files.AppendIfUnique(f);
+ // We also need to look at external modules in the case of -gmodules as they
+ // contain the support files for libc++ and the C library.
+ sc.comp_unit->ForEachExternalModule([&files](lldb::ModuleSP module) {
+ for (std::size_t i = 0; i < module->GetNumCompileUnits(); ++i) {
+ const FileSpecList &support_files =
+ module->GetCompileUnitAtIndex(i)->GetSupportFiles();
+ for (const FileSpec &f : support_files) {
+ files.AppendIfUnique(f);
+ }
}
- }
-
- for (const SourceModule &m : sc.comp_unit->GetImportedModules())
- m_include_directories.emplace_back(m.search_path.GetCString());
-
- // Check if we imported 'std' or any of its submodules.
- // We currently don't support importing any other modules in the expression
- // parser.
- for (const SourceModule &m : sc.comp_unit->GetImportedModules())
- if (!m.path.empty() && m.path.front() == "std")
- return {"std"};
-
- return {};
+ });
+ // Try to create a configuration from the files. If there is no valid
+ // configuration possible with the files, this just returns an invalid
+ // configuration.
+ return CppModuleConfiguration(files);
}
bool ClangUserExpression::PrepareForParsing(
@@ -502,14 +514,21 @@
SetupDeclVendor(exe_ctx, m_target);
- std::vector<std::string> used_modules = GetModulesToImport(exe_ctx);
- m_imported_cpp_modules = !used_modules.empty();
+ CppModuleConfiguration module_config = GetModuleConfig(m_language, exe_ctx);
+ llvm::ArrayRef<std::string> imported_modules =
+ module_config.GetImportedModules();
+ m_imported_cpp_modules = !imported_modules.empty();
+ m_include_directories = module_config.GetIncludeDirs();
LLDB_LOG(log, "List of imported modules in expression: {0}",
- llvm::make_range(used_modules.begin(), used_modules.end()));
+ llvm::make_range(imported_modules.begin(), imported_modules.end()));
+ LLDB_LOG(log, "List of include directories gathered for modules: {0}",
+ llvm::make_range(m_include_directories.begin(),
+ m_include_directories.end()));
UpdateLanguageForExpr();
- CreateSourceCode(diagnostic_manager, exe_ctx, used_modules, for_completion);
+ CreateSourceCode(diagnostic_manager, exe_ctx, imported_modules,
+ for_completion);
return true;
}
diff --git a/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.h b/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.h
index ac28ac4..d94f9cc 100644
--- a/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.h
+++ b/lldb/source/Plugins/ExpressionParser/Clang/ClangUserExpression.h
@@ -179,7 +179,6 @@
lldb::addr_t struct_address,
DiagnosticManager &diagnostic_manager) override;
- std::vector<std::string> GetModulesToImport(ExecutionContext &exe_ctx);
void CreateSourceCode(DiagnosticManager &diagnostic_manager,
ExecutionContext &exe_ctx,
std::vector<std::string> modules_to_import,
diff --git a/lldb/source/Plugins/ExpressionParser/Clang/CppModuleConfiguration.cpp b/lldb/source/Plugins/ExpressionParser/Clang/CppModuleConfiguration.cpp
new file mode 100644
index 0000000..47c0345
--- /dev/null
+++ b/lldb/source/Plugins/ExpressionParser/Clang/CppModuleConfiguration.cpp
@@ -0,0 +1,82 @@
+//===-- CppModuleConfiguration.cpp ----------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "CppModuleConfiguration.h"
+
+#include "ClangHost.h"
+#include "lldb/Host/FileSystem.h"
+
+using namespace lldb_private;
+
+bool CppModuleConfiguration::SetOncePath::TrySet(llvm::StringRef path) {
+ // Setting for the first time always works.
+ if (m_first) {
+ m_path = path.str();
+ m_valid = true;
+ m_first = false;
+ return true;
+ }
+ // Changing the path to the same value is fine.
+ if (m_path == path)
+ return true;
+
+ // Changing the path after it was already set is not allowed.
+ m_valid = false;
+ return false;
+}
+
+bool CppModuleConfiguration::analyzeFile(const FileSpec &f) {
+ llvm::SmallString<256> dir_buffer = f.GetDirectory().GetStringRef();
+ // Convert to posix style so that we can use '/'.
+ llvm::sys::path::native(dir_buffer, llvm::sys::path::Style::posix);
+ llvm::StringRef posix_dir(dir_buffer);
+
+ // Check for /c++/vX/ that is used by libc++.
+ static llvm::Regex libcpp_regex(R"regex(/c[+][+]/v[0-9]/)regex");
+ if (libcpp_regex.match(f.GetPath())) {
+ // Strip away libc++'s /experimental directory if there is one.
+ posix_dir.consume_back("/experimental");
+ return m_std_inc.TrySet(posix_dir);
+ }
+
+ // Check for /usr/include. On Linux this might be /usr/include/bits, so
+ // we should remove that '/bits' suffix to get the actual include directory.
+ if (posix_dir.endswith("/usr/include/bits"))
+ posix_dir.consume_back("/bits");
+ if (posix_dir.endswith("/usr/include"))
+ return m_c_inc.TrySet(posix_dir);
+
+ // File wasn't interesting, continue analyzing.
+ return true;
+}
+
+bool CppModuleConfiguration::hasValidConfig() {
+ // We all these include directories to have a valid usable configuration.
+ return m_c_inc.Valid() && m_std_inc.Valid();
+}
+
+CppModuleConfiguration::CppModuleConfiguration(
+ const FileSpecList &support_files) {
+ // Analyze all files we were given to build the configuration.
+ bool error = !std::all_of(support_files.begin(), support_files.end(),
+ std::bind(&CppModuleConfiguration::analyzeFile,
+ this, std::placeholders::_1));
+ // If we have a valid configuration at this point, set the
+ // include directories and module list that should be used.
+ if (!error && hasValidConfig()) {
+ // Calculate the resource directory for LLDB.
+ llvm::SmallString<256> resource_dir;
+ llvm::sys::path::append(resource_dir, GetClangResourceDir().GetPath(),
+ "include");
+ m_resource_inc = resource_dir.str();
+
+ // This order matches the way Clang orders these directories.
+ m_include_dirs = {m_std_inc.Get(), m_resource_inc, m_c_inc.Get()};
+ m_imported_modules = {"std"};
+ }
+}
diff --git a/lldb/source/Plugins/ExpressionParser/Clang/CppModuleConfiguration.h b/lldb/source/Plugins/ExpressionParser/Clang/CppModuleConfiguration.h
new file mode 100644
index 0000000..8e892e3
--- /dev/null
+++ b/lldb/source/Plugins/ExpressionParser/Clang/CppModuleConfiguration.h
@@ -0,0 +1,84 @@
+//===-- CppModuleConfiguration.h --------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef liblldb_CppModuleConfiguration_h_
+#define liblldb_CppModuleConfiguration_h_
+
+#include <lldb/Core/FileSpecList.h>
+#include <llvm/Support/Regex.h>
+
+namespace lldb_private {
+
+/// A Clang configuration when importing C++ modules.
+///
+/// Includes a list of include paths that should be used when importing
+/// and a list of modules that can be imported. Currently only used when
+/// importing the 'std' module and its dependencies.
+class CppModuleConfiguration {
+ /// Utility class for a path that can only be set once.
+ class SetOncePath {
+ std::string m_path;
+ bool m_valid = false;
+ /// True iff this path hasn't been set yet.
+ bool m_first = true;
+
+ public:
+ /// Try setting the path. Returns true if the path was set and false if
+ /// the path was already set.
+ LLVM_NODISCARD bool TrySet(llvm::StringRef path);
+ /// Return the path if there is one.
+ std::string Get() const {
+ assert(m_valid && "Called Get() on an invalid SetOncePath?");
+ return m_path;
+ }
+ /// Returns true iff this path was set exactly once so far.
+ bool Valid() const { return m_valid; }
+ };
+
+ /// If valid, the include path used for the std module.
+ SetOncePath m_std_inc;
+ /// If valid, the include path to the C library (e.g. /usr/include).
+ SetOncePath m_c_inc;
+ /// The Clang resource include path for this configuration.
+ std::string m_resource_inc;
+
+ std::vector<std::string> m_include_dirs;
+ std::vector<std::string> m_imported_modules;
+
+ /// Analyze a given source file to build the current configuration.
+ /// Returns false iff there was a fatal error that makes analyzing any
+ /// further files pointless as the configuration is now invalid.
+ bool analyzeFile(const FileSpec &f);
+
+public:
+ /// Creates a configuraiton by analyzing the given list of used source files.
+ ///
+ /// Currently only looks at the used paths and doesn't actually access the
+ /// files on the disk.
+ explicit CppModuleConfiguration(const FileSpecList &support_files);
+ /// Creates an empty and invalid configuration.
+ CppModuleConfiguration() {}
+
+ /// Returns true iff this is a valid configuration that can be used to
+ /// load and compile modules.
+ bool hasValidConfig();
+
+ /// Returns a list of include directories that should be used when using this
+ /// configuration (e.g. {"/usr/include", "/usr/include/c++/v1"}).
+ llvm::ArrayRef<std::string> GetIncludeDirs() const { return m_include_dirs; }
+
+ /// Returns a list of (top level) modules that should be imported when using
+ /// this configuration (e.g. {"std"}).
+ llvm::ArrayRef<std::string> GetImportedModules() const {
+ return m_imported_modules;
+ }
+};
+
+} // namespace lldb_private
+
+#endif