traced_probes: fix tid/pid incremental mapping and reduce overhead

This CL:
1) Fixes an architectural bug related to the way tid/pid mapping is
   established. The current proto structure, in fact, is strictly
   hierarchical and requires that a process and all its threads are
   dumped together. This has two problems: (i) it either requires to
   always dump all threads for a process or re-dump the process
   record for each thread; (ii) doesn't deal with the fact that new
   threads can be spawned after the initial dump.
2) Fixes a performance bug, reducing traced_probes cpu usage by more
   than 30% when process metadata collection is enabled.
3) Removes unused/forked code and puts everything into
   process_stats_data_source.cc.

In terms of risk this CL should be rather safe. The entire
process_stats_data_source.cc is an optional module and can be
disabled at any point through the TraceConfig.

Bug: b/78453224
Bug: b/78106582
Bug: b/77316877
Change-Id: I4592c9c0317c6f66b68357684ce9cd0577c9bdb5
14 files changed
tree: ae02c35b199d40ed837d99bf0a6940881bc7b3ad
  1. build_overrides/
  2. buildtools/
  3. debian/
  4. docs/
  5. gn/
  6. include/
  7. infra/
  8. protos/
  9. src/
  10. test/
  11. tools/
  12. .clang-format
  13. .gitignore
  14. .gn
  15. .travis.yml
  16. Android.bp
  17. Android.bp.extras
  18. BUILD.gn
  19. codereview.settings
  20. MODULE_LICENSE_APACHE2
  21. NOTICE
  22. OWNERS
  23. perfetto.rc
  24. PRESUBMIT.py
  25. README.chromium
  26. README.md
README.md

Perfetto - Performance instrumentation and tracing

Perfetto is an open-source project for performance instrumentation and tracing of Linux/Android/Chrome platforms and user-space apps.
It consists of:

A portable, high efficiency, user-space tracing library
designed for tracing of multi-process systems, based on zero-alloc zero-copy zero-syscall (on fast-paths) writing of protobufs over shared memory.

OS-wide Linux/Android probes for platform debugging

  • Kernel tracing: a daemon that converts Kernel Ftrace events into API-stable protobufs, on device, with low overhead.
  • I/O tracing
  • Many new probes coming soon: heap profiling, perf sampling, syscall tracing.

Web-based frontend
A UI for inspection and analysis of traces (coming soon).

Batch processing of traces
A python / C++ (TBD) library for trace-based metrics (coming soon).

Perfetto Stack

Goals

Perfetto is building the next-gen unified tracing ecosystem for:

  • Android platform tracing (Systrace)
  • Chrome platform tracing (chrome://tracing)
  • App-defined user-space tracing (including support for non-Android apps).

The goal is to create an open, portable and developer friendly tracing ecosystem for app and platform performance debugging.

Key features

Designed for production
Perfetto's tracing library and daemons are designed for use in production. Privilege isolation is a key design goal:

  • The interface for writing trace events are decoupled from the interface for read-back and control and can be subjected to different ACLs.
  • Despite being based on shared memory, Perfetto is designed to prevent cross-talk between data sources, even in case of arbitrary code execution (memory is shared point-to-point, memory is never shared between processes).
  • Perfetto daemons are designed following to the principle of least privilege, in order to allow strong sandboxing (via SELinux on Android).

See docs/security-model.md for more details.

Long traces
Pefetto aims at supporting hours-long / O(100GB) traces, both in terms of recording backend and UI frontend.

Interoperability
Perfetto traces (output) and configuration (input) consists of protobuf messages, in order to allow interoperability with several languages.

See docs/trace-format.md for more details.

Composability
As Perfetto is designed both for OS-level tracing and app-level tracing, its design allows to compose several instances of the Perfetto tracing library, allowing to nest multiple layers of tracing and drive then with the same frontend. This allows powerful blending of app-specific and OS-wide trace events. See docs/multi-layer-tracing.md for more details.

Portability
The only dependencies of Perfetto's tracing libraries are C++11 and Protobuf lite (plus google-test, google-benchmark, libprotobuf-full for testing).

Extensibility
Perfetto allows third parties to defined their own protobufs for:

Allowing apps to define their own strongly-typed input and output schema. See docs/trace-format.md for more details.

Docs