Perfetto in Node.js

[kjin]

Perfetto (source) is an open-source tracing system that has been adopted in Android and Chromium. We are proposing to introduce it into Node as well; in short, it would replace some of Node's current implementation for writing trace events.

What it is

The website has more detail, but for our purposes Perfetto is a (C++) library that defines interactions between producers and consumers of trace events (and other instrumentation data). Producers and consumers don't communicate directly but through a central "tracing service". Trace events are flushed from producers to the tracing service via a shared memory buffer.

Perfetto supports, and provides an implementation for, communication between producers/consumers and the tracing service over IPC, via protobufs. I believe this is largely meant for OS-wide trace collection. For Node, we would most likely not use IPC, and just have everything be in-process, which Perfetto also supports.

The trace events implementation in Chromium is currently using Perfetto.

How Node benefits from it

Perfetto would replace the current mechanism for buffering and writing trace events. It would be arguably more stable than the current trace event writer implementation in Node, and also provide a standardized set of config options (such as buffer size, file options) for trace events. It would also be easier to reason about having multiple consumers for trace events (writing to file thru command line options; inspector; JS API) as this is intrinsic to Perfetto's design.

The current system for creating trace events would not change; the trace event macros as they are now would remain the same.

Current plans

I'm currently working on a demo, and can talk and answer questions at the Diagnostics meeting tomorrow and in future sessions.

[hashseed]

Do you think it makes sense to give a short presentation at the Diagnostics summt?

[kjin]

@hashseed Yep! I won't be able to make it in person but should be able to present remotely at that time.

[mcollina]

Definitely, this is important.

Can you please include in your presentation:

1. a timeline for replacing trace_events with perfetto, which Node version would you target?
2. can trace_events and perfetto be supported at the same time?
3. how would interact with async_hooks and in general custom/user events?
4. comparstive benchmarks if there is time, as our trace event implementation is fast.

[joyeecheung]

Thanks for the explanation, looking forward to your presentation!

There were descriptions about Perfetto being able to address some threading issues that comes from the current trace events implementation. I am curious, how would this framework integrate with the workers? Will it enable us to control e.g. configurations of trace events in workers? Currently only the main thread is allowed to do that, because the tracing controller is global and per-process, but if Perfetto is designed for OS-wide trace collection and provides communication with IPC, maybe there is a different story for workers?

[mhdawson]

What platforms does Perfetto support?

[kjin]

@mcollina Thanks for the pointers! I don't anticipate any change for adding trace events, so I don't think there would be any new/changed interactions with async_hooks.

@joyeecheung I believe there is a task queue abstraction in Perfetto that would be good for managing trace events coming from different threads, I will be looking into it more over the next few days.

@mhdawson It seems like wherever Chromium can run, so there might be issues on non-Windows/macOS/Linux builds. Maybe we could make it conditional, or otherwise see how much work it takes to add Perfetto support for other platforms.

[cjihrig]

2. can trace_events and perfetto be supported at the same time?

and

there might be issues on non-Windows/macOS/Linux builds

If trace_events and perfetto can coexist, then feel free to ignore the following the question.

If trace_events and perfetto can't both be used at the same time, wouldn't platform support for perfetto be an issue? trace_events are currently targeting being a tier 1 supported tool, and platform support would bump perfetto to at least tier 2.

[ofrobots]

I think of Trace Events as the API we use in core, and expose via the command line flags. I view Perfetto to be the implementation details of how it is the trace buffer is implemented and managed. They will co-exist.

[mhdawson]

@ofrobots even if they co-exist, having a different implementation on some platforms does not necessarily make sense to me as it may end up being neglected.

I think we should include seeing if it builds on all of the Node.js supported platforms as part of the integration. I suspect in many cases the additional require changes might not be that significant. We would want a path to getting those upstreamed.

[jasnell]

major +1 on seeing this move forward.

a timeline for replacing trace_events with perfetto, which Node version would you target?
can trace_events and perfetto be supported at the same time?

As I understand it, perfetto is a completely replacement for the underlying trace events implementation. All of the existing macro and API hooks continue to work, but instead of directing the output to the existing JSON writer or inspector target, it writes to perfetto. This means that how we instrument for trace events does not change, but how we consume them would. This would have an impact for tools like clinicjs but there's a massive upside that makes it worth it.

how would interact with async_hooks and in general custom/user events?

As @ofrobots mentions, there should be no change here, at least on the event production side.

[mmarchini]

I think it would be interesting to understand (from a user perspective) what are the advantages of using Perfetto. What can we do with Perfetto that we can't do with the current trace events implementation? Any chance we can have a Perfetto demonstration in the Summit?

[hashseed]

Not sure whether a working prototype will be ready. @kjin can probably speak to that. But I expect that the presentation on Perfetto will bring clarity about the benefits.

[mmarchini]

A demonstration on what Perfetto can do on Chromium would be enlightening too.

[ofrobots]

I think it would be interesting to understand (from a user perspective) what are the advantages of using Perfetto.

Perfetto is an implementation detail on how the trace buffers are handled, so from the user-perspective, the only visible change will be 'less crashy'.

[kjin]

Presentation: https://docs.google.com/presentation/d/1iS3f4BfOmbdCO2AXFIi9Sii6jDQy4za2CSN2cuVEedM
Architecture diagram: https://drive.google.com/file/d/1oEZ5MnsJbp1_SIDYGLm0xQAfRvibsxGz (open with https://draw.io, there are four pages)
Code: https:/kjin/node/tree/perfetto (cannot be built directly, must be built via node-ci: https:/kjin/node-ci/tree/perfetto)

Will get back to this thread on:

• binary size.
• whether we can support unimplemented macros.
• role of TraceObject.
• what mvp is needed to check platform compatibility.

[kjin]

@mhdawson Re: binary size -- I saw the binary size rise from ~47.5 to ~48.8 MB in my local build after adding Perfetto. For the record, the Perfetto CLI (a separate build target in Perfetto) is ~1.15MB.

@jasnell IIUC when you were mentioning the TraceObject class, were you referring to the rigidity of the current default V8 tracing controller in that TraceObject cannot dynamically include additional fields, and so might not be able to support arbitrary additional trace event fields? If so I believe using Perfetto's writing implementation might be able to help there -- as protobuf messages written by the Perfetto Trace Writer can contain arbitrary new fields.

[kjin]

A working repo has been created: https:/nodejs/perfetto-nodejs-wip

[mhdawson]

@kjin any update on this front?

[kjin]

Yep, the Perfetto API has changed significantly since I opened the issue (to better suit the Node use case, in fact), which is why I haven't done anything in the way of opening a PR yet. But we (incl. the V8 and Perfetto teams at Google) are still actively working on this. Apologies for the lack of updates!

[mmarchini]

Removing from the Diagnostics agenda since there has been no updates in the last few months. Feel free to re-add to the agenda if anyone wants to discuss this in the next meeting.

[kjin]

Hey @mmarchini (and others), sorry for the silence here. Unfortunately I don't have the bandwidth to move forward with this project at this point.

If anybody has an interest in picking this up, please let me know and I can point you towards the right people to contact. Thanks!

[jasnell]

Absolutely want to see this move forward but no bandwidth on my end either. I'll ask around a few folks. It would be helpful, @kjin, to get a quick summary on where the work left off, what is still needed to do, and how one would get started.

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.