Aaron Klotz at Mozilla

My adventures as a member of Mozilla's Performance Team

Detecting Main Thread I/O With SPS

| Comments

In bug 867762 I am landing I/O interpose facilities for SPS. This feature will allow main thread I/O to be displayed in the profiler UI.

There are a few components to this patch. Classes that implement the IOInterposerModule interface are responsible for hooking into an arbitrary I/O facility and calling into IOInterposer with those events.

The IOInterposer receives those events and then dispatches them to any registered observers. For the purposes of SPS there is only one observer, ProfilerIOInterposeObserver, however I wrote this keeping in mind that we may eventually want to use the I/O interpose facilities for other purposes (such as shutdown write poisioning).

For bug 867762 I have implemented two modules. NSPRInterposer hooks into NSPR file I/O and generates events whenever a PR_Read, PR_Write, or PR_Sync occurs. The second module, SQLiteInterposer, leverages the SQLite VFS that we use for telemetry to tap into reads, writes, and fsyncs generated by SQLite. In the future we expect to expand this into several additional modules. Some ideas include a module that uses Event Tracing for Windows to read events from the Windows kernel and a module that interposes calls to read, write, and fsync on Linux.

On the observer side of things, for now we simply insert a marker into the profiler timeline whenever main thread I/O is reported. Here is a sample screenshot of the markers in action (the pink stuff for readers who are unfamiliar with SPS markers):

In bug 867757 (under review) this will become more sophisticated, as SPS will immediately sample the callstack of the thread whose I/O has been intercepted. This annotated stack will be inserted directly into the timeline.

There will need to be some UI changes for this to be presented in a reasonable way, but with both existing patches applied the data is already useful. By firing up Cleopatra and filtering on either NSPRInterposer or SQLiteInterposer, you can isolate and view the main thread I/O stacks:

Hopefully these patches will prove to be beneficial to our efforts to eliminate main thread I/O.

Comments