Path Profiling and LightWeight Profiling

Path Profiling

Path profiling is a means of collecting an increased amount of context about the actual behavior of an application, beyond simple execution counts on a given statement or control flow edge. That is, if a program executes the following sequnce of functions, statements, or basic-blocks:A B C A B D B, the paths are: ABC, ABD, B. The completion of a cycle denotes the beginning of a new path.
Use of path profiles can yield significant benefits to developers in terms of:
Personal investigations into Path Profiling: Larus' Whole Program Paths thoroughly captured my attention to the point of inspiring me to attempt something similar, but via a more dynamic approach. It seemed to me that Larus' approach of instrumenting the application in such a way as to precompute the possible paths, collecting the complete path trace, and then post-processing it would not scale well for long-running applications. So I set out to try to achieve a dynamic variant. The first attempt was as an ATOM tool for Alpha, but had a variety of problems. The sources for these (and other) ATOM tools are
here. Eventually, I was able to implement a more dynamic system (in C as a PIN tool) - the basic approach is to observe that a path can be represented as a cycle of Basic Blocks, and the addresses of the Bbs would be sufficient to construct the paths. This would be very lightweight in terms of instrumentation-time overhead. The dynamic compression portion of the problem was an additional challenge, as it would seem to be desirable to carry and save enough context to enable playback starting at an approximately arbitrary instruction without having to decompress the entire trace, or decompress starting from the beginning of the trace. The resultant content (and some additional notes) are:
As part of an exercise to significantly improve my C++ skills, a from-scratch rewrite was undertaken of the original tooling, and with the intent of investigating the possibility of capturing entire data address access traces via the same general compression approach. The code for that set of tools is newpathp.tar.gz. With this, I can capture information about the nature of various paths - two of the logfiles from running whetstone are: whet.5266.ipath.log and whet.5266.istrata.log. A subset of SPEC CPU2006 was run with and without the path profiler, three runs each
Overhead for non-temporal path collection:
BenchmarkRef ratioPath profiler ratioOverhead

Futures: If you find this useful for research purposes, I would appreciate hearing from you about it.

LWPtools: LightWeight Profiling

LightWeight Profiling is very low overhead, per-thread, user-space profiling mechanism designed by
AMD. One of the key aspects is the capability of performing value profiling.
LWP is currently implemented in a number of their shipping processors (as of 2013). Until now, there does not seem to have been any sort of wrapper library to simplify development of performance tools based upon this feature, and thus, this page.
This library is licensed under the GPL. Contact me if you wish to discuss the availability of the library under different licensing terms.

Phase Changes by applications

Various research publications have shown that applications change phases during the course of their runs; to a code transformation/optimization system, having more context enables more opportunities for code improvement. Paths are one way to represent these phase changes. I collected some
experimental results via a binary instrumentation toolkit called ATOM a number of years ago. The full page of those ATOM tools and more results. Here is a gif of the phase plot of for some of the hot paths:

Technical References

Various publications have shown that it is possible to capture entire application (istream) traces in a compact, lossless form. Some of the relevant publications include:

Copyright 2004 - 2015 Richard Gorton -
Please Contact me if you wish to use any of the code on this page for commercial purposes.