More profiling goodies

Time for another update! Most of the changes happened behind the scenes. I wasted a few days because of my naive assumption that the text rendering process presented in the previous post actually works. It seems to, but apparently it causes some GTK calls to randomly hang. And I do mean randomly. The error doesn’t even seem to be related to text rendering per se, but it stops occurring as soon as I remove the call to drawPixBuf. This is something to look into later, but there’s no point pursuing it at the moment.

The changes on the surface are not really spectacular, but they might be useful. After giving up on my text rendering ambitions I added cost centre lists next to the graphs, and also the ability to toggle accumulated and separate views. The lists can be sorted according to the total cost for each cost centre, and as we hover over the graph, the cost centre under the cursor is highlighted on the list. See for yourself:



The extra options can be found under the fourth button next to the left arrow in the graph headers.

As for library interfaces, one major change is the removal of the network protocol for the most part. My original idea was to create a remote controllable graphing application, but I ended up factoring graph rendering routines into a library, which removed the need for serialising various commands. The other change is the introduction of the Stats module, which makes it possible to efficiently query the heap profile in various ways. In particular, you can get the maximum individual cost, the maximum total cost (at a given moment) and the integral of any cost centre for any time range, and you can also extract samples collected during a given time interval. All this requires a preprocessing phase which can be performed in nearly linear time.

From now on, I’ll be concentrating on the interface of the history manager, and after adding some features I’ll also write a tutorial/manual for the applications.

Introducing the heap profile manager

Well-well, it’s been almost two weeks since the last release, so I decided to show you the beginnings of the heap profile viewing application. After thinking about the alternatives I decided to follow in the steps of others who make core tools and use gtk2hs for the interface. Since I had a working grapher in OpenGL, it was also straightforward to keep it. I started out as a complete beginner in gtk2hs last week, and it proved to be one of the smoothest library learning experiences so far. Even getting several OpenGL drawing areas to work simultaneously went without a hitch. The UI started out as something more complex, but after some experimentation I arrived at the present design:

You can load several graphs in each column (note that multiple selection is enabled in the file open dialog), and move them left and right between columns as you wish. Columns can also be created and destroyed at will. This initial version doesn’t have any more interaction, e.g. you cannot zoom in on areas you’re interested in. Consequently, graphs with an occasional high spike can be unpleasant to look at, as illustrated by the example on the right.

The next step is obviously to add the features that make comparison easier, like displaying scales, displaying identical cost centres with identical colours, adding zooming and panning, comparing a given cost centre from separate runs on the same graph and so on. Also, the profile loader could be probably speeded up a bit by not building a map from times to samples if we convert it into a list anyway.

If all this works okay, I can also add some export plugins (e.g. ps and svg) to finally retire hp2ps.

Oh, the button next to the left arrow doesn’t do anything yet.

Remote profiling at your fingertips

Yes, it’s time for another code release after a long hiatus. You can grab the source from the repository, and it should even be able to compile without any major hiccup. The grapher is now capable of connecting to a profile relay server that can broadcast the heap profile of its associated process on the fly. Stopping either the process, the server or a grapher client is handled gracefully. Thanks to decoupling the grapher and the process in the remote case, it is now possible to attach an observer to a program that was started earlier. Here’s the proof:

It’s hopefully obvious that the clients were attached at increasing times from top to bottom, and I flipped the mode of the middle one just for the fun of it.

Unfortunately, even though the code to achieve all this is quite small (hey, thanks Haskell!), getting it to work at a reasonable level of stability took longer than expected. Therefore, instead of adding remote control features to the grapher right away, I’ll start preparation for the history manager, otherwise I might not be able to get it working before the final GSoC deadline.

More colourful graphs

In order to make it easier to compare cost centres, I added a cumulative view that works the same way as hp2ps. Generating appropriate colours is tricky when you don’t know the data in advance, but the result is at least satisfactory for the time being. Hovering over an area causes the name of the respective cost centre to be displayed in the top left corner, as illustrated by the following screenshots:



The vertical magnification of the graph depends on the last 50 samples, so a huge spike will squeeze the diagram only for a limited amount of time, as you can see on the screenshots too.

Of course, the line graphs are also available, but the two can’t be toggled during a single run yet, so here’s a picture from another one (the lines are broken at places, because the Gnome screenshotter can’t seem to keep up with the animation):



What’s next? The first thing to do is to optimise graphing, because everything is recalculated for each frame in the current snapshot, and it doesn’t take long for the visualiser to stress the CPU more than the program we’re profiling... Pushing everything into display lists should take care of that. The next step is to add the means to stop the animation and look around in the graph, and also the option to display only the last portion while animating the graph.

When the grapher works fine as a standalone application, I’ll start extending it with remote control capabilities as outlined on the project wiki.

The first graphs

Well, after a few days of other distractions I got back to hp2any and implemented a simplistic graphing utility on top of the core library. Here is the first test run featuring a little game I’m working on in my similarly little spare time:



Oops, of course I forgot about the evils of laziness right away... I removed some debug text, but still calculate the values that would be displayed, which are therefore never evaluated and hold on to various big structures from every past frame. The situation is much better after adding an exclamation mark at the right spot:



There’s no public code yet, but I hope to get it in a much better shape in a few days.