What else to log

[Arie Shoshani <arie@math.lbl.gov>]

Doug,

I saw in the minutes of the last conference call that you 
said:

"It will be good to think about what information might be 
available in the iterator that could be added to these plots 
as well."

I discussed this with Luis and here is what we think:

When we plot log information now, we sometimes see long 
processing time.  See for example:
http://gizmo.lbl.gov/sm/netlogger.images/oct4th_run1.gif

The long purple lines show that there is a long delay 
between the time that a file was given to the EI 
(file_pushed) and the time it started to use it 
(file_retrieve).  In addition, there are unusually long 
processing times for each file after it is retrieved.  From 
this plot alone we have no idea why.

Thus, I think it will be useful to log what the EI is doing: 
the time that each event is read and passed to the User Code 
(UC), the time that the UC is done processing each event, 
etc.  This will be more important to know when events are 
being passed over the network to Linux boxes.

As to the plotting of such information, we think it may be 
possible to put that on the same netlogger plot, but the 
scale of event processing is much smaller that file caching, 
so we may want to display that on a separate plot.

Comment?

Arie.