1. Offline Performance Monitoring for Linux Abhishek Shukla

2. File Format for Saving data Executables Writing to File Reading from File Showing statistics on a graphical user interface What are limitations of this software ? What still it to be done ? Usability of this software Outline

3. Offline monitoring is just keeping a close eye on what happened in past. Which magical stick does one need to wave to get this done ? I want to know the past in which I lived. But why don’t I know my past which I lived myself ? Well, I was sleeping that time and could not write in my diary. What should I do ? May be I can plan my future better after knowing my mistakes in past. Offline Monitoring

4. Now I’ll write about my present in my diary In future, I’ll able to know all details of my past But it was so simple ? Well, still most people doesn’t do it. How to do it for computers, they can do exactly what they are told to? Just write everything about present in a persistent space and read it later when any question about past is asked Offline Monitoring (cont..)

5. Why ? It’s better when things are organized rather than they are randomly located here and there. But I don’t like having complex rules to follow Fine, this file format doesn’t look very complex  File Header  First Sample  Second Sample  Third Sample ( this goes on & on ) File Format

6. One value to verify that it is smartertop generated file Size of system related performance counters in Byte Size of process related performance counters for one process in Byte Size of File Header to verify that file is not generated by smartertop which has different size of File Header. Different version might have this field different Record Header length Clock Interrupts per second Some information about this system Size of memory page in bytes OS version information File Header

7. Snapshot of all current activities. But I won’t need details of all activities in future. Then don’t store those activities Store those things which you might need in future. Format of one sample Header for sample Compressed system related statistics Compressed process related statistics What are samples ?

8. Current Time Size of compressed system related counters for this sample Size of compressed process related counters for this sample Interval since last sample (in seconds) Number of Process in this sample Header for Sample

9. Memory Stats Interface Stats CPU Stats Disk Stats Network Stats System Related Counters

10. CPU stats Disk Stats Memory Stats Few other things like scheduling policy, state of process etc. Process Related Counters

11. There are two actually smartertop smartertop-gui What does smartertop do? Writing details to file What does smartertop-gui do? Reading details from file and showing them in a graphical user interface Executables

12. After each d seconds, many functions are called which read many counters from many files fill them in an array (of structure) Compute difference between current counter and previous counters Write this in a file Smarter Top

13. Snapshot of all current counters are taken after each d seconds First snapshot of system level counters is taken by reading following files general statistics from /proc/stat load average values from /proc/loadavg virtual memory statistics from /proc/vmstat Memory related statistics from /proc/meminfo interface statistics from /proc/net/dev IP v4 statistics from /proc/net/snmp IP v6 statistics from /proc/net/snmp6 Disk statistics from /proc/diskstats Code Walk Through

14. Then calculate difference between current counters and last counters ( system level) Snapshot of process level counters is taken by reading following files general statistics from /proc/ /stat /proc/ /status /proc/ /io get state of threads from /proc/ /task/ /stat full command line from /proc/ /cmdline Continued…

15. But this way we got counters of processes which have not exited till the time snapshot was taken. What about processes which exited in last cycle ? For this, read process records from the account-file, which are written since the previous cycle On Ubuntu, account file is /var/log/account/pacct On other distribution it could be /var/log/pacct /var/account/pacct Continued…

16. First I checked which file is present among them Then verify whether it is really in use by forking a child process which immediately finishes Continued…

17. Then calculate difference between current counters and last counters Compress system level counters by using compress function of zlib Compress process level counters by using compress function of zlib Write compressed system status structure to file Write compressed list of process status structure to file DONE WITH WRITING TO FILE Continued…

18. XML UI definition are specified in smartertop.glade file GtkBuilder reads this textual description of a user interface and instantiates the described objects. It holds a reference to all objects that it has constructed and drops them when it is finalized. gtk_builder_get_object() is used to access widgets by names assigned to them in UI definition okay, now we are done with how GUI was made Smartet Top GUI

19. Now let's see how contents are being fed to UI Where is all data ? All data is available in the file we created using smartertop Open that file, do some checks to verify whether this file is really created by “smartertop”. Now read one sample, send it to a function which will show it in a gui This sample includes system level counters as well as process level counters Keep on doing this for all samples which comes in specified range on respective events Continued…

20. Files are present day wise, filename are of smartertop_YYYYMMDD format Time windows across days are not supported Nethog facility is not implemented Process wise network usage details Limitation

21. Make Last Page Empty Overcome those limitations Better User Interface What still is to be done ?

22. People and organizations who need to monitor their resources offline Who want to analyze things about resources Who’ll use this software

23. My Email-ID abhishek_shukla@students.iiit.ac.in Thank You