Monday, August 31, 2009

Oprofile for IO bound apps

I am using Oprofile to look into the performance of a Mysql Storage engine. Upon reading further into the oprofile docs I find:

"OProfile is oriented to finding problems with CPU-limited processes. OProfile does not identify processes that are asleep because they are waiting on locks or for some other event to occur (for example an I/O device to finish an operation)."

That is kind of a drag since the results I am expecting to see include the largest bottlenecks for IO and waiting for locks.

From my experience with gprof I seem to remember the same thing: CPU profiling primarily.

Does any one know of a good off the shelf tool that can integrate CPU profiling with IO? I don't even expect to see anything out there that can include time waiting for locks because it is not generic.

4 comments:

  1. Definitely interested in the results of this. I'm not a DBA, but do sysadmin-for-hire type stuff for customers, and more and more frequently I'm having to help customers identify why their "Database is slow". 75% of the time it's fairly obvious (obvious bad queries), probably 25% of the time its a lot less obvious and some form of in depth profiling would be helpful.

    ReplyDelete
  2. Ivan,

    I am facing the same problem. Currently I am instrumenting the mysql code to understand I/O bottlenecks. It is painful but works.

    -Shirish

    ReplyDelete
  3. We believe instrumenting the code is the only way toward satisfactory in-depth profiling. MySQL's "Performance Schema" has instrumentation and allows its retrieval via SQL tables. It's available in source form on launchpad.

    Peter Gulutzan
    Sun Microsystems / MySQL

    ReplyDelete
  4. dtrace and/or poor man's contention profiler ;-))

    ReplyDelete