12-Jun-2007 Sakai Metrics at Indiana University1 That Which is Monitored, Improves Dan McKee & Megan May

12-Jun-2007Sakai Metrics at Indiana University2 At Indiana University we believe, That Which is Monitored, Improves How does Indiana do this? Extended Performance Metrics [EPM] Objective today: to present three such metric categories that are in use today Course site tool usage metrics Project site tool usage metrics Database performance metrics

12-Jun-2007Sakai Metrics at Indiana University3 Sakai at Indiana University Background Sakai is called Oncourse CL at Indiana University Supports over a 100,00 users Measuring Migration It runs on a Virtualized IBM AIX (Unix) Server Its Database Management System is Oracle 10g Size is approximately 60 Gbs The typical number of CL Sessions at any one point in time is around 5,000

12-Jun-2007Sakai Metrics at Indiana University4 EPM*: Sakai Tool Usage Metrics Background Designed to be Database Platform Independent Based upon data contained within the Sakai Application/Database Metrics are populated via PERL scripts Data Structures are: Designed to ease reporting and sql query construction Compatible with a variety of SQL Tools Aqua Data Studio, DBVisualizer, SQL Developer,SQL4X and others Tool Usage Metric DDL is available at: (see file epm_tool_proj_ _ddl.sql) SQL scripts and Perl scripts used to build and maintain EPM are available at the above link. (* Extended Performance Metrics)

12-Jun-2007Sakai Metrics at Indiana University5 EPM: Tool Usage Metrics Fact Table

12-Jun-2007Sakai Metrics at Indiana University6 EPM: Tool Usage Dimensions

12-Jun-2007Sakai Metrics at Indiana University7 EPM:Tool Usage Metrics ER* Diagram * ER - Entity Relational

12-Jun-2007Sakai Metrics at Indiana University8 EPM: SQL to Populate Tool Usage Example SQL used to extract Tool Usage SQL to Extract Tool Usage for Syllabus [SYL] select substr(site_id,1,2)TM, '20'||substr(site_id,3,2)YR, substr(site_id,6,2)CMP, count(distinct(Site_Id)) CNT from onc.sakai_site_property where site_id like 'FA06%’ and SITE_ID in (select CONTEXTID from onc.sakai_syllabus_item where ID in (select surrogatekey from onc.sakai_syllabus_data where not (title is null))) group by substr(site_id,1,2),substr(site_id,3,2),substr(site_id,6,2); File containing complete sql for extracting Tool usage is available at: (see file epm_tool_upd_qry.sql) Perl script used to extract and populate the Tool Usage Metrics is available at the above link (see file update_epm_tool_ft.pl)

12-Jun-2007Sakai Metrics at Indiana University9 Tool Usage by Semester for All Course Sites FA05SP06SU06FA06 % Change FA05 to FA06 Announcement % Assignment % Calendar % Chat % Discussion % Drop Boxes % Archive % Gradebook % Message Center % PM % DF % Podcasts---21N/A Post 'Em % Resources % Site Info Groups % Syllabus % Test & Surveys CL % Test & Survey (original)---385N/A # Active % Sample Report

12-Jun-2007Sakai Metrics at Indiana University10 EPM: Usage Charts

12-Jun-2007Sakai Metrics at Indiana University11 EPM: How is the migration progressing? Course Section Adoption % Using CMS% Using original Oncourse% using Oncourse CL FA0543%71%29% SP0636%60%40% SU0622%61%39% FA0637%47%53% SP0736%41%59%

12-Jun-2007Sakai Metrics at Indiana University12 EPM: Tool Usage Reports Fall 06 by Campus BLCOEAINKOSBNWSE Announcement Assignment Calendar Chat Discussion Drop Boxes Archive Gradebook Message Center PM DF Podcasts Post 'Em Resources Site Info Groups Syllabus Test & Surveys (CL) Test & Survey (original) Wiki # Active

12-Jun-2007Sakai Metrics at Indiana University13 EPM: Tool Usage Charts Fall 06 Campus Usage Breakdown BLCOEAINKONWSBSE % Using CMS29%56%34%46%36%30%31%38% % Using original28%49%70%64%36%40%53%61% % using CL72%51%30%36%64%60%47%39%

12-Jun-2007Sakai Metrics at Indiana University14 EPM: Sakai Project Metrics Background Designed to be Database Platform Independent Based upon data contained within the Sakai Application/Database Metrics are populated via PERL scripts. Data Structures are: Designed to ease reporting and sql query construction Compatible with a variety of SQL Tools Aqua Data Studio, DB Visualizer, SQL Developer, SQL4X, and others Project Count Metric DDL is available at: (see file epm_tool_proj_ _ddl.sql ) SQL scripts and Perl scripts used to build and maintain EPM are available at the above link

12-Jun-2007Sakai Metrics at Indiana University15 EPM: Project Metrics Fact Table

12-Jun-2007Sakai Metrics at Indiana University16 EPM: Project Metrics Dimensions

12-Jun-2007Sakai Metrics at Indiana University17 EPM:Project Metrics ER* Diagram * ER - Entity Relational

12-Jun-2007Sakai Metrics at Indiana University18 EPM: SQL to Populate Project Metrics Example SQL used to extract Project Counts Project Count for Syllabus [SYL] select count (site_id) PRJ from onc.sakai_site where type = 'project' and SITE_ID in (select distinct (CONTEXTID) from onc.sakai_syllabus_item where ID in (select surrogatekey from onc.sakai_syllabus_data where not (title is null))); File containing complete sql is available at: (see file epm_proj_upd_qry.sql ) Perl script used to extract and populate the Project Count Metrics is available at the above link (see file update_epm_proj_ft.pl )

12-Jun-2007Sakai Metrics at Indiana University19 EPM: Project Metric Report and Chart Project Sites Announcement479 Assignment159 Calendar297 Chat151 Discussion170 Drop Boxes50 Archive189 Gradebook102 Message Center250 PM212 DF112 Podcasts13 Post 'Em17 Resources528 Site Info Groups75 Syllabus145 Test & Surveys116 Wiki113

12-Jun-2007Sakai Metrics at Indiana University20 EPM: Oracle Database Performance Metrics Background Oracle Database Performance Metrics are NOT Database Platform Independent -- Oracle 10g Specific Based upon data contained within: Oracle Automatic Workload Repository [AWR] Other External Sources Database Extended Performance Metrics are populated via Perl. Data Structures are: NOT Specifically designed to ease reporting and sql query construction (these structures are supplied by Oracle not Sakai) compatible with a variety of SQL Tools Aqua Data Studio DB Visualizer SQL Developer SQL4X And others Oracle Database EPM DDL is available at: (see file epm_ora_db_ddl.sql) SQL scripts, Perl scripts and documentation used to build and maintain Database EPM are available at the above link.

12-Jun-2007Sakai Metrics at Indiana University21 EPM: Oracle Database Metrics from Oracle’s AWR include: Buffer Cache Hit Ratio -- AWR CPU Usage Per Sec -- AWR CPU Usage Per Txn (server) Current Logons Count Cursor Cache Hit Ratio Database CPU Time Ratio Database Time Spent Waiting Host CPU Utilization (%) Library Cache Hit Ratio PGA Cache Hit % RAM USAGE Response Time Per Txn RowCache Hit Ratio SQL Response Time (%) Shared Pool Free % Sorts in Memory

12-Jun-2007Sakai Metrics at Indiana University22 EPM: Oracle Database Metrics Whose Source is External to Oracle’s AWR include: (Metrics populated from external sources via Perl scripts) CL Sessions -- Sakai Database CL User Count -- Sakai Database Size of Database -- Sakai Database Percent Used -- Sakai Database Space Used -- Sakai Database CPU USAGE -- System Administration / AWR RAM USAGE -- System Administration / AWR

12-Jun-2007Sakai Metrics at Indiana University23 EPM:Database Performance ER* Diagram * ER - Entity Relational

12-Jun-2007Sakai Metrics at Indiana University24 EPM: SQL to Populate Database Performance Metrics Example SQL to Oracle’s Response Time per Transaction select metric_name, to_char(begin_time,'MM-DD HH24') HR, round((minval/10),1) "Min", round((maxval/10),1)"Max", round((average/10),1) "Avg” from sys.dba_hist_sysmetric_summary where begin_time >= sysdate - 2 and begin_time <= sysdate and metric_id = 2109 order by begin_time; Perl script used to extract and populate the Tool Usage Metrics is available at: (see files extended_perf_metric_update.pl and extended_perf_metric_usercnt.pl )

12-Jun-2007Sakai Metrics at Indiana University25 EPM: Database Performance Reports Example Response Time Report MetricHourMinMaxAvg Response Per Txn Response Per Txn Response Per Txn Response Per Txn Response Per Txn Response Per Txn

12-Jun-2007Sakai Metrics at Indiana University26 Example Response Time Chart EPM: Database Performance Charts

12-Jun-2007Sakai Metrics at Indiana University27 Available to Oracle Enterprise Manager [Grid] Via Database Links Views from the Sakai Database into your OEM Repository Contact us if you have such an interest: EPM: Database Performance Charts

12-Jun-2007Sakai Metrics at Indiana University28 That Which is Monitored, Improves Questions?