1. José Paulo Leal | Ricardo Queirós CRACS & INESC-Porto LA Faculdade de Ciências, Universidade do Porto Rua do Campo Alegre, 1021 4169-007 Porto PORTUGAL PExIL Programming Exercises Interoperability Language

2. Outline 1. Introduction 2. Learning Objects Standards 3. PExIL definition  Textual elements  Specification elements  Program elements 4. PExIL validation  Generating a IMS CC package  Describing crimsonHex programming exercises 5. Conclusions and Future Work

3. 1. Introduction  Learning Objects (LO)  container with educational resources and metadata  fundamental for producing, sharing and reusing content  may include exercises to be automatically evaluated  uses IMS QTI to describe questions and tests a LOM extension ideal for simple exercises insufficient for complex exercises IMS CP package imsmanifest.xml Metadata (LOM) Resources (LOM+ QTI) exercise assets

4. 1. Introduction  Complex exercises (programming exercises) require  specialized content description collection of resources test cases, solution programs, exercise descriptions, feedback configuration data compilation and execution lines, time and memory limits  communication with heterogeneous systems Learning Object Repositories (LOR) Evaluation Engines (EE) Integrated Development Environments (IDE)

5. 1. Introduction  Challenges of programming exercises as Learning Objects  interdependent resources  processed in different moments of the exercises’ life-cycle  manual creation using standards is time-consuming error-prone creation selection presentation resolution evaluation

6. assets XML manifest XML manifest 2. Learning Objects Standards  Learning Objects Standards levels:  packaging – IMS CP, ADL SCORM, IMS CC  metadata – IEEE LOM, Dublin Core  exercises – IMS QTI metadata resources resource Exercise description (IMS QTI) PDF, HTML, SWF files

7. 3. PExIL definition  PExIL  Programming Exercise Interoperability Language  consolidate all the data required in the programming exercise life-cycle  XML dialect formalised using a XML Schema  organized in three groups of elements: textual elements specification elements program elements

8. 3. PExIL definition  Textual elements  Textual information (e.g. title, date, challenge)  Used for Presentation selection

9. 3. PExIL definition  Specification elements  set of restrictions used for generating specialized resources test cases (input and respective output file) feedback exercise statement (part)

10. 3. PExIL definition  Example: Given 3 numbers verify that the last number is between the first 2 <feedback when=”$num1>$num2”> Numbers that limit the range can be given in descending order True False INPUTINPUT OUTPUTOUTPUT

11. 3. PExIL definition  Program elements  references to programs as external resources solution program correctors  metadata about those resources compilation and execution line Hints  used for test case generation, evaluation

12. 3. PExIL definition  Interdependence TypeElement Phases SelectionPresentationResolutionEvaluation Textual titlexx creation/authors/authorxx creation/datexx creation/purposexx challengex contextx skeletonxx Specificationinput/specificationxxx output/specificationxxx Program solutionxx correctorx hintsxx

13. 4. PExIL validation  We validate the PExIL definition according to:  its usefulness input of a set of programming exercise life-cycle tools generation of a IMS CC learning object package  its expressiveness ability to define a set of programming exercises in a repository extracted from the UVA on-line judge (EduJudge)

14. 4. PExIL validation  The Generation tool:  uses as input: a valid PExIL instance a program solution file  and generates 1. an exercise description 2. a set of test cases and feedback files 3. a valid IMS CC manifest file. 4. an IMS CC LO package 4 4

15. 4. PExIL validation 1. Exercise description generation...

16. 4. PExIL validation 2. Test cases and feedback generation  the test values:  cover the range limits of all variables – T1-T8 (2 3 = 8)  are constrained by the feedback (at least one) – T5, T6  are generated Randomly – T9, …Tn $num2”>Numbers that … Var.T1T2T3T4T5T6T7T8T9Tn n100001000 RR n2001000 00 RR n3010000 0 0 RR

17. 4. PExIL validation 3. Manifest generation - metadata  IMS CC (Common Cartridge) manifest  PExIL mapped to IEEE LOM using templates Data TypeSchemaElement path Title LOMlomcc:general/lomcc:title PExILexercise/title Date LOMlomcc:lifecycle/lomcc:contribute[lom:role=’Author’]/lom:date PExILexercise/creation/date Author LOMlomcc:lifecycle/lomcc:contribute[lom:role=’Author’]/lom:entity PExILexercise/creation/authors/author/v:VCard/v:fn Purpose LOMlomcc:general/lomcc:coverage PExILexercise/creation/purpose

18. 4. PExIL validation 3. Manifest generation – resources  resource elements references generated assets of the IMS CC package IMS CC manifest resources element

19. 4. PExIL validation  UVA on-line judge (EduJudge) programming exercises  validate the expressiveness of the PExIL definition  cover the requirements (input/output constraints of the exercise) random subset of repository (crimsonHex) 24 exercises – (1% of a total of 2393 exercises)

20. 5. Conclusion and future work  PExIL – a XML dialect for programming exercises  Used in programming exercises’ life-cycle:  to present the exercise description in different formats  to regenerate test cases  to produce feedback to the student  PExIL schema is available for download and testavailable

21. 5. Conclusion and future work  Our plans are  to finish the development of the generator deployable as an IDE plug-in command line  to support PExIL in different classes of tools learning object repositories - crimsonHex contest management systems – Mooshak development support tools – Eclipse, Visual Studio Express learning management systems – Moodle, Sakai

22. Questions? Authors José Paulo Leal zp@dcc.fc.up.pt http://www.dcc.fc.up.pt/~zp zp@dcc.fc.up.pt http://www.dcc.fc.up.pt/~zp Ricardo Queirós ricardo.queiros@eu.ipp.pt http://www.eseig.ipp.pt/docentes/raq ricardo.queiros@eu.ipp.pt http://www.eseig.ipp.pt/docentes/raq Thanks!