2. Development of an Assessment System for Engineering Courses Using Petri-Net Tung-Nan Institute of Technology, Taiwan* Tamkang University, Taiwan** Yu-Hur Chou* Hsin-Yih SHYU**

3. Outline 1.ObjectivesObjectives 2.System ArchitectureSystem Architecture 3.System Analyses and DesignSystem Analyses and Design 4.System Demonstrations (Test Demo.)System Demonstrations (Test Demo.) 5.ConclusionsConclusions

4. Objectives 1.To specify how to design and develop a partial scoring assessment system. 2.This assessment system can make inference diagnosis in order to investigate examinee’s misconceptions and make the reasonable scoring for engineering courses. 3.Reinforced Concrete Design is the objective course.

5. System Architecture

6. System Analyses and Design (1/7) 1.Defining Subject Concepts.Defining Subject Concepts. 2.Defining Subject Calculations.Defining Subject Calculations. 3.Evaluation between Concepts and Calculations.Evaluation between Concepts and Calculations. 4.Construct the Logic Structure of Correct Answer.Construct the Logic Structure of Correct Answer. 5.Partial Scoring Function.Partial Scoring Function. 6.Misconception Evaluation.Misconception Evaluation. RC Beam Analysis (with Single Layer Bar) as example Procedures:

7. System Analyses and Design (2/7) 1. Defining Subject Concepts: Item Concept name Item Concept name 1 Required Resisting Strength 12Parameter of Equivalent Stress Block 2Stress-Strain Diagram (Reinforcement) 13Equivalent Stress Block 3Modulus of Elasticity (Reinforcement) 14Concept of Under reinforced Case 4No. Of Reinforcing Steel15Concept of Over reinforced Case 5Area of Reinforcing Steel16Concept of Balanced Case 6Modulus of Elasticity (Concrete)17Concept of Proportional Ratio 7Ultimate Strain of Concrete18Equilibrium of Forces 8Compressive Strength of Concrete 19Solution of Linear Equations 9Capacity Reduction Factor20Central of Cross-Section 10Nominal Resisting Strength21Solution of second order equation 11Designed Resisting Strength

8. System Analyses and Design (3/7) 2. Defining Subject Calculations: Item Calculation Name Item Calculation Name 1Balanced Depth of Neutral Axis12Depth of Equivalent Stress Block 2Modulus of Elasticity (Concrete)13Depth of Neutral Axis (Balance) 3Area of Bar14Stress of Reinforcement (Balance) 4Depth of Equivalent Stress Block (Balance) 15Stress of Reinforcement (Under reinforcement) 5Parameter of Equivalent Stress Block 16Nominal Resisting Moment (Tension) 6Under reinforced Case17Nominal Resisting Moment (Compression) 7Area of Bar (Balance)18Total Tensile Force 8Balanced Reinforcement Ratio19Depth of Neutral Axis (Under reinforcement) 9Over reinforced Case20Depth of Neutral Axis (Over reinforcement) 10Balanced Case21Stress of Reinforcement (Over reinforcement) 11Total Compressive Force22Ultimate Resisting Moment

9. System Analyses and Design (4/7) 3. Evaluation between Concepts and Calculations: Relationship between Q i and C j Concept C j C1C1 C2C2 C3C3 …….CnCn Calculation Item Q i Q1Q1 e 11 e 12 e 13 …….e 1n Q2Q2 e 21 e 22 e 23 …….e 2n Q3Q3 e 31 e 32 e 33 …….e 3n …. …….…. QmQm e m1 e m2 e m3 …….e mn e ij = 1 (related ), e ij = 0 (irrelated )

10. System Analyses and Design (5/7) 4. Construct the Logic Structure of Correct Answer: An independent unit The Rule-based Petri-Net The model of the Rule-based Petri-Net

11. System Analyses and Design (6/7) 5. Partial Scoring Function: Score = WC x GCI + WS x CRC Where Score: Final score. GCI: The value of Goldsmith’s Closeness Index (Similarity between correct and examiner's concept map). CRC: The value of Correct Rate of Calculations. WC: Weight of GCI (0~1). WS: Weight of CRC (1-WC). CRC = Σ( P i x D i ) / Σ D i ( i = 1 ~ m ) Where CRC: The value of Correct Rate of Calculations. P i : The value of each calculation (0 or 1). D i : The difficulty value of each calculation (1(easy) ~9(hard)). m: The number of calculations within a test item.

12. System Analyses and Design (7/7) 6. Misconception Evaluation: (1) Complete correctness: (2) No reply, but system infers complete correctness: (3) Correct calculation (wrong answer from mistake inheritance): (4) No reply, but system infers correct calculation: (5) Wrong answer: (6) No reply: Calculation judgment CRV (i) = WC x C (i) + WS x P i ( i = 1 ~ m ) Where CRV (i) : The Complete Rating Value of calculation i. C (i) : The closeness coefficient of calculation i.

13. System Demonstrations (1/8) Entry Screen (Select the Test Subject)

14. System Demonstrations (2/8) Select Concepts and Test Style

15. System Demonstrations (3/8) Problem’s Description

16. System Demonstrations (4/8) Answer Evaluator

17. System Demonstrations (5/8) Concept Mapping Reply System

18. System Demonstrations (6/8) Assessment Results

19. System Demonstrations (7/8) Scoring Instruction

20. System Demonstrations (8/8) Misconception Evaluation

21. Test Demonstration (1/6) Step 1: Select Test Subject. Step 2: Select Test Concepts and Style.

22. Test Demonstration (2/6) Step 3 : Problem description (generated by system)

23. Correct Answer Test Demonstration (3/6) Step 4: User’s Reply Answer 36.43 cm 0.85 30.97 cm 86.99 cm 2 21.85 cm 159.18 T 1(True) 36.42 cm 0.8 29.14 cm 81.87 cm 2 23.21 cm 159.21 T

24. Step 5: Assessment Begin Test Demonstration (4/6)

25. Step 6: Scoring Instruction Test Demonstration (5/6)

26. Step 7: Diagnosis Instruction Test Demonstration (6/6)

27. Conclusions 1.This assessment system adopts the dynamic state to make test items. The values of the test variables are generated randomly. Therefore, the same test problem will be shown within different variable values for different tests, to prevent students from memorizing the answers. 2.We also purpose a Rule-based Petri-Net model for building the logic structure of correct answer. This logic structure can match the dynamic pattern and random parameters completely. 3.This system also adopts concept-mapping environment as examiner’s reply system, can acquire enough information to investigate examinee’s misconceptions and measure final score by using partial scoring strategy.

28. The End Thanks