1. Alternative Assessment with Electronic Concept Mapping Software Dianne Raubenheimer and Kevin Oliver, NCSU

2. 1. What is Concept Mapping

3. What is concept mapping? concept mapping is a technique for organizing and representing information a true concept map must include: –core concepts - usually enclosed in circles or boxes –relationships illustrated by lines and arrows connecting concepts AND by propositions or statements on those lines that explain the nature of the relationship Books Pages Binding are made of

4. From: http://cmap.ihmc.us/http://cmap.ihmc.us/

5. How are maps used? for assessment to gain insight into student knowledge structures--valid understandings and misconceptions for group thinking to collectively brainstorm and structure issues relevant to a topic for planning to organize information for a report, presentation, debate, or discussion for research to analyze how groups respond to or perceive something

6. Concept Map Forms a concept map can take many forms, provided it includes concepts and illustrates relationships through BOTH lines AND proposition statements: spider, with information organized outward around a central topic

7. Concept Map Forms hierarchy, with information structured from most to least important; or from most inclusive and general to less inclusive and specific

8. Concept Map Forms flowchart, with information ordered in a linear format

9. Concept Map Forms systems, with information ordered like a flowchart with inputs and outputs

10. Non-Examples A concept map is a very specific thing; it is NOT any of these things: A flow chart of a process that has no proposition statements explaining the relationship between stages

11. Non-Examples an organizational chart that has no propositions or statements that explain how different people support one other

12. Non-Examples Story Webs: analyze various literary elements (plot, characterization, theme), dissect characters, plots and sequence of events, to learn about composition

13. Non-Examples a mind map is similar to a concept map, with a central topic main themes are noted on "branches" main themes are expanded into details propositions explaining relationships between concepts are not included

14. 2. What is the Process for Developing a Map?

15. Select Domain select a domain of knowledge somewhat familiar to the person constructing the map (e.g., air pollution) the domain should be limited to a manageable size: –a segment of text –a lab activity –part of a course

16. Identify Key Concepts in Domain list key concepts in domain (e.g. flowering plants, dicotyledon, monocotyledon, etc…) order concepts from the most inclusive or general to the less inclusive and specific –Types of - dicotyledon, monocotyledon –Roots – fibrous, tap, tuberous, aerial, etc. –Leaves – simple, compound, pinnate, etc. –Stems – herbaceous, woody –Flowers – actinomorphic, zygomorphic

17. Construct Preliminary Map use software or post-it notes to arrange concepts visually attempt different structures: hierarchical, spider, flowchart, systems

18. Examine Relationships insert lines/arrows and write preliminary proposition statements that explain relationships between concepts look for cross-links between different domains of knowledge identified (if topic includes multiple domains) help students understand every concept "could be" related to every other concept, but they must evaluate and choose the most important concept relationships

20. string maps illustrate poor understanding of a knowledge structure

21. 3. What are Some Strategies for Employing Concept Maps in the Classroom?

22. Strategies: Complete Fill-in fill-in blanks in pre-structured map from a list of concepts helps students to consolidate understanding of concepts and relations among them

23. Strategies: Partial Fill-in

24. Strategies: Fill-in-the-Propositions

26. Strategies: Pre-Selected Term Mapping fossil fuels deforestation clouds atmospheric water vapor ocean CO2 Industrial Revolution longwave solar radiation global warming human activity evaporation transpiration greenhouse gas concentrations ice albedo photosynthesis feedback loops shortwave solar infrared radiation Design a concept map to show your understanding of CO2, a greenhouse gas, and its past and future effect on climate. Use the following concepts:

27. Strategies: Seeded Term Mapping also called "micro-mapping" provide students with a list of "starter" or "seed" terms (5-6) students must use these terms in their map AND use another 5-6 terms from their own knowledge of the topic

28. E.g. via Powerpoint and E-mail assign articles to read on a selected topic group students and provide each group with a Powerpoint file to pass around via e-mail slide 1 contains the concept map; each student adds 3-4 items per round (e.g., concepts, links, propositions) slide 2 contains names of group members slides 3-4 contain comments from students describing changes they made to the group file, or relevant suggestions Strategies: Collaborative Mapping

29. Strategies: Advanced Organizer develop an expert map of a course or a course topic provide to students as an advance organizer prior to teaching and for review purposes

30. Unguided Maps Once students are familiar with concept mapping processes, they can also be asked to do unguided maps, e.g. –After reading a section of text, –At the end of a section of course work. –At the end of the course. This helps with identifying and tying together core concepts, and also acts as a learning tool for test revision.

31. 4. How are Concept Maps Used for Assessment?

32. ‘Live’ links available at: http://www4.ncsu.edu/~cdrauben/concept%20maps/Assessing%20Concept%20Maps.htmlhttp://www4.ncsu.edu/~cdrauben/concept%20maps/Assessing%20Concept%20Maps.html

33. Items Commonly Assessed were the correct concepts or nodes identified? do the lines and arrows correctly link concepts together into clusters or hierarchies? do the propositions or statements on the lines between concepts accurately describe how two concepts are related? how detailed are the resources or elaborative examples attached to the concepts (e.g., Web links, articles, annotations, notes, excerpts cut and pasted from a class discussion board, etc.)

34. 4 Scoring Methods (McClure, Sonak & Suen, 1999) were the correct concepts or nodes identified? do the lines and arrows correctly link concepts together into clusters or hierarchies? do the propositions or statements on the lines between concepts accurately describe how two concepts are related? how detailed are the resources or elaborative examples attached to the concepts (e.g., Web links, articles, annotations, notes, excerpts cut and pasted from a class discussion board, etc.)

35. Structural Scoring Novak Scoring Protocol hierarchical structure, concept-links, cross-links, and examples

36. Sample Instruction Sheet (Dempsey et al, 2002) 1.Think about the following questions: What is climate? How is climate studied? What factors determine climate? How has climate differed in the past? What can cause climate to change? 2.Recall the instructions for how to draw a hierarchical concept map. 3.Construct a concept map of your ideas about climate, starting here: Climate

37. Sample Scoring Protocol a sample scoring protocol for this "climate" map assessment: –half a point for each of 12 specific concepts –one point for each of up to four coherent, labeled, hierarchical lines of connections related to prompting questions –extra points for good cross connections between hierarchies –note: large differences had to be reconciled between two scorers; the two scores averaged

38. Relational Scoring Quality or importance of each individual concept-link, without regard to the overall structure of the map, e.g. one point for each correct link and proposition. Modified relational scoring where, concept-links, cross-links, and examples were assessed using the following scale (West, et al) : –invalid relationship between concepts (0 points); –valid relationship between concepts but propositional label is incorrect (1 point); –valid relationship and propositional label correct but lacks foundational or core relationship to subject matter (2 points); and –valid relationship and propositional label and foundational or core relationship apparent (3 points).

39. Holistic Scoring Using a rubricrubric Example http://www.markville.ss.yrdsb.edu.on.ca/ history/west/west_conceptmap.pdf Combining structural and relational scoring

40. Other Scoring ideas Concept List: Creating Instability Cut these out to make it easier to move them around on the incomplete concept map (Dempsey et al, 2002) bottom of layer warms some parcels become warmer than neighboring parcels lapse rate of some parcels decreases warm air advects into bottom of layer cold air advects into top of layer environmental lapse rate decreases top of layer cools lifted layer expands (deepens), so top rises farther and hence cools more than bottom cold air advects into top of layer top of layer cools more than bottom layer undergoes (large-scale) forced lifting water evaporates into parcels

41. Other Scoring ideas Points for each correct concept placed in the partially completed map

42. Some issues structural scores increase significantly after instruction (West, et al, 2002) relational method is the most reliable across raters (Rye & Rubba, 2002, West et al, 2002) scores obtained from the correctness of propositions (relational method) were most likely to correlate with state and national standardized tests (Rye & Rubba, 2002) but, relational method is less sensitive in measuring changes and differences over time (West, et a.)

43. 5. What Electronic Tools are Available to Support Concept Mapping?

44. Mapping Tools There are several commercial mapping tools: Some Examples: Microsoft Visio--business & technical diagrams SmartDraw--mind map templates, timeline templates, flowcharts, org charts, network diagrams, floor plans

45. Kidspiration and Inspiration--used heavily in K-12 settings, education templates and clip art, move between map and outline modes, attach notes, audio, hyperlinks, and other external resources, export function MatchWare OpenMind--create mind maps with templates, export function Mapping Tools

46. cMap– Free online tool built for collaborative concept mapping, can attach video, images, files, links, and other external resources, export function Mapping Tools

47. From Concept to Content Maps increasingly, mapping programs allow people to attach resources to concepts (e.g., notes, links, images, files), reminiscent of e-portfolio systems selecting and adding content resources to concepts provides another layer of detail and helps to show the types of information a student associates with a concept (e.g., a specific research article) encourage students to annotate or add notes to explain the relationship of the attached resources to the concept; otherwise you may be forced to "guess" why they chose to include a resource

48. References Dempsey, D., O'Sullivan, K., & White, L. (2002). Action research for SFSU's NASA-NOVA course. Retrieved September 27, 2005, from the World Wide Web: http://funnel.sfsu.edu/courses/gm310/assessment/ McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map assessment of classroom learning: Reliability, validity, and logistical practicality. Journal of Research in Science Teaching, 36(4), 475- 492. Rye, J. A., & Rubba, P. A. (2002). Scoring concept maps: An expert- based scheme weighted for relationship. School Science and Mathematics, 102(1), 33-44. Trochim, W.M.K. (2004). An introduction to concept mapping for planning and evaluation. Retrieved September 27, 2005, from the World Wide Web: http://www.conceptsystems.com/papers/intro_article.cfm http://www.conceptsystems.com/papers/intro_article.cfm West, D.C., Park, J.K., Pomeroy, J.R., & Sandoval, J. (2002). Concept mapping assessment in medical education: A comparison of two scoring systems. Medical Education, 36, 820-826.