PISA Mathematics Framework and Curriculum Review Sean Close St Patrick’s College, Dublin 9 Second PISA National Symposium Regency Hotel, Dublin 9 Wednesday, April 6 th 2005

Outline  Why PISA in Mathematics?  Theoretical Basis of PISA mathematics Framework  Mathematisation  Dimensions of PISA mathematics Framework  PISA Mathematics Framework and National Curricula  PISA Mathematics framework and Irish Junior Cycle Mathematics Curriculum

Why PISA in Mathematics? Focuses on assessing how well students can use the mathematics they have learned in realistic situations (mathematical literacy) rather than on assessing common curricular elements across countries Move towards mathematisation and problem-based teaching and learning (situated learning) and away from mathematics as isolated sets of concepts, principles and procedures taught by exposition and practice Move towards more broad based and valid assessment (Authentic Assessment)

Definition of mathematical Literacy Mathematical Literacy is an individual's capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to engage in mathematics, in ways that meet the needs of that individual's life as a constructive, concerned, and reflective citizen.

Theoretical Basis of PISA Sociocultural Literacy Situated Learning Authenthic Assessment

Sociocultural Literacy Mathematics is seen as a language which has design features (e.g. concepts, procedures), and to be mathematically literate means being able to use some of these features in different social functions e.g. knowing how to divide two numbers means knowing a design procedure but does not mean that one understands its structure or can use it to find, for example, the number of coaches needed for a school tour

Situated Cognitive Learning Concerned with how one becomes mathematically literate and therefore how to assess levels of mathematical literacy. Learning in any conceptual domain is a result of experiences – activities designed to facilitate movement from informal ideas to more formal and abstract ways of representing and reasoning in the domain. Assumes that learning in a domain should begin with a situated problem that makes sense to the learners

Authentic Assessment Move away from making measurable mathematics more important and towards making important mathematics more measurable. Expand assessment tasks to include more complex and situated problems. Not enough to assess basic concepts and procedures, also need to assess how well students can use such design features of mathematics to solve increasingly complex problems. Process should be assessed as well as content – PISA does this using three Competency Classes (Reproduction, Connections, Reflection).

This is an ‘authentic’ performance based test as opposed to the more ‘traditional’ test where students would simply be asked to write a description of how to climb a tree

Mathematisation (1) Starting with a problem situated in reality; (2) Organising it according to mathematical concepts; (3) Gradually trimming away the reality through processes which promote the mathematical features of the situation and transform the real problem into a mathematical problem that faithfully represents the situation; (4) Solving the mathematical problem; and (5) Making sense of the mathematical solution in terms of the real situation.

Horizontal Mathematisation Main focus of PISA

Vertical Mathematisation Real-world problem Problem in mathematics world Mathematics world Real world Horizontal Mathematisation Problem structured and formalised Vertical Mathematisation Main focus of school mathematics

Dimensions of PISA Mathematics Framework Mathematical Situations and Contexts: Personal; social/occupational; public; scientific Mathematical Content: The mathematical content is described as four problem areas that encompass the kinds of problems that arise through interaction with day to day phenomena. For PISA purposes, these are called the four ‘Overarching Ideas’:- Quantity; Space and Shape; Change and Relationships, Uncertainty.

Processes/Competencies: Basic processes: Reasoning; Argumentation; Communication; Modelling; Problem posing and solving; Representation; using symbolic, formal and technical language and operations; Use of aids and tools Competency Classes Competency Class 1 (Reproduction); Performing specific calculations, solving equations, reproducing memorised facts or ‘solving’ well-known routine problems. Competency Class 2 (Connections); Integrating information, making connections within and across mathematical domains, or solving problems using familiar procedures in contexts. Competency Class 3(Reflection); Recognising and extracting the mathematics in a situation, using that mathematics to solve problems, analysing and developing models and strategies, or make mathematical arguments and generalisations.

Note: Assessing mathematical process is not easy since exemplary problems that have the potential to promote the development of mathematical processes in students can be reduced to a set of routinized skills. For example, in England, where the assessment of mathematical investigations was introduced into the GCSE examination in the 1980s, in many classrooms this was taught as an additional piece of content, with students given a set of procedures to follow. One of the challenges facing teachers in implementing new curricula is how to integrate mathematical process skills with content topics/objectives.

Question 4: CONTINENT AREA (M148Q02) Process: Competency class 2 Content: Space and shape Situation: Personal Estimate the area of Antarctica using the map scale. Show your working out and explain how you made your estimate.

A school class wants to rent a coach with a driver for an excursion and contacts three companies. Company A charges an initial rate of 375 ZED plus 0.5 ZED per km driven. Company B charges an initial rate of 250 ZED plus 0.75 ZED per km driven, whereas Company C charges a flat rate of 350 ZED up to 200 km plus 1,0 ZED per km beyond 200 km. Which company should the class choose, if the excursion involves a total travel distance of somewhere between 400 and 600 km?

1000 Zed is put into a savings account at a Bank. There are two choices: one can get a rate of 4% OR one can get a10 Zed bonus from the bank and a 3% rate. Which option is better after one year? After two years?

PISA Mathematics Framework and National Curricula  Substantial variation in content, methodology and assessment of mathematics curricula across countries.  Traditional assessment frameworks use a two-dimensional Content/Cognitive Behaviour matrix to relate assessment objectives/tasks to curriculum areas – leads to atomisation and fragmentation of curriculum and constrains assessment  PISA framework has three dimensions – Situations, Content (overarching ideas), and Competencies classes (clusters of process skills) - difficult to relate PISA framework to curriculum framework of many countries – but should be done!

Approaches to Relating PISA framework and National Curricula 1.Analyse each PISA mathematics item independently and list the specific content and cognitive process/behaviour involved and then compare these listings with those of a country’s curriculum. Problems inherent in this approach are decomposition and decontextualisation of the mathematics into a hierarchy of isolated skills and concepts and lack of consideration of the relationships among them. (Romberg and Zarinnia, 1987) 2.Analyse each item or response code and list the mathematical competency cluster which is needed to produce the response and then compare the resulting matrix with the country’s curriculum framework. Such an approach would enable a review team to design a curriculum which addresses mathematical literacy with appropriate emphasis on tasks and investigations involving reproduction, connections, and reflection competency clusters.

PISA Mathematics framework and Irish Junior Cycle Mathematics Curriculum How do the goals of the JC mathematics curriculum compare with the PISA goal of mathematical literacy? How familiar are the PISA items to Irish JC students in terms of content, context, process, and format? How well do the PISA framework and items map onto the Irish JC mathematics curriculum and examination?

Curriculum AspectSource of Information Goals and principlesCurriculum documents SyllabusCurriculum Documents Taught curriculumTextbooks, surveys, teachers test papers Assessed CurriculumExamination papers Achieved CurriculumExamination results Alignment of Curriculum and Assessment

PISA goals and Goals of Junior Cycle Mathematics Curriculum The main goals of the JC mathematics curriculum focus more on mathematics needed for continuing education and less on the mathematics needed for life and work. The 10 sub-goals (objectives) listed have some similarities with PISA framework but 6 of these sub-goals are not currently assessed in any formal way. (Objectives relating to mathematics in unfamiliar contexts, creativity in mathematics, motor skills, communicating, appreciation, and history of mathematics.) Raises question as to whether or not goals which are not going to be taught or assessed should be included in official curriculum documents

PISA Item Familiarity Ratings About 1/3 the concepts in PISA2003 items were unfamiliar to Higher and Ordinary Level students, and about ½ were unfamiliar to Foundation Level students About 2/3 of the contexts in PISA2003 items were unfamiliar to Higher and Ordinary Level students, and about 4/5 were unfamiliar to Foundation Level students About 2/3 of the formats in PISA2003 items were unfamiliar to Higher and Ordinary Level students, and about 4/5 were unfamiliar to Foundation Level students Items in Uncertainty category were least familiar but students performed well on them! Items in Reflections cluster of competencies were least familiar

PISA Test /Junior Cert Curriculum Mappings Junior Cert Strands (Higher and Ordinary) % of PISA items Number Systemsc. 1/10 Applied Arithmetic and Measurec. 1/3 Algebrac. 1/20 Statisticsc. 1/5 Functions and graphsc. 1/25 Sets0 Geometry0 Trigonometry0 Not in Junior Certc. 3/10

PISA’s 4 subscale categories (Quantity, Space and Shape, Change and Relationships, Uncertainty) did not map well on to the 8 Junior Cert Syllabus content areas. None of the PISA Shape and Space items mapped onto the Geometry strand of JC Syllabus but mapped, instead, onto the Applied Arithmetic and Measure strand of the JC Syllabus. The PISA Change and Relationships items map onto 5 content areas of the JC Syllabus About 1/3 of the PISA items did not map onto any JC Syllabus content area. Note: PISA’s content categories were designed to describe content in terms of classes of problems occurring in the real world and may cut across several content areas

Junior Certificate Mathematics Examination and PISA Framework Junior Certificate Examination questions involve, in the main, the Reproduction cluster of competencies with a some on part (c)s involving the Connections cluster of competencies. The JCE questions are also mainly intramathematical (vertical mathematising), with some context based questions – though often rather contrived and predictable. The partial credit system in use by examiners may over-rate performance by giving marks for somewhat trivial workings.

Limitations of the Framework from the perspective of Irish JC Curriculum  Does not have situations involving the JC curricular content areas of Trigonometry, Coordinate Geometry, Proof, aspects of Algebra, and Set Theory  Much greater emphasis on Statistics, Applied Arithmetic and Measure content areas and on informal geometry than in JC curriculum  Very few intra-mathematical tasks involving vertical mathematisation

Limitations of the PISA assessment  Time constraints  Lack of calculator appropriate tasks  No team based problem-solving  No assessment of computer based work and problem- solving  Low stakes assessment

List of basic mathematical topics in PISA 2003 Pilot Study Quantity : Perimeter and area of 2D shapes. Volume and surface area of cube, cuboids, cylinders, cones, pyramids, spheres. Volume/area relationships. Area/diameter relationship in circle. Scale in maps & plans. International Time zones. Buying/Selling/Costs/Interest/discount/taxes. Ratio, proportion & percent. Convert currencies. Units of measure. Compound growth. Exponential & scientific notation. Indices. Timetables. Shape & Space: Elementary triangle theorems/relationships. properties of 2D shapes. Geometric number patterns, Cartesian coordinates. Types of triangles. Tiling & symmetry. 2D & 3D transformations, Similar triangles. Projective geometry, Pythagoras’ Theorem. Change & Relationship: Substitution & transformation of formulae. Solution of linear equations. Connecting physical processes with graphs. Speed / time relationship. Additive & multiplicative patterns. Area / volume relationships. Sets & Venn diagrams. Number theory - divisibility. Prime numbers. Permutations & combinations. Graphs of functions. Integers. Uncertainty: Graphs of relationships between variables, Complex graphical data. Tables of data on two or three variables. Selecting suitable graphs for data sets. Listing outcomes of random events. Scatter plots. Frequency distributions. Mean. mode, median, quartiles.