Mathematical Thinking CATs

WHY USE THE MATH CATs?
The Mathematical Thinking Classroom Assessment Techniques (Math CATs) are designed to promote and assess thinking skills in mathematics. 

WHAT ARE THE MATH CATs?
The Mathematical Thinking Classroom Assessment Techniques (Math CATs) are designed to promote and assess thinking skills in mathematics. Few faculty have difficulty finding or developing tools which assess the specific mathematical techniques which they teach; a challenge which faculty do face is to find ways to promote and assess the development of mathematical thinking - notably to help students know what to do when faced with problems which are not identical to the technical exercises commonly encountered in mathematics classes. 

Here we define mathematical thinking as 
...the development of a mathematical point of view - valuing the process of mathematization and abstraction and having the predilection to apply them; and the development of competence with the tools of the trade, and using those tools in the service of the goal of understanding structure. (Schoenfeld, 1992)  

The Math CATs are designed to address this challenge by offering ways to assess and instill a broad range of the mathematical thinking skills important for students in the science, mathematics, engineering, and technology disciplines. These skills include: 
•	checking results and correcting mistakes (Fault finding and fixing) 
•	making plausible estimates of quantities which are not known (Plausible estimation) 
•	modeling and defining new concepts (Creating measures) 
•	judging statements and creating proofs (Convincing and proving) 
•	organize unsorted data and draw conclusions (Reasoning from evidence) 

WHAT IS INVOLVED?
Instructor Preparation Time:	Minimal if use existing tasks.	
Preparing Your Students:	Students will need some coaching on their first task.	
Class Time: 	Some tasks take 5 minutes; others as much as 45 minutes.	
Disciplines: 	Varies with specific CAT.	
Class Size: 	Any.	
Special Classroom/Technical Requirements: 	None except for Reasoning from Evidence CAT.	
Individual or Group Involvement: 	Either.	
Analyzing Results: 	Varies with specific CAT.	
Other Things to Consider: 	Fairly demanding task for students who are unfamiliar with open-ended problems.	

Description of the 5 Math CATs

1.	Fault finding and fixing
There are a number of well-known misconceptions held by students of mathematics, many of which persist undetected into the college years. These misconceptions need to be identified and remedied to avoid major conceptual problems later. Many of the examples make use of common misconceptions, and so the task can play a diagnostic role. 

The tasks in this package offer students a number of mathematical mistakes which they are asked to diagnose and rectify. These require students to analyze mathematical statements and deduce from the context the part that is most likely to contain the error (there may be more than one possibility), explain the cause of the error and rectify it. Such tasks can be quite demanding. It is often more difficult to explain the cause of another's seductive error than to avoid making it oneself. Contexts include percentages, graphical interpretation, and reasoning from statistical data (download tasks). 

Example: Double Coin Toss
- I'll toss two coins.
- If they both come up heads then Jane wins.
- If they both come up tails then Ben wins.
- If we get one head and one tail then I win.

Explain why this is not a fair game. 
(Answer.) 

2.	Plausible estimation (Fermi problems)
Plausible Estimation consists of a one or two easily-stated questions which at first glance seem impossible to answer without reference material, but which can be reasonably estimated by following a series of simple steps that use only common sense and numbers that are generally known or are amenable to estimation. 

Plausible Estimation tasks involves students in an activity central to modelling in science, other areas of intellectual activity, and in everyday life. The core skill is to create (or check) estimates of quantities that, at first glance, seem unknowable. Students are also required to communicate their assumptions and results and check the plausibility of their answers. In addition, Plausible Estimation requires students to practice arithmetic fluency, ability to handle large numbers, and conversion of units (download tasks). 

Example: How many babies are born in the United States each minute?
(Answer.) 

3.	Creating measures
Creating Measures consists of a series of questions that prompt students to evaluate an existing measure of an intuitive concept , and then create and evaluate their measure of this concept. 

We constantly "mathematize," or construct measures for, physical and social phenomena and use these models to make decisions about our everyday lives. These can vary from measures of simple quantities (such as "speed" or "steepness") to complex and subjective social ones (such as "quality of life" or "best universities"). Since these measures are mathematical models of some phenomenon, they are open to criticism and improvement, especially when considering their usefulness. These tasks provide a fun and interesting way to assess your students' abilities to "mathematize" concepts and show students that there can be many different formal, quantitative measures of such concepts. More importantly, they emphasize that measures differ in their utility; some are more useful than others in representing concepts (download tasks). 

Example: Steepness 

Without measuring anything, put the above staircases in order of steep-ness.

4.	Convincing and proving
This CAT introduces the notions of convincing and proving and illustrates several kinds of proofs commonly encountered in mathematics. These tasks are intended to assess how well students are able to argue logically, use examples and counterexamples to support their reasoning and identify breakdowns in rational argument. In addition, some tasks reveal common student misconceptions students make in their reasoning (download tasks). 

There are two types of tasks: 
1.	Evaluate a set of statements as "always, sometimes or never true".
	Students are expected to offer examples, counterexamples, and reasons for their decisions. 
2.	Evaluate "proofs" and distinguish the correct from the flawed. 

Example: If two rectangles have the same perimeter, they have the same area.
Is this always, sometimes or never true? 
(Answer.) 

5.	Reasoning from evidence
This CAT requires students to analyze unsorted data. The tasks will assess students' abilities to organize information, represent it in a meaningful way, and draw sensible conclusions. It is an important skill especially for students in a SMET discipline, to be able to analyze and interpret data, and argue critically and make informed decisions based on sound reasoning obtained from this data (download tasks). 

Example: In this example, you are a Road Safety Advisor.
Your task is to produce some suggestions about how road safety in Smallville might be improved. 

To help you, below you have a map of Smallville and a database of traffic accidents that took place during the last year. These figures show the time and place of the accident, details of the victim and the type of vehicle that caused the accident. (Times are given as decimals, to make graphing easier). 

Your task is to: 
1.	Find the trouble spots in the town. 
2.	Try to decide why they are trouble spots. 
3.	You have $100,000 to spend on improving road safety. 

Goals

Instruction in mathematics should help students: 
•	become independent learners, interpreters, and users of mathematics; 
•	feel confident in their ability to do mathematics; 
•	develop mathematical thinking, to analyze and understand, and to perceive structure and structural relationships; 
•	develop analytical skills, and the ability to reason in extended chains of
•	argument; 
•	understand important concepts; 
•	to have a broad range of approaches and techniques; 
•	by providing a broad range of problems; 
•	focus on conceptual understanding and technical skills; 
•	apply what they know to new contexts; 
•	present clear, coherent arguments; 
•	develop precision in written and oral presentations; 
•	with a sense of what mathematics is and how it's done. 

If students are to achieve these goals, then an appropriate intellectual environment in which they learn mathematics must be created. The MathCATs provide materials which support the creation of such learning environments. 

Theory & Research

On the nature of mathematics
Mathematics...today is a diverse discipline that deals with data, measurements, and observations from science; with inference, deduction, and proof; and with mathematical models of natural phenomena, of human behavior, and of social systems...

In addition to theorems and theories, mathematics offers distinctive modes of thought which are both versatile and powerful, including modeling, abstraction, optimization, logical analysis, inference from data, and use of symbols. Experience with mathematical modes of thought builds mathematical power -- a capacity of mind of increasing value in this technological age that enables one to read critically, to identify fallacies, to detect bias, to assess risk, and to suggest alternatives. Mathematics empowers us to understand better the information-laden world in which we live.  (National Research Council, 1989, pp. 31-32).

This quotation describes a number of distinct features of mathematics; a body of knowledge; a set of tools to be used in everyday life and by the scientific community; and a set of powerful thinking tools.  Mathematics is seen as discipline which empowers its students.  It follows that education in mathematics should set out to address each of these features.

Challenges facing teaching
Teachers of undergraduate mathematics and other disciplines that rely on mathematics face a number of challenges. Serious conceptual problems have been documented in students who seem appropriately qualified.  Armstrong and Croft (1999) set diagnostic tests of mathematics on entry to engineering programs, and showed several areas of weakness. For example, around 20% of students had problems dealing with significant figures, and around 15% had problems with decimals. Student ratings of their confidence that they understood and could apply mathematics ranged from about 60% for the graph of a linear function, to about 10% for polar co-ordinates.  Faculty are often dissatisfied with student knowledge on entry to freshman mathematics classes (the Royal Society, 1998).  Particular problems are associated with: student fluency and accuracy; failure to understand the connectedness of mathematics; a lack of understanding of proof and the need for rigorous argument; and an inability to solve non-standard problems (e.g. Tall, 1992).

A number of commentators have identified problems with conventional approaches to the teaching of mathematics in high schools and colleges.  Weaknesses are related to pedagogy and assessments which emphasize the mastery of mathematical techniques, but emphasize neither the conceptual side of mathematics nor the development of the habits of mind that characterize mathematical thinking.  Traditional testing methods in mathematics have often provided limited measures of student learning, and equally importantly, have proved to be of limited value for guiding student learning. The methods are often inconsistent with the increasing emphasis being placed on the ability of students to think analytically, to understand and communicate, or to connect different aspects of knowledge in mathematics (e.g. Ridgway, 1988; Brown, Bull and Pendlebury, 1997). 

One consequence of this type of curriculum and assessment system is that students learn in school that problems mostly have neat, unique solutions, and that methods to solve problems will be provided to them.  For example, in the 1983 National Assessment of Educational Progress, nine students out of ten agreed with the statement "There is always a rule to follow in solving mathematics problems" (NAEP, 1983, pp. 27-28).  Over time students come to adopt a passive role, and think of mathematics as a dead body of knowledge which they have to memorize, rather than as a set of higher-order thinking tools which will increase their abilities to deal with a complex world. (e.g. Carpenter, Lindquist, Matthews, & Silver, 1983; Schoenfeld 1992). 

Developing Mathematical Thinking
...the reconceptualization of thinking and learning that is emerging from the body of recent work on the nature of cognition suggests that becoming a good mathematical problem solver - becoming a good thinker in any domain - may be as much a matter of acquiring the habits and dispositions of interpretation and sense-making as of acquiring any particular set of skills, strategies, or knowledge. (Resnick, 1989, p. 58).

Thinking mathematically depends on a number of different components (Schoenfeld, 1992), notably core knowledge, problem solving strategies, effective use of one's resources, having a mathematical perspective, and active engagement in the practice of mathematical thinking.  Mathematics instruction must present experiences which develop student knowledge in each of these areas.

Mathematics in the classroom should model these elements if students are to come to understand and use mathematics and to learn to think mathematically.  Learning mathematics is about learning to work in the ways that mathematicians work, and is about acquiring the thinking skills that mathematicians use.  These skills are important for scientists as well as mathematicians.  Pólya (e.g. 1954, 1957) argued that mathematics resembles the physical sciences in its dependence on conjecture, insight, and discovery.  He argued that for students to understand mathematics, their experience with mathematics must be consistent with the way mathematics is done by mathematicians.

There is an extensive body of knowledge comparing the knowledge of experts and novices (e.g. Ericsson and Charness, 1994 for a review across disciplines; Schoenfeld, 1985 for studies in undergraduate mathematics) which can be mined for ideas on appropriate teaching strategies; and a great many studies which show the effectiveness of particular teaching methods (e.g. Palinscar and Brown, 1984).  Accessible accounts of the literature are provided by Schoenfeld (1983, 1985) and Bransford, Brown, and Cocking (eds.) (1999).

Developing Assessment
Improved assessment systems may help with these problems. There is evidence that educational attainment can be raised by better assessment systems (Black and William, 1998; Dassa, Vazquez-Abad, and Ajar; 1993).  Such assessment systems are characterized by: a shared understanding of assessment criteria; high expectations of performance; rich feedback; and effective use of self-assessment peer assessment, and classroom questioning.

The intention of the mathCATs is to improve the quality of both formative and summative assessment systems, and thereby to improve the quality of undergraduate mathematics teaching and learning.

Links
MARS web site (http://www.nottingham.ac.uk/education/MARS/)

Mathematics Association of America (http://www.maa.org/Welcome.html)

Cooperative Learning in Undergraduate Mathematics Education (Project
CLUME) (http://www.uwplatt.edu/~clume/)

Association for Research in Undergraduate Mathematics Education (http://www.maa.org/data/t%5Fand%5Fl/Arume1.html)

The Math Forum (http://forum.swarthmore.edu/)

Sources
Balanced Assessment Group (1998, 1999). Balanced Assessment for the
Mathematics Curriculum: High School Assessment (2 volumes). White
Plains, NY: Dale Seymore. 

Balanced Assessment Group (1998, 1999). Balanced Assessment for the
Mathematics Curriculum: Advanced High School Assessment (2 volumes).
White Plains, NY: Dale Seymore. 

Gold, B., Keith, S., and Marion, W. (Eds.). (1999). Assessment Practices in
Undergraduate Mathematics. Washington, DC: MAA. 

References
Armstrong, P., and Croft, A. (1999). Identifying the learning needs in mathematics of entrants to undergraduate engineering programmes. European Journal of Engineering Education, 14(1).

Black, P., and William, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-73.

Bransford, J., Brown, A., and Cocking, R. (eds.) (1999). How People Learn. Washington, DC: National Academy Press.

Brown ,G., Bull, J., and Pendlebury, M (1997). Assessing Student Learning in Higher Education. London: Routledge.

Carpenter, T. P., Lindquist, M. M., Matthews, W., & Silver, E. A. (1983). Results of the third NAEP mathematics assessment: Secondary School. Mathematics Teacher, 76 (9), 652-659.

Dassa, Vazquez-Abad, and Ajar; 1993

Ericsson, K., and Charness, N. (1994). Expert performance: its structure and acquisition. American Psychologist, 49, 725-745.

The London Mathematical Society (1995). Tackling the Mathematics Problem. London: London Mathematical Society.

Marton, F, and Saljo, R. (1976). On qualitative differences in learning: I – outcome and process. British Journal of Educational Psychology, 46, 4-11.

National Assessment of Educational Progress. (1983). The Third National Mathematics Assessment: Results, trends, and issues (Report No. 13-MA-01).  Denver, CO: Educational Commission of the States.

National Research Council. (1989). Everybody Counts: A report to the nation on the future of mathematics education. Washington, DC: National Academy Press. 

Palincsar, A., & Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1(2), 117-175.

Pólya, G. (1954). Mathematics and Plausible Reasoning (Volume 1, Induction and analogy in mathematics; Volume 2, Patterns of plausible inference). Princeton: Princeton University Press.

Polya, G. (1957). How to Solve It: a new aspect of mathematical method. Princeton, NJ:Princeton University Press.

Resnick, L. (1989). Treating mathematics as an ill-structured discipline. In R. Charles & E. Silver (Eds.), The Teaching and Assessing of Mathematical Problem Solving. Reston, VA: National Council of teachers of Mathematics.

Ridgway, J. (1988).  Assessing Mathematical Attainment. Slough: NFER-Nelson.

Ridgway, J., Swan, M., and Burkhardt, H. (2001, in press). Assessing Mathematical Thinking via FLAG. In D. Holton and M.Niss (eds.): Teaching and Learning  Mathematics at University Level - An ICMI Study. Dordrecht: Kluwer Academic Publishers.

The Royal Society (1998). Mathematics Education pre-19. London: The Royal Society.

Schoenfeld, A. (1983). Problem solving in the mathematics curriculum: A report, recommendations, and an annotated bibliography.  Washington, DC:Mathematical Association of America.

Schoenfeld, A. (1985). Mathematical Problem Solving.  New York: Academic Press.

Schoenfeld, A. H. (1992). Learning to think mathematically: problem solving, metacognition, and sense-making in mathematics. In D. Grouws, (ed.). Handbook for Research on Mathematics Teaching and Learning. New York: MacMillan. pp. 334-370.

Tall, D. (1992). The transition to advanced mathematical thinking: functions, limits, infinity and proof.  In D.A.Grouws (ed.) Handbook of Research on Mathematics Teaching and Learning. New York: Macmillan.

Malcolm Swan
Mathematics Education
University of Nottingham
Malcolm.Swan@nottingham.ac.uk

Most assessment practices seem to emphasise the reproduction of imitative, standardised techniques. I want something different for my students. I want them to become mathematicians - not rehearse and reproduce bits of mathematics.

I use the five 'mathematical thinking' tasks to stimulate discussion between students. They share solutions, argue in more logical, reasoned ways and begin to see mathematics as a powerful, creative subject to which they can contribute. Its much more fun to try to think and reach solutions collaboratively. Assessment doesn't have to be an isolated, threatening business. 

Not just answers, but approaches.

Malcolm Swan is a lecturer in Mathematics Education at University of Nottingham and is a leading designer on the MARS team. His research interests lie in the design of teaching and assessment. He has worked for many years on research and development projects concerning diagnostic teaching (including ways of using misconceptions to promote long term learning), reflection and metacognition and the assessment of problem solving. For five years he was Chief Examiner for one of the largest examination boards in England. He is also interested in teacher development and has produced many courses and resources for the inservice training of teachers. 

Jim Ridgway
School of Education
University of Durham
Jim.Ridgway@durham.ac.uk

Thinking mathematically is about developing habits of mind that are always there when you need them - not in a book you can look up later.

For me, a big part of education is about helping students develop uncommon common sense. I want students to develop ways of thinking that cross boundaries - between courses, and between mathematics and daily life.

People should be able to tackle new problems with some confidence - not with a sinking feeling 'we didn't do that yet'. I wanted to share a range of big ideas concerned with understanding complex situations, reasoning from evidence, and judging the likely success of possible solutions before they were tried out. One problem I had is that my students seemed to learn things in 'boxes' that were only opened at exam time. Thinking mathematically is about developing habits of mind that are always there when you need them - not in a book you can look up later.

You can tell the teaching is working when mathematical thinking becomes part of everyday thinking. Sometimes it is evidence that the ideas have become part of the mental toolkit used in class - 'lets do a Fermi [make a plausible estimate] on it'. Sometimes it comes out as an anecdote. On graduate told me a story of how my course got him into trouble. He was talking with a senior clinician about the incidence of a problem in child development, and the need to employ more psychologists to address it. He 'did a Fermi' on the number of cases (wildly overestimated) and the resource implications (impossible in the circumstances). He said there was a silence in the group...you just don't teach the boss how to suck eggs, even when he isn't very good at it. He laughed.

Jim Ridgway is Professor of Education at the University of Durham, and leads the MARS team there. Jim's background is in applied cognitive psychology. As well as kindergarten to college level one assessment, his interests include the uses of computers in schools, fostering and testing higher order skills, and the study of change. His work on assessment is diverse, and includes, the selection of fast jet pilots, and cognitive analyses of the processes of task design. In MARS hhe has special responsibility for data analysis and psychometric issues, and for the CL-1 work. 

About MARS
The Mathematics Assessment Resource Service, MARS, offers a range of services and materials in support of the implementation of balanced performance assessment in mathematics across the age range K to CL-1. MARS is funded by the US National Science Foundation, and builds on earlier funding which began in 1992 for the Balanced Assessment Project (BA) from which MARS grew. 

MARS offers effective support in: 
The Design of Assessment Systems: assessment systems are tailored to the needs of specific clients. Design ranges from the contribution of individual tasks, through to full scale collaborative work on test development, scoring and reporting. Clients include Cities, States, and groups concerned with educational effectiveness, such as curriculum projects and professional development initiatives. 

Professional Development for Teachers: most teachers need help in preparing their students for the much wider range of task types that balanced performance assessment involves. MARS offers professional development workshops for district leadership and 'mentor teachers', built on materials that are effective when used later by such leaders with their colleagues in school. 

Developing Design Skills: many clients have good reasons to develop their own assessment, either for individual student assessment or for system monitoring. Doing this well is a challenge. MARS works with design teams in both design consultancy and the further development of the team's own design skills. 

To support its design team, MARS has developed a database, now with around 1000 interesting tasks across the age range, on which designers can draw, modify or build, to fit any particular design challenge.