Since announcing changes to the Math and Science diploma exams (and in fact even before), there has been a wide damnation and derision from the public media and significant concern expressed by the ATA, some teachers and students. Interestingly, the e-mails I get from students tend to be in class or school groups. There is also a significant size Facebook group which I will visit regularly. As well there is some discussion on our Speak Out website. If nothing else, once again people are talking about education and at least in a minimal way discussing what makes our system effective. This is a good thing.

In order to have good public discussion, there needs to be good information. It is not necessary that we agree, only that all options and facts are available to inform discussion. For this reason it would be useful if media actually looked before they jumped. Usually I find Paula Simons witty and informed even when we disagree, but when she spoke of "dumbing down" exams it was a cruel blow, neither witty nor informed.

So why did we make the change? In early September, we communicated to School Boards that effective immediately the Math and Science diploma exams would no longer have the written part A. The part B would be adapted to contain additional multiple choice and numeric questions. Decisions like this are not just dropped out of the sky. There is rationale.

This fall, I am talking with school boards, the ATA, the ASBA, ASCA and other education stakeholders about the need to use the economic and fiscal situation as a platform for transformational change. We have spent most of the past year talking about the future of education through the Inspiring Education dialogue which peaks at the fall conference Oct 18, 19, 20. Historical methodology would suggest that process be sequential, ie. the full Inspiring Education dialogue be complete before we start any change process.

Of course that is not a reality in the world of today. If we know anything about what needs to change it is our processes themselves. It is no longer possible to study and reflect over several years before moving - the result if you do is that you implement a solution for five years ago rather than for tomorrow. Government needs more nimble process — but that is a topic for another day — I digress.

In speaking with boards about transformational change, I proposed some straightforward lenses through which to look at our practice. I call it a value review. Look at every thing we do in the system, whether at the board level, the regional level or system wide. Ask the question — does it add value? Does it help achieve our desired outcomes? Is it focused on student success? If not then we really need to consider whether we should still do it. Ask the question, even if it does add value, is it too resource intensive for the result? What is the value equation? Is there a better way to accomplish the result with fewer resources? And again — what is the benefit to students? These are not just questions we need to ask in times of fiscal stress, but it seems very difficult to get focus at other times. I indicated that everything is on the table. If the department is requiring something which does not have value, then identify it, we can discuss it, and if we cannot demonstrate value we should stop asking for it.

As part of that process, I asked our department to do its own value review. To look at everything we do with those lenses. Everyone in the system knows the emphasis put on assessment and accountability by the Department of Education. In order to demonstrate we mean what we say I asked Learner Assessment Branch to review our exams and assessment processes and identify areas which could be changed or discontinued while still achieving desired outcomes. Removing the written portion of the Math and Science exams were identified as one of the options to look at. While other options required more work, I chose to move on this option immediately for a number of, I believe, very valid reasons.

Almost since the beginning of the new exam structure involving Written (Part A) and Multiple Choice (Part B) exams there have been complaints from teachers, school boards and students about the number of exams, the stress on students studying for and writing two exams per subject, etc. The written portions were, of course, separated and administered earlier to allow for marking time. The exam period was thus stretched out. The time taken from instruction was a concern. etc.

When the departmental exam data was examined, it was clear that there is a very high correlation between the Part A and Part B results on Math and Science Exams. In other words students writing a written exam were getting the same results as they got on the Part B multiple choice and numeric response exam. As this is an exit exam — the results are not used to assist in student instruction for the student writing it — it is in fact the result for the student that is important. If both exams get the same result it is not necessary to do both.

This partially also responds to the rather shallow comment about "dumbing down" the exams. Exam creation is not a simple exercise. A significant amount of expertise and work goes into the creation of exams which are valid and reliable. A properly crafted test will appropriately determine whether a student has grasped the concepts being tested. Multiple choice and numeric questions and exams have been thoroughly authenticated in this regard. The suggestion that a "multiple guess" exam does not properly test is not accurate. (The entrance exam to become registered in the engineering profession is valid, reliable and rigorous – and is all multiple choice). Mathematical calculation indicates that the odds are 1:1billion against getting a passing result by guessing. Many responders have expressed concern about students being able to "show their work" and "get part marks" for written responses. The data shows that this is more perception than reality.

Some, including the ATA, are concerned about sending a signal that math and science literacy are not considered important. In reviewing the exams themselves, it becomes clear that the "written" portion is not really about math and science literacy. The questions which I have reviewed could very easily be covered in numeric or multiple choice without any loss of validity. These are not essay questions. Even those requiring short answer lend themselves to cryptic phrases or even single words for which the marker looks. From my perspective, the issue of math and science literacy is important and most appropriately dealt with in the classroom on a continual basis using assessment of, for and as learning tools. Our qualified teachers have the opportunity daily to make informed observations and judgments looking at student performance. Assessment in the classroom is a very important and substantive part of the process and should not be discounted.

The departmental exam is weighted at 50% of the final mark. The purpose for this is to provide for a standard across the province which is valid and respected. It "evens up" variations in marking from classroom to classroom and district to district such that a mark on an Alberta Diploma and indeed the Diploma itself has meaning. The Alberta Diploma and marks therein are accepted at par by post-secondary institutions across the world. That is not the case for some other provinces. As such while there have often been suggestions that the departmentals are over weighted at 50%, any change in the weighting must continue to protect the overall validity of the diploma. That said, there is no reason why we should not discuss weighting — particularly in the area where we have dropped the written portion — to for example allow that portion of the mark ascribed to the written portion (15%) be also assigned at the classroom level — if appropriate standard assessment rubrics and tools can be developed and used to continue to maintain consistency in assessment across the province.

This has been a very interesting and learning experience. I asked people to look at things that could be changed. I wanted to show that I meant to take action. I started with a change to departmental examinations; an area no one ever thought could be changed. I did it based on data and reason — an informed choice responding to student and system demand. I normally would not move without more consultation however believed that in this case it was very clear the extra effort by students and the extra cost to the system did not drive extra value in terms of success for students, and I believed it very important to make a change which would be considered significant to demonstrate that I clearly mean everything is on the table. I believe it was the right choice because I have not seen any evidence that shows me the short writing questions on the math and science exams benefit students. This process has reinforced however the knowledge that people are very passionate about education and about their children and that there are many opinions and views to consider. Our processes must be nimble, but they still must be collaborative and allow all voices to be heard. Lesson learned.

SUMMARY

The exam results for Part A and Part B are essentially the same therefore both are not required. With some modest changes to the Part B exam the concepts examined in Part A can be covered validly with Multiple Choice and numeric response. Learner Assessment advises that the correlation between part A and part B is very high.

Some say removing the written does not allow for students to show their work or get part marks thus hurting students who do better in that part — but this is more perception than reality as demonstrated by the high correlation between the two parts.

Some are concerned that this signals that Math and science literacy are not important. The written tests in fact do not test math and science literacy. It should not be necessary to drive teaching to the exam. Teachers in classrooms are better positioned to ensure literacy is included, developed and assessed.

Weighting of exams is open for discussion, but any change must meet the criteria of providing a valid, provincial standard to maintain the integrity of the Diploma.

While decisions should be reasoned and informed with data, our processes must be nimble yet collaborative, and in all cases results must be student centred.