Education studies

Overview

Alberta works with organizations like the Council of Ministers of Education, Canada (CMEC), the Organization for Economic Cooperation and Development (OECD), and the International Association for the Evaluation of Educational Achievement (IEA) to administer studies assessing Alberta student performance and abilities.

A key objective of Alberta’s education system is ensuring that Alberta students receive a high-quality education and develop knowledge, skills and competencies that will help them succeed in an increasingly global and interconnected world.

Participation in international studies provides Alberta with reliable and internationally comparable information to evaluate Alberta's education system. The information collected in international studies is an important tool to monitor the quality of Alberta’s education system, and can be used to evaluate Alberta’s curriculum and other policies and programs.

These studies also give the Alberta government important information about contextual factors that may impact student performance on the assessments. Some of this information is only made available through the international studies.

The number of countries participating in these studies continues to grow. This is an indication that the value of international studies and data is widely recognized.

Government carefully considers each decision to participate in a pan-Canadian or international study to ensure each study is an appropriate use of research funding, educator/student time and other system resources.

Programme for International Student Assessment (PISA)

PISA is an international assessment of the skills and knowledge of 15-year-old students. It provides information about a range of factors that contribute to the success of students, schools, and education systems.

PISA has been conducted every 3 years since 2000. In each PISA cycle, a different domain (reading, mathematics, or science) is the focus of more in‑depth investigation.

Assessments in Alberta are administered in English or French.

Canada’s fees to the Organisation for Economic Cooperation and Development (OECD) for participation in PISA are paid by Employment and Skills Development Canada (ESDC). There is no direct cost to Alberta or other Canadian provinces.

PISA was created in 1997 by the OECD to monitor student achievement within a common, internationally agreed framework.

Learn more about how PISA works.

Measurement

Participating students:

complete an assessment booklet from the major domain being tested (reading, math or science), and one or more minor domain
answer a short questionnaire, which includes:
- scales to measure their attitudes
- questions to collect information on their backgrounds

School principals also fill out a questionnaire.

PISA assesses students near the end of their compulsory education to see if they have acquired the knowledge and skills that are essential to fully participate in society.

It looks at how reading, math, and science are understood in a way that connects to everyday problems and situations.

PISA does not assess how well students have learned a specific curriculum or if students have memorized facts.

Focus for the assessment changes each administration, but always includes questions from all 3 domains (reading, math, and science). The focus in the 2009 assessment was on reading. In 2012, it was on math. And in 2015, it was on science.

Participation

Approximately 20,000 Canadian students from approximately 1,000 schools take part in each PISA assessment.

In Alberta, approximately 2,500 students from 114 schools participated in PISA 2015.

International participation in PISA has grown steadily:

32 countries participated in 2000
41 countries participated in 2003
57 countries participated in 2006
65 countries participated in 2009 and 2012
72 countries participated in 2015

Since PISA assesses 15 year-olds regardless of how many years of education they may have had, the majority of Alberta students who write are in Grade 10, while some are in Grade 9.

Learn more about PISA participation.

Selection process

Each international participant provides the study organizers with a complete list of all schools in which 15-year-olds are taught. The OECD then draws a sample of schools that will be asked to participate. These schools are then asked to provide a list of all 15-year-old students from which a sample of students, to a maximum of 42 per school, is drawn by the OECD.

The Alberta government and participating schools have no control over this process.

Participants are allowed to request that certain types of schools be excluded from participation. In Alberta, federally-funded schools are excluded, and so are schools with fewer than 5 students who are 15.

Individual students can also be excluded. For example, students with severe cognitive disabilities or students who cannot speak the language of the test can be exempted.

Exclusion rates are carefully monitored and must not exceed 5% of all students in the country or province.

When less than 80% of selected students participate, the country needs to show that those who didn’t participate are similar to those who did. For instance, they should still represent a similar socio-economic background and show similar results on other, similar assessments.

Learn more about the selection process.

Score calculation

PISA assessment scores are calculated to get the OECD average score of 500 within a standard deviation of 100. This means that two-thirds of students in OECD countries would have a score between 400 and 600 points.

Students can be grouped into one of 6 proficiency levels. Level 2 (with the score cut-off point of 410) is considered to be the “baseline level” that is required to participate fully in society.

PISA tests a sample of students, so the scores obtained for Alberta are only estimates. As estimates, they have a certain level of ‘imprecision’. Therefore, when presented in tables, these scores are reported alongside “standard errors.” This is intended to alert the reader to the fact that the “true” score that would be based on testing all students would likely fall within a certain range around the estimate that PISA produces.

This also explains why scores for individual participating countries may differ from each other, but are not necessarily statistically different.

The same applies to changes in scores over time. For example, Alberta scores in all 3 subject matters tested in PISA 2015 are different from the scores obtained in previous years. However, the only statistically significant difference pertains to how mathematics scores in 2012 and 2015 are different from the score in 2000.

Benefits

Participation provides data to help us:

inform policy and program development
better understand the strengths and weaknesses of Alberta 15-year-old students in the key areas of reading, math, and science
learn about best practices from other high-performing jurisdictions
support professional development for teachers to address areas of concern
clarify programs of studies where necessary

While PISA is only one piece of all the data collected by Alberta Education, it is a valuable resource to see how our students perform compared to students around the world in terms of their knowledge and skills.

Participation in several cycles of these studies also provides Alberta with trend data. This shows us how student performance changes over time and how factors contributing to that change in performance evolve.

Trend data helps us to better understand the possible causes of changes in performance over time.

Results

Results over time

Science – Alberta scored 541 in 2015. The results have been stable since 2006.
Reading – Alberta scored 533 in 2015. The score has declined since 2000, but is not statistically different from where it was in 2000 (550).
Mathematics – Alberta scored 511. Alberta’s score is now lower than it was in 2000 (549).

2015 results

2012 results

Questionnaires

One of PISA's most important sets of data are contextual information from student and principal questionnaires. These take about 35 and 45 minutes, respectively, to complete. The responses to the questionnaires are analysed together with the assessment results to provide a broader and more nuanced picture of student, school, and system performance.

The questionnaires gather information about:

students and their family backgrounds, including their economic, social, and cultural backgrounds and resources
aspects of students’ lives, including their:
- attitudes towards learning
- habits and life in and outside of school
- family environment
aspects of schools, including:
- the quality of the schools’ human and material resources
- public and private management and funding
- decision-making processes
- staffing practices
- the school’s curricular emphasis and extracurricular activities offered
context of instruction, including:
- institutional structures and types
- class size
- classroom and school climate
- reading activities in class
aspects of learning, including:
- students’ interest
- motivation
- engagement

The questionnaires from all assessments are available on the PISA website.

Trends in International Mathematics and Science Study (TIMSS)

TIMSS is a large-scale assessment designed to inform educational policy and practice by providing an international perspective on teaching and learning in mathematics and science.

TIMSS was first administered in 1995 and reports every 4 years.

Measurement

Participating students complete tests in mathematics and science achievement, and answer questionnaires on their background and experiences in learning mathematics and science at school.

TIMSS tests are based on assessment frameworks developed after extensive analysis of national curricula and are created to ensure that comparisons between countries are as fair as possible.