Overview

The Dossier Diploma Exam Module is a useful tool for easy access to the content supplied by Alberta Education as individual files (pdf summaries, raw data files). The goals of the module are:

• Easy access to the data for administrators (not having to track down/request the individual files)

  • Access to data based on users PowerSchool access (which schools are available to view)

• Expanded reporting based on data tables sourced from the pdf files to show charts and data visualizations for the data

• Additional reporting in the form of trends

• Individual student view (all diploma exams written with details

Importing

The main goal of the import feature is to allow the source .zip file from Alberta Education (single writing term results) to simply be dropped into the import page, with Dossier then extracting all the raw data from the following files:

• Authority Level Student Results

  • example: ELA3105-001-[A.####]-Student.csv

• Authority and School Level question results

  • example: ELA3105-001-[A.####]-Tble7.2.csv or ELA3105-001-[S.####]-Tble7.2.csv

Setup

At this point in time there is no ability to get all the setup information required for each exam directly from Alberta Education. Some examples of these items are:

• Exam Structure and naming and max score of components

  • Exam general break down: Reading (Equated)/Written Response, Multiple Choice/Numerical Response, …

  • Exam detail break down: Reporting Categories, Topic, …

• Provincial Scores

  • Provincial results for the -Tble7.2 files are available but

  • Participation numbers, Exam Results (averages, A, B, C, F, …) and general break down are only located in the pdf files

To be available for import these values need to be entered into the database.

Structure

The current structure starting in 2018 (missing the pandemic years) is available in the following file:

Open

The first worksheet “Main Section Cutoffs” show the Sections of each exam and how they are displayed/combined in the reporting. In some cases there is a Rollup Name where multiple marked categories are reported as one such as the following selection in which Multiple Choice and Numerical Response (although they have different max scores) are reported together on the summary page in Dossier (similar to the pdf reports)

Open

The second worksheet “Exam Components Max Score“ shows for each exam the lower level breakdown available on the student page. This is a very key part of the set up as the import uses this definition to import the columns in the student results file after the main exam results (common to all result files). The example below shows that the ELA3104 exam for 202308 has 14 of these columns:

Open

Scores

As noted above to be able to compare district or school results to the provincial results the values from the pdf must be entered. This includes:

• Table 1

  • Provincial % of Student who achieved Acceptable or Excellence

  • Provincial # of Students

• Table 2

  • Provincial results for A, B, C, F results for School-awarded, Diploma and Blended mark as well as Average Score and Standard Deviation

• Table 5

  • Provincial Result numbers

The following image shows all the data that currently must be manually created prior to importing successfully, noting Overall, School Awarded, Diploma and Blended Mark

As all this data is inputted manually:

1. There is time required before current results can be imported and available

2. As careful as a developer is, mistakes can be made entering the information for up to 318 exams a year

The Future

We will endeavor to engage with Alberta Education to have this data available in electronic form so that on import Dossier can automatically get the cutoffs, section information and provincial results either directly from Alberta Education or from a staging area that we can populate via data supplied.

Results

As any system calculating results from raw data, there can be discrepancies. There are 3 main items to detail.

Rounding

Since we do not know exactly what functions/rules are used by Alberta Education to calculate scores from raw data, it is possible to see slight (typically decimal value) discrepancies.

Form Differences

Since Dossier Reports irrespective of form, this can be seen in Dossier - Diploma - Average Scores and Standard Deviations content (source - Table 5 in the Diploma Exam pdf reports).

We utilize Form 1 (unless there is no Form 1 Examples) to display the Provincial Scores and Standard Deviation, so this will differ from the content in a Form 2 pdf report.

As well, all students are used in the calculation of School/Division results so the numbers will not match exactly the pdf results (if multiple forms were written) but will be in a range between the 2 for the most part.

Blended Mark

This is the area most likely to see possible discrepancies between the pdf reports and the data displayed in Dossier. The reason behind this is the source of Blended Mark student values. There are currently 2 possible sources of Blended Mark:

1. General - If the student Historical Grades page in PowerSchool have been had Marks imported:

   1. The School Awarded Mark is the Percent field in the stored grade

   2. The Diploma Exam Mark on this page is not used as we use the imported student diploma exam results

   3. The Final (Blended) Mark field (bottom of the stored grades page) identifies the score to be used as Blended Mark in the Diploma module

2. Edmonton Public Schools - EPS supplies their own Teacher Awarded Grades content to be imported and utilized within Dossier. This data set includes values for the School Awarded and Blended marks similar to the description in the General case

In the Dossier Diploma module we identify the course record in the source data as the most recent course result prior to the writing of the diploma exam matching the course code mapping.

The course code mapping allows the connection for the Alberta Education course code to the Diploma Examination record for Diploma Exams taken in French since the Diploma exam code may be different than the source data course code in these instances. The mapping is provided below:

Whatever the source used for Blended Mark, there may be discrepancies as we do not have the raw data used by Alberta Education in the creation of the pdf reports or understanding of the rules used if a student has taken the course multiple times (best, most recent to the exam, etc.)

Specifically for Edmonton Public Schools it was noted that in the raw data, the blended mark added to the source records set it for all instances of the course taken by the student, including taking the course again AFTER the diploma exam. This typically results in a slightly higher calculated aggregate blended mark as a student taking the course again is typically trying to achieve a higher mark.