Issues register

EdTech Assessment Toolkit

Automated Essay Scoring (Australia)

Automated scoring model

Automated essay scoring (AES) is employed in various contexts and applications. AES systems utilise artificial intelligence and natural language processing techniques to evaluate and grade essays without human intervention. Though it’s widely adopted across diverse educational scenarios, its most prevalent use is in educational testing. AES leverages machine learning methods to classify, score, or rank essays.

In 2018, the introduction of AES was proposed for the National Assessment Program – Literacy and Numeracy (NAPLAN) in Australia. The deployment of AES sparked significant controversy within the Australian school sector, leading to its suspension for marking online writing tests. Currently, automated scoring is only routinely used for certain aspects of NAPLAN. Concerns raised by teachers, teachers’ unions, principals, and parents encompassed the de-professionalisation of teachers, inconsistent infrastructure across Australian schools, an absence of regulation or guidelines regarding recourse options and challenging automated scores in high-stakes testing, and a lack of transparency from examination authorities (Kalervo et al., 2022). Other instances of AES in action include standardised tests like the Graduate Record Examination (GRE) and the Test of English as a Foreign Language (TOEFL).

Snapshot (July 2023)

System task/function: Automated the marking of assignments
Model: Automated Scoring Model
Deployment: Automated Essay Marking
Location of application: Various locations, see NAPLAN for Australia
Rationale for introduction: Automatically grade student’s achievements on the National Assessment Program for Literacy and Numeracy
Vendor: Australian Curriculum, Assessment and Reporting Authority
Pricing: Not disclosed
Data and computation: Student exam data, predetermined marking rubrics
Inequalities/harms: The use of proprietary EdTech raises concerns about independent evaluation of their functionality and impact, including potential harms to different populations, and the technical expertise required to undertake these audits is often beyond the scope of schools (see Kalervo et al., 2022)
Status: Active
Authority/regulation: National level
Unintended consequences: The untransparent use of AES systems undermines trust in the scores produced, students study for the test, rather than improving assessed skills.
Sanction/redress: N/A

References/further reading

Gulson, K., Thompson, G., Swist, T., Kitto, K., Rutkowski, L., Rutkowski, D., Hogan, A., Zhang, V., Knight, S. (2022). Automated Essay Scoring in Australian Schools: Key Issues and Recommendations. White Paper, November 2022. Education Innovations White Paper Series ISSN 2653-6749. Sydney Social Sciences and Humanities Advanced Research Centre (SSSHARC), University of Sydney, Australia.

Hao, K. (2019). This is how AI bias really happens—and why it’s so hard to fix. Retrieved from