Skip to main content

EdTech Assessment Toolkit

Education counter archive

A tool to explore how a range of technologies, stakeholders, and power relations shape socio-technical systems over time.

How to use the education counter archive

Explore the education counter-archive examples to learn how culture shapes edtech
Select a technology which you would like to know more about in this multidimensional way
Use the template to research your chosen technology and expand dialogue about edtech developments

Background

There are growing calls to surface the historical processes, conditions, and conflicts associated with emerging technologies in education (Williamson and Eynon, 2020). The process of ‘education counter-archiving’ is: “a tool for retracing the cultural records of data infrastructures and inequalities in education” (Swist & Gulson, 2023). This process aims to surface the hidden values, uncertain archives, and ethico-politics which imbue algorithms and automated decision-making (Kitchin, 2017; Agostinho et al., 2019; Amoore, 2021). The purpose of this tool is to understand how different cultural records from the past can surface issues and possibilities for shaping edtech assessment in the present.

There are two examples in the education counter-archive

NYC school choice counter-archive

NYC School choice counter archive
Data, algorithm, & AI-based technologies: NYC school admission matching algorithm

Dataset inputs utilised are student/family choices, plus school and system preferences, within the Department of Education MySchools portal; the computer code executes the task based upon tentatively matching student with school rankings until a stable match is reached (Marian 2021). This cultural record shows how school admissions in NYC are now organised using a matching algorithm so as to assign students to schools more efficiently.

Innovation and industry: Automate the Schools K-12 Information System

In 1988, funded by the State of New York, a bespoke K-12 Student Information system called Automate the Schools (ATS) was written and initially developed over the course of 6 months using Computer Corporation of America’s Model 2014 database management software (Kumar 2000). The ATS system handled high school allocation systems, over 4000 daily administrative operations, and was networked across nearly 1200 official schools. The ATS system included: ‘student registration/enrollment, student demographics, parent and home address information, daily attendance, subject class attendance, health and immunization, Title-1 collection (low-income families data), and elementary and middle school report cards’ (Kumar 2000). The significance of this cultural record is that it begins to surface the scale of infrastructure required to operate increasingly networked school information systems within which automated decision-making, such as school choice algorithms, operates.

Philanthropy and funding: Portfolio School Districts Model

New York is one of over 20 US districts participating in the ‘portfolio school districts’ network which aims to erode older ‘traditional school districts’ in favour of new ‘portfolio school districts’ (Lake and Hernandez 2022). Decentralization, charter school expansion, reconstituting/ closing ‘failing schools’, and test-based accountability are key elements of this approach; such ‘urban school decentralization’ is based on the premise that if ‘education vendors compete on the basis of proposed innovations, with a school super- intendent monitoring activities, children will receive greater opportunity for aca- demic success’ (Saltman 2010: 1). The portfolio agenda has strengthened over time since the early 2000s, with a new range of philanthropic organisations distributing money to push the movement across the USA (Barnum 2017). The value of this cultural record is that it surfaces how philanthropy and funding practices concentrate capital and political power to instill a stock market–inspired style of education reform with closer ties to competing education vendors (Williamson 2018).

Policy and legislation: Charter School Initiative

While over 160 schools were closed in NYC during the tenure of Bloomberg, the charter school initiative was one part of emphasizing choice in the system, as ‘small schools of choice’ (Elwick 2017). However, the closed schools were mostly large high schools in areas of disadvantage (Elwick 2017) and mostly attended by students of colour. The cultural record highlights both how school choice creates forms of comparative data (e.g., on student outcomes) and how school choice algorithms enter heavily racialised choice policies.

Spatiality and socioeconomics: NYC Income and Racial Makeup Map

The links between demography, geography, and schooling are key to understand- ing how school choice operates in NYC. Hemphill and Mader’s (2015) interactive maps suggest there is not always a causal link between school segregation and housing in NYC. The merit of adding this cultural record is that the maps suggest parents are utilising school choice. This record highlights some reasons why choice is seen as necessary in NYC and already established prior to high school and the use of a school choice algorithm

Activism and communities: NYCIntegrate Algorithm Prototype

In 2019, a group of NYC high school and undergraduate students, part of youth-led organisation called IntegrateNYC, joined an all-day hackathon to create an algorithm prototype that could better reflect the city’s diversity. The proposed idea from this event was a new algorithm could potentially ‘boost disadvantaged stu- dents higher up in the matchmaking process, provided they have already passed a school’s screening process’ with the following features and priorities: highly corre- lated with race such as a student’s census tract, whether they receive free or reduced- price lunch, and whether English is their second language (Cassano 2019). The benefit of this cultural record is that it raises the role of particular advocacy groups, such as IntegrateNYC.

KEY TO NYC SCHOOL CHOICE COUNTER-ARCHIVE

Data, algorithm, & AI-based technologies: NYC school admission matching algorithm

  • Dataset inputs utilised are student/family choices, plus school and system preferences, within the Department of Education MySchools portal; the computer code executes the task based upon tentatively matching student with school rankings until a stable match is reached (Marian 2021).
  • This cultural record shows how school admissions in NYC are now organised using a matching algorithm so as to assign students to schools more efficiently.

Innovation and industry: Automate the Schools K-12 Information System

  • In 1988, funded by the State of New York, a bespoke K-12 Student Information system called Automate the Schools (ATS) was written and initially developed over the course of 6 months using Computer Corporation of America’s Model 2014 database management software (Kumar 2000). The ATS system handled high school allocation systems, over 4000 daily administrative operations, and was networked across nearly 1200 official schools. The ATS system included: ‘student registration/enrollment, student demographics, parent and home address information, daily attendance, subject class attendance, health and immunization, Title-1 collection (low-income families data), and elementary and middle school report cards’ (Kumar 2000). 
  • The significance of this cultural record is that it begins to surface the scale of infrastructure required to operate increasingly networked school information systems within which automated decision-making, such as school choice algorithms, operates.

Philanthropy and funding: Portfolio School Districts Model

  • New York is one of over 20 US districts participating in the ‘portfolio school districts’ network which aims to erode older ‘traditional school districts’ in favour of new ‘portfolio school districts’ (Lake and Hernandez 2022). Decentralization, charter school expansion, reconstituting/ closing ‘failing schools’, and test-based accountability are key elements of this approach; such ‘urban school decentralization’ is based on the premise that if ‘education vendors compete on the basis of proposed innovations, with a school super- intendent monitoring activities, children will receive greater opportunity for aca- demic success’ (Saltman 2010: 1). The portfolio agenda has strengthened over time since the early 2000s, with a new range of philanthropic organisations distributing money to push the movement across the USA (Barnum 2017).
  • The value of this cultural record is that it surfaces how philanthropy and funding practices concentrate capital and political power to instill a stock market–inspired style of education reform with closer ties to competing education vendors (Williamson 2018).

Policy and legislation: Charter School Initiative

  •  While over 160 schools were closed in NYC during the tenure of Bloomberg, the charter school initiative was one part of emphasizing choice in the system, as ‘small schools of choice’ (Elwick 2017). However, the closed schools were mostly large high schools in areas of disadvantage (Elwick 2017) and mostly attended by students of colour.
  • The cultural record highlights both how school choice creates forms of comparative data (e.g., on student outcomes) and how school choice algorithms enter heavily racialised choice policies.

Spatiality and socioeconomics: NYC Income and Racial Makeup Map

  • The links between demography, geography, and schooling are key to understand- ing how school choice operates in NYC.  Hemphill and Mader’s (2015) interactive maps suggest there is not always a causal link between school segregation and housing in NYC.
  • The merit of adding this cultural record is that the maps suggest parents are utilising school choice. This record highlights some reasons why choice is seen as necessary in NYC and already established prior to high school and the use of a school choice algorithm

Activism and communities: NYC Integrate Algorithm Prototype

  • In 2019, a group of NYC high school and undergraduate students, part of youth-led organisation called IntegrateNYC, joined an all-day hackathon to create an algorithm prototype that could better reflect the city’s diversity. The proposed idea from this event was a new algorithm could potentially ‘boost disadvantaged stu- dents higher up in the matchmaking process, provided they have already passed a school’s screening process’ with the following features and priorities: highly corre- lated with race such as a student’s census tract, whether they receive free or reduced- price lunch, and whether English is their second language (Cassano 2019). 
  • The benefit of this cultural record is that it raises the role of particular advocacy groups, such as IntegrateNYC.

Boston bus routing counter-archive

Boston bus routing counter archive
Data, algorithm, & AI-based technologies: School time selection algorithm

School districts frequently face challenges in balancing various competing objectives, such as student health, special education programs, staff and parent schedules, regulations, and the concerns of other public stakeholders. To address these diverse needs, the initial algorithm for selecting school start times was developed as a cost-calculation algorithm for the Boston Public Schools Transportation ChallengeThe value of this cultural record is that it demonstrates how this algorithm engaged the following problems: The algorithm introduced an optimization model for the School Time Selection Problem (STSP). The STSP model relied on a school bus routing algorithm called “biobjective routing decomposition” (BiRD). BiRD calculates and merges solutions for subproblems using mixed-integer optimization to determine the best routing option. With BiRD, researchers developed a tractable proxy for transportation costs, paving the way for the formulation of the School Time Selection Problem.

Innovation and industry: Vehicle routing and bell time convergence

The algorithm is based on decomposition strategies commonly used for vehicle routing optimisation (Santini, Schneider, Vidal, & Vigo, 2023). While algorithms in their early development in the 1950s primarily consisted of linear programming, decomposition strategies gained wider adoption in the 1980s to solve the traveling salesman problem. This problem aims to determine the shortest possible route for a salesman, ensuring each city is visited exactly once before returning to the origin city. Today, this method is frequently used in real-time scenarios where dynamic changes, such as a vehicle breakdown or a new customer request, can be accommodated The significance of this cultural record is to show that while school bus routing was not new, using it to determine school bell times was a novel approach (see Bertsimas, Delarue, & Martin, 2019).

Philanthropy and funding: The Boston Public School transport challenge

In 2017 the Boston Public Schools district reached out to the community with transport challenge competition. The district’s transportation costs had been a significant concern, accounting for $110 million or 11% of the district’s budget in FY16. Through this challenge, the district hoped to tap into advances in transportation and mapping technology to develop new algorithms and approaches for efficient and accurate bus routing. The primary objective of the competition was to reduce transportation costs for school districts by optimizing the reuse of buses between schools. This required determining the best school start times and bus routes. The method implemented by the winning solution put forward by the researcher team was generated in just about 30 minutes. This is a stark contrast to the several weeks the district transportation staff typically spend manually building school bus routes using existing software (MIT 2017).  This cultural record signals the ways in which technology-based competitions are sometimes utilised as ways to generate, identify, and embed solutions  to complex problems (with significant operational and financial implications). The district superintendent Tommy Chang highlighted that this bus routing model would not only free up millions of dollars for reinvestment into schools but also significantly reduce traffic and carbon dioxide emissions.

Policy and legislation: Timetabling and bussing management

School bell times have long been debated across research literature. Where later school start times have emerged as a potential policy to improve the sleep and educational outcomes of teenagers (Fuller & Bastian, 2022). However, accommodating later bell times is challenging as it may interfere with peak traffic hours and clash with the schedule of parents. For example, the Boston Public School district operates across three different bell times: 9:30, 8:30 and 7:30. The challenge arose because there is an imbalance in bell times, with more buses serving the 8:30 a.m. slot compared to the other two times. Ideally, the Boston Public School district aims for each bus to serve three schools at each time slot, but the current imbalance leads to inefficiencies and increased costs. Moreover, feedback suggests that many school communities do not favor the 9:30 a.m. start time (Boston Public Schools 2017).  This cultural record indicates that, despite efforts over the years, the Boston Public School district has not found a balanced management approach that considers school preferences. They sought external assistance to address this issue for the 2018-19 academic year, hoping that expertise from the broader community could enhance efficiency, reduce environmental impacts, and benefit nearly 100,000 students and their families.

Spatiality and socioeconomics: An school-by-school learning tool experiment

A central aspect of the controversy surrounding the bell times in the Boston school district is rooted in spatial and socioeconomic considerations. This school-by-school tool experiment allows users to see the impacts of the algorithm to particular schools. Furthermore, civil rights advocates noted that parents of color more likely to have lower-wage jobs would be disproportionately impacted due to difficulties in changing schedules, or paying for additional child-care (Scharfenberg 2018). The merit of this cultural record is to indicate the spatial and socio-economic tensions associated with an algorithm which sought to challenge the existing norms by emphasizing considerations of equity. However, it encountered resistance from a vocal group of higher-income families, predominantly white, in a school district where 85% of the students come from non-white backgrounds. The school district asserts that the prevailing bell times disproportionately disadvantage the majority of these students (Ito 2018). This opposition highlighted the complexities of addressing equity in a diverse educational environment.

Activism and communities: Change.org petition

Ultimately, the algorithmically determined changed bell times were not implemented by the Boston Public School district. The initial plan was to change 85% of the schools’ start times, with a median change of one hour. This magnitude of change led to strong vocal opposition from some school communities that would have been affected negatively, evident in an online petition to stop immediate changes on school start times in Boston. Researchers note that the new bell times were opposed by white parents (see Ito 2018) as well as civil rights organisations (see Scharfenberg 2018).  The value of this cultural record is the role different communities play in raising concerns about introduced solutions. The team collaborated with the city and engaged with the community to enhance the original algorithm by considering equity as a factor in partnership with the Boston school system (Boston Public Schools 2017). Their revised plan for school start times considered equity, as the existing times primarily disadvantaged lower-income families, and recent research indicating potential health and economic drawbacks of early school starts for teenagers.

KEY TO BOSTON BUS-ROUTING COUNTER-ARCHIVE

Data, algorithm, & AI-based technologies: School time selection algorithm

  • School districts frequently face challenges in balancing various competing objectives, such as student health, special education programs, staff and parent schedules, regulations, and the concerns of other public stakeholders. To address these diverse needs, the initial algorithm for selecting school start times was developed as a cost-calculation algorithm for the Boston Public Schools Transportation Challenge
  • The value of this cultural record is that it demonstrates how this algorithm engaged the following problems: The algorithm introduced an optimization model for the School Time Selection Problem (STSP). The STSP model relied on a school bus routing algorithm called “biobjective routing decomposition” (BiRD). BiRD calculates and merges solutions for subproblems using mixed-integer optimization to determine the best routing option. With BiRD, researchers developed a tractable proxy for transportation costs, paving the way for the formulation of the School Time Selection Problem.

Innovation and industry: Vehicle routing and bell time convergence

  • The algorithm is based on decomposition strategies commonly used for vehicle routing optimisation (Santini, Schneider, Vidal, & Vigo, 2023). While algorithms in their early development in the 1950s primarily consisted of linear programming, decomposition strategies gained wider adoption in the 1980s to solve the traveling salesman problem. This problem aims to determine the shortest possible route for a salesman, ensuring each city is visited exactly once before returning to the origin city. Today, this method is frequently used in real-time scenarios where dynamic changes, such as a vehicle breakdown or a new customer request, can be accommodated
  • The significance of this cultural record is to show that while school bus routing was not new, using it to determine school bell times was a novel approach (see Bertsimas, Delarue, & Martin, 2019). 

Philanthropy and funding: The Boston Public School transport challenge

  • In 2017 the Boston Public Schools district reached out to the community with transport challenge competition. The district’s transportation costs had been a significant concern, accounting for $110 million or 11% of the district’s budget in FY16. Through this challenge, the district hoped to tap into advances in transportation and mapping technology to develop new algorithms and approaches for efficient and accurate bus routing. The primary objective of the competition was to reduce transportation costs for school districts by optimizing the reuse of buses between schools. This required determining the best school start times and bus routes. The method implemented by the winning solution put forward by the researcher team was generated in just about 30 minutes. This is a stark contrast to the several weeks the district transportation staff typically spend manually building school bus routes using existing software (MIT 2017). 
  • This cultural record signals the ways in which technology-based competitions are sometimes utilised as ways to generate, identify, and embed solutions  to complex problems (with significant operational and financial implications). The district superintendent Tommy Chang highlighted that this bus routing model would not only free up millions of dollars for reinvestment into schools but also significantly reduce traffic and carbon dioxide emissions.

Policy and legislation: Timetabling and bussing management

  • School bell times have long been debated across research literature. Where later school start times have emerged as a potential policy to improve the sleep and educational outcomes of teenagers (Fuller & Bastian, 2022). However, accommodating later bell times is challenging as it may interfere with peak traffic hours and clash with the schedule of parents. For example, the Boston Public School district operates across three different bell times: 9:30, 8:30 and 7:30. The challenge arose because there is an imbalance in bell times, with more buses serving the 8:30 a.m. slot compared to the other two times. Ideally, the Boston Public School district aims for each bus to serve three schools at each time slot, but the current imbalance leads to inefficiencies and increased costs. Moreover, feedback suggests that many school communities do not favor the 9:30 a.m. start time (Boston Public Schools 2017). 
  • This cultural record indicates that, despite efforts over the years, the Boston Public School district has not found a balanced management approach that considers school preferences. They sought external assistance to address this issue for the 2018-19 academic year, hoping that expertise from the broader community could enhance efficiency, reduce environmental impacts, and benefit nearly 100,000 students and their families.

Spatiality and socioeconomics: An school-by-school learning tool experiment

  • A central aspect of the controversy surrounding the bell times in the Boston school district is rooted in spatial and socioeconomic considerations. This school-by-school tool experiment allows users to see the impacts of the algorithm to particular schools. Furthermore, civil rights advocates noted that parents of color more likely to have lower-wage jobs would be disproportionately impacted due to difficulties in changing schedules, or paying for additional child-care (Scharfenberg 2018).
  • The merit of this cultural record is to indicate the spatial and socio-economic tensions associated with an algorithm which sought to challenge the existing norms by emphasizing considerations of equity. However, it encountered resistance from a vocal group of higher-income families, predominantly white, in a school district where 85% of the students come from non-white backgrounds. The school district asserts that the prevailing bell times disproportionately disadvantage the majority of these students (Ito 2018). This opposition highlighted the complexities of addressing equity in a diverse educational environment.

Activism and communities: Change.org petition

  • Ultimately, the algorithmically determined changed bell times were not implemented by the Boston Public School district. The initial plan was to change 85% of the schools’ start times, with a median change of one hour. This magnitude of change led to strong vocal opposition from some school communities that would have been affected negatively, evident in an online petition to stop immediate changes on school start times in Boston. Researchers note that the new bell times were opposed by white parents (see Ito 2018) as well as civil rights organisations (see Scharfenberg 2018). 
  • The value of this cultural record is the role different communities play in raising concerns about introduced solutions. The team collaborated with the city and engaged with the community to enhance the original algorithm by considering equity as a factor in partnership with the Boston school system (Boston Public Schools 2017). Their revised plan for school start times considered equity, as the existing times primarily disadvantaged lower-income families, and recent research indicating potential health and economic drawbacks of early school starts for teenagers.