Technological Due Process

Citation: Danielle Keats Citron (2007) Technological Due Process. Washington University Law Review (RSS)
Internet Archive Scholar (search for fulltext): Technological Due Process
Download: http://ssrn.com/abstract=1012360
Tagged:

Summary

Quote:

"This century’s automated decision making systems combine individual adjudications with rulemaking while adhering to the procedural safeguards of neither."

Author categorizes and provides case studies of policy automation failure and causes.

Author's recommendations include:

• Rules-vs-standards literature can guide what to automate
• Decisions best addressed with standards, explicitly or implicitly requiring human discretion, should not be automated.
• Address providing adequate notice
• Generate audit trails supporting automated decision
• Administrative hearing officers should receive training in automation bias, other fallibility
• Require administrative hearing officers to explain reliance on automated decision, in detail
• Automated systems should be designed with transparency and accountability as primary objectives
• Automated systems source code should be released to the public
• Automated systems must be tested
• Explore ways for the public to participate in building of automated systems
• Consider refraining from automating policy that has not undergone formal or informal rulemaking procedures

Theoretical and Practical Relevance

Quotes re transparency/opacity:

The opacity of automated systems shields them from scrutiny.28 Citizens cannot see or debate these new rules.2

Automated public benefits systems arguably flout these transparency mandates. For instance, the source code of CBMS has not been published for review.

Moreover, even if the trade secrets exemption applies here, the refusal to produce the code of automated decision-making systems allows agencies to enforce laws that no one can see or monitor, which is the very antithesis of open government.

Normatively, then, policy changes accomplished through code should be reviewed with no deference at all. In practice, however, the reverse is true. Because code is hard to unearth, and because of automation bias, encoded policy tends to receive even more powerful deference than that provided under Chevron.