I INTRODUCTION

Law has always been tempted by the language of certainty. Legislatures seek clarity, courts seek consistency, and administrators seek efficiency. In New South Wales, these impulses have become increasingly pronounced as legal institutions confront mounting caseloads, resource constraints and public demands for speed. Against this background, artificial intelligence is often presented as an answer rather than a question: a neutral technology capable of streamlining decision-making, eliminating inconsistency and reducing human error.

Yet this framing obscures a fundamental truth. Law is not merely a system for producing outcomes; it is a human practice of judgment, explanation and responsibility. The maxim lex non est machina captures this insight. Law does not operate automatically. It requires interpretation, discretion and moral reasoning. The danger posed by artificial intelligence to the NSW legal system lies not in technological malfunction, but in the gradual re-imagining of law as something that can be optimised, automated and detached from human judgment.

The argument can be made that the increasing use of AI and automated decision-making in NSW risks undermining core principles of legality unless constrained by a clear commitment to human judgment. The rule of law, procedural fairness and judicial accountability all presuppose decision-makers capable of reasons and responsibility. If those qualities are displaced by opaque systems, law may continue to function—but it will no longer judge.

II THE RULE OF LAW IN NEW SOUTH WALES

The rule of law in NSW is grounded in the idea that public power is exercised by identifiable legal actors. Judges, magistrates, ministers and officials are named not as formalities, but because law insists that authority be traceable. Judicial review, statutory interpretation and administrative supervision all depend on the existence of a decision-maker whose reasoning can be examined and whose authority can be challenged.

This principle finds its clearest constitutional expression in Kirk v Industrial Court of New South Wales, where the High Court held that State Supreme Courts must retain supervisory jurisdiction to correct jurisdictional error.[1] The decision affirmed that legality is not exhausted by compliance with procedure or efficiency of outcome. Rather, legality requires that decisions be made according to law and remain amenable to judicial scrutiny. That scrutiny presupposes reasons, discretion and judgment.

Similarly, in Plaintiff S157/2002 v Commonwealth, the High Court described judicial review as an essential characteristic of the rule of law.[2] Although a Commonwealth case, its reasoning resonates deeply within NSW public law. Judicial review is not simply a mechanism for correcting mistakes; it is a constitutional safeguard against unaccountable power. Where decision-making is mediated or dominated by algorithmic systems whose internal logic cannot be interrogated, that safeguard is weakened.

Automated decision-making complicates these foundational assumptions. If a system generates or materially determines an outcome, identifying the true decision-maker becomes difficult. Is it the official who approved the system, the programmer who designed it, or the algorithm itself? NSW public law has no satisfactory answer to this question, because it has always assumed that law governs people, not processes.

III AUTOMATED DECISION MAKING

Although Robodebt was a Commonwealth scheme, it remains the most instructive Australian example of how automation can corrode legality. In Amato v Commonwealth, the Federal Court held that the automated income-averaging process used to raise debts was unlawful because it lacked statutory authority.[3] The legal reasoning was straightforward. The deeper significance of the case lay in how automation facilitated systemic illegality.

The Royal Commission into the Robodebt Scheme revealed an administrative culture that treated system outputs as presumptively correct and individual hardship as collateral.[4] Decision-makers relied on automation to such an extent that legal responsibility was diffused across departments and contractors. The system produced decisions, but no one owned them. Individuals attempting to challenge debts encountered an apparatus incapable of explanation or discretion.

For NSW agencies increasingly deploying automated systems in areas such as compliance, licensing and eligibility assessment, Robodebt provides a clear warning. Automation does not neutralise power. It reshapes it. When statutory powers are exercised through systems rather than judgment, the risk is not merely technical error but structural illegality. Administrative law in NSW has long insisted that power be exercised for proper purposes, reasonably and according to law.[5] Those requirements do not disappear because a computer is involved.

IV PROCEDURAL FAIRNESS, REASONS AND THE LIMIT OF AUTOMATION

Procedural fairness occupies a central place in NSW administrative law. From Kioa v West onwards, Australian courts have emphasised that affected persons must be given a meaningful opportunity to be heard unless excluded by clear statutory language.[6] This requirement is not ornamental. It reflects a deeper commitment to dignity, participation and restraint in the exercise of power.

The giving of reasons is a critical aspect of this commitment. In Wainohu v New South Wales, the High Court held that the absence of reasons undermined the integrity of the decision-making process and public confidence in justice.[7] Reasons are not merely explanations of outcome. They are demonstrations of responsibility. They show that a decision has been reached through reasoning rather than calculation.

Artificial intelligence challenges this understanding. While AI systems can generate text purporting to explain decisions, such explanations lack the qualities that give legal reasons their normative force. They do not reflect deliberation, cannot respond to individual circumstances in a moral sense, and cannot be held accountable for error. If procedural fairness is reduced to interaction with an automated interface, it risks becoming formalistic rather than substantive.

NSW courts have consistently resisted formalism where it undermines fairness.[8] Allowing machine-generated explanations to substitute for human reasons would represent a significant retreat from this tradition.

V AI AND RISK ASSESSMENT IN CRIMINAL JUSTICE

The criminal justice system presents one of the most sensitive contexts for algorithmic decision-making. Under the Bail Act 2013(NSW), courts must assess whether an accused person poses an unacceptable risk if released.[9] This assessment already involves judgment under uncertainty, but it remains anchored in evidence, discretion and proportionality.

The introduction of algorithmic risk-assessment tools would fundamentally alter this process. Such tools promise consistency and objectivity, yet international experience suggests that they often reproduce historical bias embedded in data.[10] Australian scholarship has cautioned that algorithmic risk tools may exacerbate the over-representation of Aboriginal and Torres Strait Islander peoples in custody.[11] In NSW, where these disparities are acute, the stakes are particularly high.

Criminal law has long resisted the idea that individuals should be judged by prediction rather than conduct. The presumption of innocence, the requirement of proof beyond reasonable doubt and the emphasis on individualised sentencing all reflect this resistance. To allow predictive algorithms to dominate bail or sentencing decisions risks transforming criminal justice into a system of risk management rather than moral accountability.

VI JUDICIAL REASONING AND THE IRREDUCABILITY OF JUDGEMENT

Judicial reasoning in NSW is not a mechanical exercise. Courts interpret statutes, develop common law principles and weigh competing values. This process involves discretion, context and judgment. As Justice Gageler has observed, legal reasoning involves reasoned elaboration rather than mere application.[12]

AI systems operate differently. They identify patterns, correlations and probabilities. Even when accurate, they lack the capacity to understand meaning, context or consequence. This distinction matters because judicial power is justified not by correctness alone, but by legitimacy. Legitimacy arises from transparent reasoning, independence and accountability.

NSW courts have repeatedly emphasised the importance of open justice and public reasons.[13] These principles are not compatible with opaque systems whose internal logic cannot be meaningfully scrutinised. If judicial reasoning becomes dependent on algorithmic recommendations that cannot be interrogated, the authority of the courts themselves may be undermined.

VII PRESERVING THE HUMAN FUNCTION OF LAW

Law does not operate in isolation from social and cultural expectations. Public attitudes towards authority, fairness and legitimacy shape how legal institutions are perceived. Popular representations of automated justice—whether in film, media or policy discourse—contribute to a growing belief that human judgment is inefficient and that automation is inherently superior.

This cultural shift matters for NSW law. If discretion comes to be seen as a flaw rather than a virtue, pressure will mount to replace judgment with systems. Over time, this may erode public tolerance for the uncertainty and deliberation that are inherent in justice. The risk is subtle but profound: law becomes administration, and judgment becomes management.

Legal institutions must resist this drift. The legitimacy of law depends not on speed or consistency alone, but on the perception that decisions are made by accountable humans exercising judgment.

Artificial intelligence has a legitimate role in supporting legal practice. It can enhance research, improve access to information and reduce administrative burden. But it must remain a tool, not a substitute for judgment. The moment decision-making can no longer be traced to a human mind capable of explanation and responsibility, law loses its distinctive character.

Preserving the notion that lex non est machina requires deliberate institutional choices. Legislatures must draft statutes that clearly allocate responsibility. Courts must remain vigilant in insisting on reasons and reviewability. Legal professionals must maintain ethical standards that recognise the limits of automation.

Efficiency is not a vice. But when pursued at the expense of judgment, it becomes corrosive.

VIII CONCLUSION

Artificial intelligence will continue to shape the NSW legal system. Its influence is neither avoidable nor inherently undesirable. But law’s legitimacy depends on preserving its human core. Judgment, discretion and responsibility are not inefficiencies to be engineered away. They are the conditions of legality itself.

Lex non est machina. Law is not a machine. If that truth is forgotten, the legal system may continue to function—but it will no longer do justice.

IX FOOTNOTES

[1] Kirk v Industrial Court of New South Wales (2010) 239 CLR 531.

[2] Plaintiff S157/2002 v Commonwealth (2003) 211 CLR 476.

[3] Amato v Commonwealth (2022) 401 ALR 1.

[4] Royal Commission into the Robodebt Scheme, Report (Commonwealth of Australia, 2023).

[5] Minister for Immigration and Border Protection v SZVFW (2018) 264 CLR 541.

[6] Kioa v West (1985) 159 CLR 550.

[7] Wainohu v New South Wales (2011) 243 CLR 181.

[8] Annetts v McCann (1990) 170 CLR 596.

[9] Bail Act 2013 (NSW).

[10] Julia Angwin et al, ‘Machine Bias’ ProPublica (online, 23 May 2016).

[11] Australian Law Reform Commission, Pathways to Justice (Report No 133, 2017).

[12] Stephen Gageler, ‘The Rule of Law and Australian Constitutionalism’ (2019) 43 Melbourne University Law Review 326.

[13] Hogan v Hinch (2011) 243 CLR 506.