I  Introduction

For centuries, the justice system has been centred around one constant: the human intellect. The ability of humans to reason, advocate, analyse and make judgements could not be replaced. Historically, the legal profession has integrated several generations of technological change without altering the fundamental nature of legal work. From quills to typewriters, typewriters to laptops, and the establishment of Westlaw and LexisNexis, each shift improved efficiency without challenging human legal reasoning. Artificial intelligence reflects a qualitative shift from this pattern. AI can produce outputs which require cognitive capacity, rather than only assisting it. AI is now integrated into the day-to-day operations of many firms, assisting with research, analysis, drafting and summarising documents. The efficiency benefits are undeniable, but so are the risks. This article examines the professional responsibility consequences of uncritical AI use through Dayal [2024][1], Valu v MIMA [2025][2] and how courts have since responded to define acceptable AI use in legal practice.

II  The Hallucination Problem

The consequences of negligent overreliance on AI first drew worldwide attention in the 2023 case of Mata v. Avianca, Inc[3]. A New York attorney represented a client in a personal injury claim brought against an airline for an alleged injury caused by a serving cart during an international flight. The plaintiff’s lawyers had filed non-existent judicial authorities, fake quotes and citations that were prepared using ChatGPT, a generative AI (Gen AI) tool. The court sanctioned the attorneys and the law firm for the breach of Federal Rule of Civil Procedure Rule 11.

This case was considered by Judge A. Humphreys in Dayal [2024], confirming that the potential harms of reliance on non-existent authorities identified by the US District Court such as resource wastage and reputational damage were relevant within the Federal Circuit and Family Court of Australia.[4] In Dayal, the solicitor submitted an unreviewed list of authorities and accompanying case summaries to the court which were prepared using an AI tool within a legal management software. Much like in Mata, the authorities generated by the AI tool were non-existent.[5] The solicitor was referred to the Office of the Victorian Legal Services Board and Commissioner (VLSB+C).[6]

Five months later, in January 2025, a similar case arose in NSW of Valu v MIMA [2025]. The applicant sought judicial review in the Federal Circuit and Family Court of Australia of a decision of the Administrative Appeals Tribunal regarding a migration matter.[7] The applicant’s legal representative (ALR) submitted seventeen Federal Court decisions with plausible citations and names as well as quotations. Each one was non-existent.[8] The solicitor had prepared their submission with ChatGPT, which had once again fabricated the citations. The solicitor explained that due to health issues and time constraints, they relied on ChatGPT, which was promoted by reputable legal software companies.[9] Judge Skaros noted that a considerable amount of the Court’s time was spent attempting to locate the authorities.[10] The solicitor was referred to the Office of the NSW Legal Services Commissioner (OLSC), the court noted that AI misuse is an increasing concern and there is public interest in the OLSC being aware of such conduct.[11]

What these cases collectively demonstrate is that blind trust cannot be placed in a generative AI output; the output must be verified. In each case, the practitioner received an output which appeared genuine, the citations and case names appeared legitimate. This is not incidental; it is how generative AI operates. AI models are trained on data, and they learn to make predictions by finding patterns within it. When AI models struggle to find real and factual information, they may generate outputs that are factually incorrect but read as legitimate outputs.[12]

III Disciplinary Consequences

A court referral of a practitioner to the legal services commissioner or board is just the beginning. The OLSC and VLSB+C may impose professional consequences including but not limited to a reprimand, an order for training, counselling or supervision, or an order recommending restrictions to practising certificate (VLSB+C may impose directly). In cases possibly amounting to professional misconduct, proceedings in the NSW Civil and Administrative Tribunal (NCAT) may ensue, possibly resulting in an order recommending the Supreme Court to remove the lawyer from the roll[13] (VLSB+C applies directly)[14].

On 19 August 2025, the VLSB+C altered the practising certificate of ‘Mr Dayal’ as identified by the Federal Circuit and Family Court in Dayal [2024]. Mr Dayal was no longer allowed to operate his own law practice, no longer allowed to practice as a principal lawyer, no longer authorised to handle trust money, only permitted to practise as an employee solicitor and was required to undertake a two-year period of supervised legal practice with quarterly reporting to the Victorian Legal Services Board.[15]

The VLSB+C described their regulatory action as a demonstration of their commitment to ensuring legal practitioners only use AI in a way consistent with their professional obligations.[16] It is important to note how this statement was framed, the VLSB+C noted that the summary of authorities was generated with AI, it was not framed as a technological failure; it was a matter of professional responsibility and a message to practitioners to inform themselves on AI if they intend on using it.[17]

IV The Courts Respond: Practice Notes and Procedural Guidance

The judicial referrals, along with the consistent expansion of AI into legal practice have prompted the courts to respond.

On 28 January 2025, the Supreme Court of NSW issued Practice Note SC GEN 23 which sets out what use is and is not permitted as well as the risks of AI. The practice note states that any material that is the subject of a statutory prohibition upon publication must not be used to train the LLM or Gen AI model. Generative AI must not be used in the creation of affidavits, witness statements, character references or any material intended to reflect the deponent or witness’ evidence/opinion and a declaration that GenAI was not used in generating its content must be present. If AI is used in preparation of written submissions or summaries, then the Author must verify its contents, ensuring that citations and authorities exist, are accurate, and relevant. Use of AI to prepare submissions does not absolve the author of their professional obligations.[18]

Practice Note SC Gen 23 states that its application excludes the use of dedicated legal research software. The Victorian Law Reform Commission, ‘Artificial Intelligence in Victoria’s Courts and Tribunals’ Report raises a concern that studies have shown that the hallucination problem persists at significant levels for specialised legal research tools.[19]

The Federal Court of Australia, on 16 April 2026, introduced its ‘Use of Generative Artificial Intelligence Practice Note’ (GPN-AI). The practice note cautions that information provided to generally accessible AI tools may become available to other people and that users do not know where the information is stored and who can access it. The practice note also states that privileged and confidential information must not be entered into a Gen AI tool.[20] This was observed in Helmold & Mariya[21] where the court noted that input of confidential documents into a Gen AI tool which stores and replicates data, may constitute breach of privilege. The practice note addresses the VLRC AI Report[22], it offered guidance that AI tools which are ‘closed’ and operate within a controlled environment where data and prompts are not remembered offer lower risk compared to ‘public’ AI tools. The practice note aligns with this, stating that its guidance is directed at use of ‘public’ AI tools. Lawyers who use ‘closed’ AI tools are expected to preserve the integrity and confidentiality of information obtained by compulsory processes in proceedings.[23]

The most recent of the three, issued on 14 May 2026, Victorian Supreme Court Practice Note SC Gen 25[24] which outlines that the court may direct users to provide further information about court documents produced with AI assistance, users must be prepared to ‘identify specific portions of Court documents which were produced using AI and be able to explain how the output was verified’.

These three practice notes reflect that AI use in legal practice operates in a window between prohibition and non-constraint, and each practice note demands human verification and transparency.

V Conclusion

The cases examined, alongside the wave of responses provided by the Courts, convey a coherent message that the legal profession is not being told to abandon AI entirely, but to take accountability for it and to engage in responsible use. The Federal Court of Australia acknowledged the potential benefits of Gen AI in GPN-AI, stating that Gen AI has the potential to improve efficiency in dispute resolution, reducing costs, improving access to justice and the quality of the administration of justice.[25] In the cases previously observed, the practitioners had mistaken the fluency and confidence of the AI output for accuracy, each case had plausible citations and quotations and seemed complete at face value. This is why reliance without review is so dangerous.

The court response clearly outlines what entails responsible use. As stated in Practice Note SC Gen 23, AI may be used for generating chronologies, indexes and witness lists; preparing briefs or drafting Crown case statements, summarising or reviewing documents and preparing written submissions or summaries of argument. When preparing argument summaries or written submissions, the author must verify that the information is accurate, relevant and that authorities exist.[26]Practitioners must also act with care and diligence to ensure that confidential and privileged information is kept private, meaning that this information should not be uploaded to a ‘generally accessible’ AI tool and should use ‘closed’ AI tools to preserve the integrity of the information. However, if outputs from closed AI tools are used for different purposes, it may constitute breach of obligations.[27]

The cases discussed covered the risks for solicitors when there is uncritical reliance on AI, however, there is a deeper issue to acknowledge. In Valu v MIMA as noted before, a considerable amount of the Court’s and associates’ time was spent trying to locate the non-existent cases.[28] This time should have been spent on other matters. The message here is that responsible use of AI is not only about individual compliance, but about maintaining the integrity and reliability of the justice system.

VI References

[1] Dayal [2024] FedCFamC2F 1166 (‘Dayal’).

[2] Valu v Minister for Immigration and Multicultural Affairs (No 2) [2025] FedCFamC2G 95 (‘Valu v MIMA’).

[3] Mata v. Avianca, Inc, 678 F.Supp.3d 443 (S.D.N.Y. 2023) (Mata v. Avianca).

[4] Dayal (n 1), [11]-[12].

[5] Ibid [1]-[5].

[6] Ibid [21]-[22].

[7] Valu v MIMA (n 2), [1]-[3].

[8] Ibid [4].

[9] Ibid [22].

[10] Ibid [10].

[11] Ibid [37]-[38].

[12] ‘What Are AI Hallucinations?’ Google Cloud (Web Page) <https://cloud.google.com/discover/what-are-ai-hallucinations>.

[13] ‘Disciplinary Action’ Office of the NSW Legal Services Commissioner (Web Page, 04 December 2023) <https://olsc.nsw.gov.au/complaints/information-about-complaints-for-the-profession/disciplinary-action.html>.

[14] ‘Our Powers’ Victorian Legal Services Board + Commissioner (Web Page, 05 June 2021) <https://lsbc.vic.gov.au/about-us/board-and-commissioner/our-powers>.

[15] ‘Statement on Mr Dayal Matter’ Victorian Legal Services Board + Commissioner (Web Page, 02 September 2025) <https://lsbc.vic.gov.au/news-updates/news/statement-mr-dayal-matter>.

[16] Ibid.

[17] Ibid.

[18] Supreme Court of New South Wales, Practice Note SC Gen 23: Use of Generative Artificial Intelligence (Gen AI), 28 January 2025 (‘Practice Note SC Gen 23’).

[19] Victorian Law Reform Commission, Artificial Intelligence in Victoria’s Courts and Tribunals (Report, October 2025) (‘VLRC AI Report’).

[20] Federal Court of Australia, Use of Generative Artificial Intelligence Practice Note (GPN-AI), 16 April 2026 (‘GPN-AI’).

[21] Helmold & Mariya (no 2) [2025] FedCFamC1A 163 (‘Helmold & Mariya’).

[22] VLRC AI Report (n 19).

[23] GPN-AI (n 20).

[24] Supreme Court of Victoria, Practice Note SC Gen 25: The Use of Artificial Intelligence by Court Users, 14 May 2026.

[25] GPN-AI (n 20) para 1.3.

[26] Practice Note SC Gen 23 (n 18) para 9B, 16-18.

[27] GPN-AI (n 20) para 4.13-4.15.

[28] Valu v MIMA (n 2), [10].