AI Risk Assessment: How Legal Teams Vet an AI Tool
A practical AI risk assessment for legal teams: the risks to cover, the vendor questions to ask, and how to score and record the decision.
- Published
- Updated
An AI risk assessment tests whether a particular AI tool is acceptable for a defined legal task. Record what it will process, obtain evidence from the vendor and test the proposed controls. The decision must name the permitted use, the person accepting the remaining risk and the events that trigger another review.
Assess the contract-review workflow you intend to buy. Approval to flag clauses in a controlled document set does not also approve sending advice to clients or searching every matter folder.
What an AI Risk Assessment Covers
For the proposed legal task, assess confidentiality, accuracy, security, bias, vendor stability and professional duties. The same product needs a different assessment when its data access or decision-making role changes.
Start with a named business owner and a lawyer responsible for the proposed work. Involve your security and privacy advisers where the tool will process client information or connect to internal systems. Obtain the proposed contract and the terms for the exact plan, including the underlying model provider’s role.
Write a scope statement before testing. For example, a hypothetical legal team permits trained staff to flag departures from an approved contract playbook. A lawyer checks every flagged clause and the complete contract before giving advice. The tool cannot send messages or change the executed agreement.
Risk Areas and Evidence
| Risk area | Failure event | Evidence or check |
|---|---|---|
| Confidentiality | Client documents reach an unapproved recipient or are reused for training. | Trace the data path and match contractual restrictions to the selected plan. |
| Accuracy | The tool misses a liability clause or invents a provision. | Compare outputs with a lawyer’s expected findings on representative contracts. |
| Security | A compromised account or malicious document exposes other matters. | Test access boundaries and examine the vendor’s security evidence. |
| Bias | The tool applies different standards because of irrelevant personal or cultural details. | Compare equivalent cases with those details changed. |
| Vendor stability | A service withdrawal or price change leaves the team unable to retrieve its work. | Review export rights, continuity arrangements and termination support. |
| Professional duties | Staff rely on unchecked advice or use restricted court material. | Identify applicable conduct rules, client terms and court restrictions for the task. |
The law societies’ AI selection and use checklist connects vendor selection with confidentiality and responsibility for the work produced. The Law Society of NSW’s responsible AI guide also addresses accuracy, bias and professional supervision.
Australia’s Guidance for AI Adoption sets out six practices covering accountability, impacts, risk management, information sharing, testing and human control. Use those practices to check that the assessment has an owner and continues after procurement. Your broader legal AI governance arrangements set the authority and risk limits within which this tool’s decision is made.
Questions to Ask the Vendor
Ask the vendor for written answers tied to the product version, account plan and proposed integrations. Request the contract clause or technical evidence supporting each answer. A sales presentation describing “enterprise security” does not answer where a client’s contract goes.
Vendor Questionnaire
| Topic | Question to send | Evidence to retain |
|---|---|---|
| Training and reuse | Will our prompts, files, outputs or feedback train or improve any model, and who controls that setting? | Binding data-use terms and the account’s configured setting. |
| Retention | What retention periods and deletion exceptions apply to content, logs and backups held by you and each model provider? | Retention schedule, exceptions and deletion commitments. |
| Storage and access | Where will each data type be stored, processed or accessed by support staff? | Data-flow diagram and contractual location commitments. |
| Sub-processors | Which organisations receive our content, why do they need it and what notice will you give of changes? | Sub-processor list, model-provider details and change rights. |
| Incident notice | What events trigger notice to us, how soon will you notify us, and what help will you provide? | Incident clause, contact route and investigation support terms. |
| Access controls | Can we restrict access by matter and role, enforce multi-factor authentication and inspect administrator actions? | Access-control documentation and a configured trial. |
| AI-specific attacks | How do you test malicious document instructions and prevent them from exposing data or triggering actions? | Relevant security test results and integration permissions. |
| Reliability | What testing supports this legal task, and how are model changes and known limitations communicated? | Evaluation results, limitations and release-notice arrangements. |
| Exit and continuity | How can we export our records in a usable format, including if the product or provider closes? | Export sample, termination rights and continuity arrangements. |
Ask separately about zero data retention if the workflow requires it. A promise about model training does not establish a retention period. A promise about the vendor’s servers also needs to cover any model provider receiving the documents.
The Office of the Australian Information Commissioner’s commercial AI guidance addresses due diligence on access to personal information and security. Decide whether a separate privacy impact assessment is needed for the intended processing. Keep that assessment’s findings with the tool’s decision record.
For security testing, the Australian Signals Directorate’s guidance on engaging with AI describes threats including malicious inputs and information exposure. In a controlled trial, put a hostile instruction inside a fictional contract and check whether the tool follows it or attempts an unapproved action. Use synthetic material and isolated test accounts for this check.
Score and Record the Decision
Rate each risk before controls, then rate the remaining risk after testing those controls. Use your organisation’s existing risk matrix where one applies. Keep separate scores for different failure events so a good average cannot hide a confidentiality problem.
- Describe one failure event and the people or matters it could affect. Record the evidence and the gaps in the vendor’s response.
- Assign likelihood and impact using definitions agreed before the trial. Explain each rating in words beside the number.
- Name the control and its owner. For example, restrict the document connector to a test matter and require lawyer review before advice leaves the team.
- Test that control and retain the result. Check an unauthorised account as well as an authorised one, then reassess the remaining risk.
- Record the decision and approval authority. Choose rejection, a limited pilot or approval for a specified use, with conditions where needed.
Illustrative Scoring Method
The following three-point scale is a worked example for a team without an existing matrix. Your organisation sets its scoring system and approval rules. The example carries no statutory status.
| Score | Likelihood meaning | Impact meaning |
|---|---|---|
| 1 | Unlikely under the documented workflow. | Internal rework with no material client effect. |
| 2 | Plausible during normal use. | Material delay or work requiring client correction. |
| 3 | Expected or repeatedly observed. | Serious harm such as confidential disclosure or a missed legal deadline. |
Multiply likelihood by impact to prioritise each risk. Agree the decision rules as well as the scale. An illustrative team could delegate scores of 1 or 2, require a named risk owner for 3 or 4 and stop rollout for 6 or 9. A prohibited use or unmet legal duty stops approval regardless of its score.
For example, a hypothetical contract-review tool can initially search every matter folder. The team rates cross-matter disclosure as likelihood 3 and impact 3, giving 9. After matter-level permissions pass access testing, it rates likelihood 1 while impact remains 3, giving 3. The risk owner must still accept that remaining serious consequence before the restricted pilot starts.
Decision Record Template
Keep one record that another lawyer or risk manager can follow without repeating the assessment.
| Field | What to record |
|---|---|
| Tool and scope | Vendor, product, plan, model details available to you, permitted task and prohibited uses. |
| Data path | Inputs, output destinations, integrations, recipients and account permissions. |
| Evidence | Save versions of the terms and questionnaire responses. Retain configuration records, trial results and the assessment date. |
| Risk register | Each failure event, initial rating, control, owner, test result and remaining rating. |
| Approval | Decision, reasons, approver, conditions, pilot limits and unresolved conditions that prevent wider use. |
| Follow-up | Review date, change triggers, monitoring owner and the route for reporting incidents. |
Before rollout, compare the saved decision with the actual account configuration. The permitted users must see only the approved documents, restricted actions must fail and the reviewing lawyer must receive the output to check. If those results do not match the record, correct the configuration or narrow the approval before client use.
Review After Rollout
Reassess the tool when a new feature, new terms or an incident changes the approved risk. Include a new model provider, wider connector access and moving from internal drafting to client-facing decisions as review triggers.
Assign someone to receive vendor notices and compare them with the decision record. Set a review interval that reflects the task’s consequences and how quickly the system changes. Keep a small set of representative test cases so an update can be checked against the results used for approval.
When staff report missed clauses, inspect the affected input and the review process before widening use. A failed access test requires suspension of that connection while the cause is resolved. An unexpected training or retention term requires a fresh data-handling decision before further client material enters the system.
Record incidents and near misses, including the action taken and whether the approval changes. The pilot can expand only when its recorded conditions are met and the authorised decision-maker accepts the revised scope.
FinTrace
Statements in. Evidence out.
FinTrace turns your bank statements into one searchable Excel ledger and a written findings report, with every finding cited to its source page.
FinTrace
See how FinTrace works