Act Alike: Are Terms Defined Consistently Across Legislation?

"Personal information" means one thing in Privacy Act 1988 while other Acts cross-reference it, narrow it to a specific section, or point somewhere else entirely. This isn't unusual: Commonwealth legislation defines the same term differently across different Acts, and there's no queryable index of where or how they diverge. Finding out which is which means opening every relevant Act and comparing definitions.

Act Alike does that comparison for you. Type a term, and it lines up every Act that defines it, side by side, sourced straight from the legislation, with a plain-language summary of how they differ.

I built Act Alike as an entry for Innovation Month 2026 (IM2026) "Build a Bureaucrat Bot", an Australian Public Service innovation challenge judged live by five Chief AI and Data Officers, entered under the Complexity Tamer category.

This isn't a hypothetical problem invented for a competition. An independent 2025 paper found the same class of issue in US tax law (arXiv:2511.11954): inconsistent definitions across a body of legislation, undetected because nothing indexes them against each other. A peer-reviewed study found it in Australian animal protection law too.

Any domain with a large, evolving body of documents that individually define their own terms has this problem; legislation just makes it easy to see.

Architecture

1. Vue frontend: takes a search term, renders the comparison, deployed to Netlify
2. FastAPI backend on Modal: a thin wrapper, no extraction logic of its own
3. DefinitionResolver: an already-shipped, already-tested component from lex-au-graph, an open source project that turns Commonwealth legislation into a queryable definition graph
4. Claude API: writes the plain-language difference summary, restricted to describing textual differences and quote-verified before anything is shown
5. Response flows back to the frontend with every claim linked to its source Act

The FastAPI layer exists to call DefinitionResolver.find_all_definitions(term) and hand the results to an LLM under a tight prompt contract.

Implementation

The Claude API system prompt restricts what the model is allowed to talk about:

_SYSTEM_PROMPT = (
    "You are assisting a legislative research tool that compares how a legal term is "
    "defined across different Commonwealth Acts of Parliament. "
    "Describe ONLY observable differences already present in the definition text provided "
    "(different thresholds, different tests, different reference points, different scope). "
    "Do NOT speculate on legislative intent, drafting history, or policy rationale. "
    "Do NOT draw any eligibility or compliance conclusion. "
    "Quote evidence verbatim from the definition text you were given. "
    "Return ONLY valid JSON — no markdown fences, no commentary."
)

Every claimed difference then has to prove itself against the source text before it's shown:

def verify_quote(quote: str, source_text: str) -> bool:
    """Check a quoted passage exists (whitespace/punctuation-insensitive) in source_text."""
    normalised_quote = _normalise(quote)
    return bool(normalised_quote) and normalised_quote in _normalise(source_text)

verified: list[VerifiedDifference] = []
for diff in data.get("differences", []):
    source = by_act.get(diff.get("act_title", ""))
    quote = diff.get("quote", "")
    if source and verify_quote(quote, source.definition_text):
        verified.append(VerifiedDifference(
            act_title=diff.get("act_title", ""),
            quote=quote,
            note=diff.get("note", ""),
        ))
 
if not verified:
    return None

An unverified claim is dropped individually, not the whole response. If nothing verifies at all, the whole summary is dropped and the caller falls back to something deterministic. This pattern, restrict what the model can claim, then check each claim independently, transfers directly to any RAG-shaped system: support docs, internal wikis, product specs, anything where an LLM synthesises across retrieved passages.

Grounding isn't a single switch. It has to be applied at every layer a model touches:

Results

"Child" is a good non-flagship example, because it's genuinely inconsistent across Acts, not just differently worded. Most Acts put the age threshold at under 18. The Income Tax Assessment Act 1936 puts it at under 16. Some Acts don't define it at all and instead cross-reference another Act's definition, which the tool renders as a distinct card style rather than folding it into the same list.

By the time of submission (17 July 2026) the underlying corpus (lex-au, a separate open legislative parsing project) covered 2,942 Acts and 28,746 defined terms; both numbers keep growing as ingestion continues, so treat them as a snapshot rather than a current count.

Where It Goes Next

The reuse-restrict-verify pattern, and the specific gap in it, generalise past legislation. Any tool that puts an LLM in front of a structured domain corpus is a candidate, whether that's legal text, internal policy documents, product specs or support knowledge bases.

I will also be building a few more enhancements to this tool and ingesting more Acts so keep an eye out for new version announcements!

Code and Further Reading

The full source is on GitHub, live at act-alike.netlify.app. The legislative parsing and definition-graph layer it wraps is a separate open project, lex-au-graph.

For the underlying research on why claim-level verification alone isn't sufficient for numeric and entity grounding, Proof-Carrying Numbers (arXiv:2509.06902) and HalluGraph (arXiv:2512.01659) are the two reference architectures behind the framing used here.

AI Tools

Claude Code was used to plan and build the app, and Claude and Google Gemini were used in drafting supporting material for the competition entry.