Artificial intelligence (AI) is transforming how professionals tackle complex tax problems, from compliance management to strategic planning. In particular, Large Language Models (LLMs) such as GPT-4 have started to show remarkable capabilities in researching legislation, drafting advisory memos, and even detecting potential compliance issues. However, the quality of AI-generated outputs largely depends on how questions (also known as prompts) are formulated. While a straightforward query might sometimes yield a passable result, optimally structuring prompts can lead to deeper insights, improved accuracy, and higher efficiency.

This post introduces three prompting techniques that tax professionals can use when engaging AI tools for tasks like drafting advice, summarising legislation, or reviewing compliance scenarios. Each technique brings its own strengths and technical rationale.

---

Context-Rich Prompting

What is this

Context-rich prompting is about giving the AI model a comprehensive background on the tax problem before asking for a solution. Instead of posing a question like, “What is the compliance obligation for a small business under Australian GST rules?” you might say:

“You are assisting a tax professional. The client is a small business in Queensland with annual revenue of AUD 1.2 million. They file quarterly BAS (Business Activity Statements) and are subject to Australian GST regulations. Please outline their main obligations under GST, referencing key legislative sections from the ATO guidelines.”

Why It Works

From a technical standpoint, LLMs analyse user prompts by looking at patterns they have learned during training. When your prompt is packed with relevant context, like the annual revenue, business structure, and reporting frequency, the model can focus on applicable tax rules. By structuring the question in a “story-like” manner, you give the AI more relevant tokens (i.e., bits of text that it uses to configure its LLM components) to work with. This helps the AI zero in on relevant patterns in its training data, minimizing vague or irrelevant content.

Behind the Scenes

Under the hood, context-rich prompts anchor the AI’s internal attention mechanisms. Modern LLMs use self-attention layers that weigh each piece of input text differently. When an LLM sees specific details—such as “AUD 1.2 million” or “quarterly BAS”—it searches its parameter space for nuances related to those concepts. This process ensures the AI scans and retrieves the right tax law references (assuming they exist in its training) or shapes its text to highlight compliance points. Effectively, you’re guiding the model’s attention to the details you care about, improving the precision of its answers.

---

Step-by-Step or “Chain-of-Thought” Prompting

What is this

Chain-of-thought prompting involves explicitly instructing the AI to break down its reasoning process into smaller steps. For instance, if you want the AI to analyse a potential Goods and Services Tax (GST) scenario, you might say:

“Explain the GST obligations for a freelance software developer earning AUD 90,000 annually. First, outline the key GST registration thresholds in Australia. Then, assess if the developer is required to register. Finally, summarise ongoing compliance requirements once registered.”

Why It Works

LLMs often generate answers based on probabilistic guesses of the next word or phrase. By specifying the logical sequence (e.g., “outline thresholds,” “assess registration needs,” “summarise compliance”), you constrain the model to follow a step-by-step approach rather than jumping straight to a conclusion. Technically, this breaks down the prompt into distinct segments within the AI’s internal context window, encouraging the model to build a structured answer.

Behind the Scenes

When you ask for a step-by-step explanation, you’re leveraging the model’s capacity for “chain-of-thought” reasoning, even though modern AI systems don’t have consciousness or true understanding. Internally, the LLM is generating partial answers for each subtask, then linking them into a coherent final response. This helps ensure each piece of the puzzle is addressed in turn, leading to more comprehensive answers and fewer omissions. For tax scenarios, where multiple statutes or thresholds might apply, this structured logic can significantly reduce confusion.

---

3. Role-Based or Instructional Prompting

What is this

Role-based prompting instructs the model to take on a specific persona or follow a defined set of rules. For example, you might begin with:

“You are a senior tax consultant specializing in Australian corporate tax. Your role is to provide a preliminary assessment, referencing relevant ATO rulings, for a client exploring an international expansion into New Zealand. Offer a concise, bullet-pointed analysis of potential compliance obligations and cross-border taxation issues.”

Why It Works

One of the underlying reasons LLMs respond effectively to role-based prompts is that they were trained on diverse text sources, academic papers, professional documents, and web content. By “assigning” a role or perspective, you essentially tap into the segments of the training data most relevant to that professional context. This helps the AI “locate” the correct tax domain knowledge stored in its parameters, leading to more contextually rich and accurate output.

Behind the Scenes

When the AI receives a role-based instruction, like “You are a senior tax consultant”, it sifts through its internal patterns to replicate the style, rigor, and detail of that role. The model’s training likely included thousands of examples of professional tax documents, advice memos, and so forth. By commanding it to assume this persona, you’re nudging its text-generation algorithms to emphasize formality, authority, and specificity. In the context of Australian tax law, the model will also highlight or reference relevant legislation or guidelines (e.g., from the ATO) when such data aligns with your instructions.

---

Best Practices for Each Technique

While each prompting method offers unique benefits, consider the following tips to get the most out of them:

1. Combine Methods: For example, start with a role-based approach (“You are a senior tax consultant”) and then layer in step-by-step instructions (“First, list corporate tax rates; second, identify common deductions”) to fully guide the AI’s reasoning.
2. Maintain Clarity: Regardless of the technique, clarity in your prompt is paramount. Use direct, unambiguous language and specify the format of the desired response (bullet points, paragraphs, numeric lists, etc.).
3. Review Outputs: Always scrutinise the AI’s response, especially for tax-related matters. AI might rely on outdated or incomplete training data, so cross-verification with the latest legislative documents and professional judgment is critical.
4. Iterate and Refine: If the answer is incomplete, revise and expand your prompt. For example, if the model misses certain legislative nuances, add explicit references to relevant ATO rulings or tax codes in the follow-up prompt.