A performance-audit question and evidence matrix is a design artefact linking audit questions to criteria, evidence, and analytical limits. It guides public service evaluation.
Distinguishing the performance-audit matrix from planning workflows and audit types
A performance-audit question and evidence matrix functions as an analytical design artefact connecting main objectives and thematic audit questions to suitable criteria, indicators, evidence sources, methods, and findings limits. Developing this matrix provides audit teams and public leaders with a structured blueprint for examining public service delivery engagements.
discuss the audit criteria with the audited entity
— INTOSAI ISSAI3000, paragraph49 (planning/conducting requirement for adopting SAIs)
The matrix must be distinguished from broader engagement planning workflows or commissioning mandates, as it focuses specifically on the logical alignment between evaluative questions and evidence gathering. While planning workflows manage project timelines and administrative budgets, the matrix establishes the substantive methodological architecture supporting audit conclusions.
Crucially, governing bodies must avoid confusing the annual audit of performance information with a standalone performance audit. In South African public governance, the Auditor-General South Africa (AGSA) audits reporting against predetermined objectives in annual reports to determine whether published performance information is useful and reliable in terms of validity, accuracy, and completeness.
In contrast, a performance audit is an independent examination evaluating whether management measures ensure public resources are procured economically and utilised efficiently and effectively. While an audit of performance information evaluates the credibility of reported statistics, a performance audit examines underlying operational performance, service delivery impact, and public value creation.
Formulating thematic audit questions and establishing suitable criteria
Under international performance auditing standards, audit objectives are articulated as an overall audit question broken down into thematic sub-questions. As a design check, make sub-questions complementary, minimise unnecessary overlap and check that they collectively address the main objective.
Audit questions are structured around the core principles of public sector performance. Economy questions evaluate whether resources of appropriate quality and quantity are obtained when needed at a suitable price. Efficiency questions examine the relationship between operational inputs and service outputs. Effectiveness questions assess whether outputs achieve intended policy goals.
Establishing suitable audit criteria provides objective benchmarks against which observed operational performance is evaluated. Suitable criteria derive from legal mandates, sound management principles, industry standards, scientific knowledge, or comparative benchmarks.
Applicable engagement criteria and audit mandates must be formally established during matrix design. Discussing proposed criteria with the audited entity ensures a common understanding of benchmarks without diminishing the auditor's independent selection authority.
Identifying evidence sources, qualitative methods, and data reliability
For each sub-question and criterion, the matrix specifies required evidence sources and qualitative collection methods. Identifying evidence requirements during design ensures that procedures gather sufficient and appropriate evidence.
Sufficiency measures evidence quantity, ensuring enough factual data is gathered to persuade a knowledgeable person that conclusions are reasonable. Appropriateness measures evidence quality, requiring data to be relevant, valid, and reliable.
Qualitative collection methods include inspecting operational records, reviewing policy documentation, observing service delivery workflows, interviewing staff, and examining facility conditions. Assessors evaluate data reliability by confirming evidence is supported by corroborating records across independent sources.
To ensure matrix design remains methodologically sound, audit teams should embed several core design routines:
- Structure thematic sub-questions directly around economy, efficiency, and effectiveness principles.
- Derive suitable audit criteria from authoritative legal mandates, policy targets, or sound management practices.
- Identify diverse, qualitative evidence sources to enable data triangulation and corroborate findings.
- Document explicit interpretation limits and scope boundaries for each findings conclusion.
- Differentiate performance audit evaluations from routine audits of published performance information.
Recognising interpretation boundaries, data limits, and institutional scope
A complete performance-audit matrix explicitly records interpretation boundaries and findings limits for each question. Performance audit evidence is often persuasive rather than conclusive, requiring professional judgment and skepticism when interpreting results.
The matrix must document data limitations, unverified operational logs, or scope restrictions to ensure audit reports remain balanced and transparent. Disclosing limitations prevents misinterpreting audit findings and maintains reader confidence in conclusions.
Advisory teams and public institutions must contextualise standard expectations regarding performance auditing. ISSAI 3000 is an international standard developed by INTOSAI for Supreme Audit Institutions (SAIs) choosing to adopt it under their mandates.
Identify the actual engagement mandate and applicable requirements. A reference to ISSAI3000 alone neither grants public audit authority nor establishes standards conformance. This matrix adapts planning principles as recommendations; any binding legislative, contractual or professional requirement should be identified explicitly for the engagement.
In South Africa, public sector performance audits conducted by the AGSA are governed by section 188 of the Constitution and the Public Audit Act. Establish the mandate and applicable requirements before agreeing the scope, evidence access and reporting route.
Annotated four-column performance-audit question and evidence matrix
| Audit Question & Main Objective | Suitable Audit Criteria | Evidence Sources & Qualitative Methods | Interpretation & Findings Limits |
|---|---|---|---|
| Economy Question: Did public health facility procurement acquire infrastructure maintenance services at competitive cost? | Use the applicable procurement requirements, verified contract terms and a justified price benchmark; identify their exact basis before testing. | Inspection of procurement files, tender evaluation logs, supplier contracts, and market price benchmark studies. | Findings evaluate procurement cost efficiency only; evaluation does not measure contractor technical performance. |
| Efficiency Question: Were maintenance teams deployed efficiently to minimise facility downtime across regional health clinics? | Agree a justified criterion for deployment and response performance using actual service commitments and reliable operational data. | Review of maintenance dispatch logs, job cards, facility downtime records, and technician interview notes. | Findings assess resource deployment efficiency; results do not evaluate medical staff operational performance. |
| Effectiveness Question: Did infrastructure maintenance restore facility functionality and meet community health service goals? | Compare completed work with verified contract specifications and the service functionality the repair was intended to support. | Physical site inspection logs, facility manager feedback forms, and clinic service operational records. | Findings evaluate service delivery restoration; results do not measure broader regional health outcomes. |
| Oversight Question: Did management institute effective oversight measures to monitor contractor performance and quality? | Identify the actual required review and acceptance steps from the approved mandate, contract and institutional controls. | Inspection of supervisory sign-off sheets, quality inspection reports, and payment authorisation vouchers. | Findings evaluate internal supervisory controls; analysis does not constitute a legal fraud investigation. |
Sources
Frequently Asked Questions
How does a performance-audit question and evidence matrix differ from a general audit plan?
A performance-audit matrix is a specific analytical design artefact connecting sub-questions directly to criteria, qualitative methods, and findings limits. A general audit plan manages overall project logistics, staffing, budgets, and administrative schedules.
Why is an audit of performance information distinct from a standalone performance audit?
An audit of performance information checks published reports for usefulness and reliability (validity, accuracy, and completeness). A performance audit independently evaluates whether management measures ensure resources are procured economically and used efficiently and effectively.
Does ISSAI 3000 impose binding statutory obligations on independent advisory firms?
No, ISSAI 3000 is an international standard created for Supreme Audit Institutions (SAIs) choosing adoption under public mandates. The engagement's actual mandate and any applicable legal, professional or contractual requirements determine its obligations. Referencing the standard alone does not establish authority or conformance.
How should an audit team manage evidence limitations identified in the matrix?
When data is incomplete or unverified, auditors assess alternative evidence sources or adjust testing scope. Any remaining evidence limitations must be explicitly documented in the matrix and disclosed transparently in final reports.
Why must audit criteria be discussed with the audited entity during matrix development?
Discussing criteria ensures a shared understanding of evaluative benchmarks between auditors and management. While discussed with the entity, final criteria selection remains the independent responsibility of the auditor.
For wider context, read the performance audit overview.