AI Assurance Engineering Skills Map: Four Role Families and Five Portfolio Projects

AI assurance asks whether the evidence is strong enough to support a decision about an AI system. A credible practitioner can trace a claim to a control and say what was tested. They can explain the limits of that evidence and show who approved the risk that remains.

This workbook builds on our guide to the nine AI assurance skills. It helps job seekers choose a route into the work, and it gives employers a way to evaluate practical ability. We use the term AI assurance engineering for work that crosses disciplines. Employers advertise the same responsibilities under governance, quality, model risk, security, audit or compliance titles.

The Four Role Families at a Glance

How to Use This Map

Read the two role maps and pick a primary role family. Complete one portfolio project, then connect it to a second one, so a reviewer can follow the evidence from requirement through testing and release and on into monitoring and remediation. Every project has a worked example and a completion test.

For your first project, governance candidates can start with Project 1, technical candidates with Projects 2 and 3, audit candidates with the audit program described above, and compliance candidates with an applicability memo linked to Project 1.

Source basis. The map and the exercises are GRC Careers editorial recommendations. They draw on an assurance-evidence research paper [1], the European Commission's explanation of EN 18286 [2] and the Financial Stability Board's final report on AI [3]. They are not an official competency framework or a clause-by-clause crosswalk of any standard.

Know what each source is. The paper is research. The FSB sound practices are recommendations for financial institutions. EN 18286 is a harmonized standard. The Commission says applying standards remains voluntary and that providers can use another framework to show compliance [4]. Providers of high-risk AI systems that implement EN 18286 get a presumption of conformity with the Article 17 quality management requirements the standard covers [2]. Our EN 18286 discussion relies on the free Commission explanation, not on the full licensed standard. Links and access details are in Primary Sources and Free Learning at the end. Commission Implementing Decision (EU) 2026/2274 published the reference to EN 18286:2026 [5]. Its presumption of conformity is limited to the requirements of Article 17 of the AI Act covered by the standard [6]; other applicable AI Act obligations still require their own assessment.

Role Map 1: Governance and Technical Assurance

Governance and Quality Management

This family makes assurance repeatable. Practitioners define the review process and assign accountable owners. They keep controlled records and make sure changes and exceptions get the right approvals. Search titles include AI Governance Analyst, AI Governance Manager, AI Quality Manager, QMS Specialist and Model Governance Lead. Titles vary by employer.

Skills to demonstrate. AI inventory and use-case intake; process mapping; control design; document and version control; risk acceptance; corrective action; supplier governance; and management reporting. Understand how an organisation-level AI management system connects to the records kept for one AI system.

Technical depth. Read an architecture diagram and an evidence manifest. Identify which model and data versions are in use and how a rollback would work. Engineering may build the tests. Checking that the record is complete belongs in the governance process.

Portfolio evidence. Produce an evidence validity matrix, a RACI chart (who is Responsible, Accountable, Consulted and Informed), an exception workflow and a corrective-action record. Show one change that invalidates earlier evidence, and name the owner who has to resolve it.

Feeder experience. Quality management, operational risk, enterprise GRC, project controls, privacy operations, regulated product development and supplier assurance.

Interview prompt. A vendor changes the underlying model without changing the product name. What needs review before the next release, and who decides whether the system can keep operating in the meantime?

Source connection. The Commission describes EN 18286 as a quality management system built around the AI system lifecycle and designed to sit on top of existing standards such as ISO/IEC 42001 and ISO 9001. It also says providers in regulated sectors can integrate management systems instead of running separate ones [2]. The role recommendations above are our reading of the work it takes to run that connection.

Technical Assurance

This family produces and validates technical evidence. Titles include AI Assurance Engineer, AI Evaluation Engineer, Model Validation Engineer, ML Security Engineer, and MLOps (machine learning operations) Engineer with governance duties.

Skills to demonstrate. Test design; reproducible evaluations; data and code lineage; dependency tracking; version control; continuous integration; telemetry; evidence integrity; and controls that block a release when required evidence is missing or invalid.

Technical depth. Python or another suitable language, Git, structured data such as JSON, test automation, APIs and basic security controls. The exact stack depends on the system. Be ready to explain false positives and false negatives and what test coverage means. Above all, explain how an authentic record differs from an adequate test.

Portfolio evidence. Build the retesting rule and the release gate in Projects 2 and 3. Include tests for an outdated model, an expired evaluation, a changed prompt and an undeclared dependency. A documented manual prototype is a good first step before you write code.

Feeder experience. Software quality engineering, data engineering, MLOps, cybersecurity engineering, model validation and reliability engineering.

Interview prompt. A signed evaluation is genuine, but it was run against an earlier data pipeline. Why could accepting it still produce the wrong release decision?

Role Map 2: Audit and Regulatory Compliance

Internal Audit

This family independently evaluates whether governance and controls work. Titles include AI Auditor, Technology Auditor, Model Risk Auditor, and Internal Audit Manager with AI coverage. The auditor needs access to the evidence and a reporting line that allows independent challenge.

Skills to demonstrate. Risk-based scoping; walkthroughs; testing control design and operating effectiveness; sampling; evidence sufficiency; issue grading; root-cause analysis; and remediation validation. Know the difference between a policy that exists and a control that operates consistently.

Technical depth. Trace a sample from an AI inventory through a release manifest, test results, approvals, production logs and an incident record. Read technical findings well enough to challenge unsupported conclusions, and bring in a specialist when you need one.

Portfolio evidence. Write a short audit program for the release gate. Select a defined sample and inspect rejected releases as well as approved ones. Record deviations. Produce one finding with criterion, condition, cause, risk, action owner and due date.

Feeder experience. Internal audit, IT audit, external assurance, financial-services controls testing, compliance monitoring and security assurance.

Interview prompt. The dashboard says every release passed. What evidence would show whether any release bypassed the gate, or whether a rejected release was later deployed?

Independence. Internal audit may advise on control design, but operational management owns the control and accepts the business risk. A portfolio can show design skills and audit skills together if it keeps the two roles clearly apart and says that it is a simulation.

Regulatory Compliance

This family works out which obligations apply and makes the compliance position traceable. Titles include AI Compliance Analyst, AI Regulatory Specialist, Product Compliance Manager and Regulatory Affairs Lead. Legal interpretation may need counsel or another qualified specialist.

Skills to demonstrate. Analysis of jurisdiction and operator role (for example, provider or deployer); intended-purpose documentation; applicability decisions; obligation registers; monitoring regulatory change; coordinating conformity assessment; documentation requests; and routing incidents or complaints.

Technical depth. Explain what the system does and which decisions people keep. Explain how versions differ from deployments, and what each evaluation actually proves. Spot when a changed use case, supplier, user group or degree of autonomy calls for a fresh applicability review.

Portfolio evidence. Write a dated applicability memo with facts, assumptions, sources, open questions, reviewer and review triggers. Link each applicable obligation to a control, an evidence owner and any gap. State the boundary of any presumption of conformity you claim.

Feeder experience. Product compliance, privacy, legal operations, financial regulation, regulated quality systems, policy implementation and regulatory affairs.

Interview prompt. What can a harmonized QMS standard establish about conformity, and which other obligations still need their own evidence?

For hiring managers. When you write the job description, evaluate demonstrated work and judgment. The titles above are search aids. They are not evidence of hiring growth or salary levels, and they say nothing about required credentials. No single certification covers all four families.

Project 1: Build an Evidence Validity Matrix

Purpose. Decide whether a test or approval still supports the candidate release. An evidence record needs a defined scope; a file name and a signature are not enough on their own. The research paper sets out when a signed evidence record is valid for a candidate. In its replay of 38,083 commits from 714 public Data Version Control pipelines, 3.3% of commits reused a stage whose imported code or referenced configuration had changed (Table 5 in version 1 of [1]). This result describes a replay of public DVC repositories under the study's dependency rules; it is not an estimated failure rate for enterprise AI systems.

The fictional case used throughout. Harbor Services is a fictional company invented for this workbook. Its internal document assistant drafts summaries for staff from approved documents and can create draft tickets. It cannot approve payments or change customer records. Candidate release H-2 uses model M-2, retrieval index R-7, prompt P-4, tool policy T-3 and governance policy G-5. It is a teaching example, and we assume no legal risk classification for it.

Worked Matrix

Build Your Own Record

For each control, record: control ID; claim; risk addressed; whether it is mandatory or optional, and on whose authority; artifact and dependency versions or hashes; intended use and deployment context; test method and dataset; result and limitations; execution date; expiry rule; authorized evidence producer; reviewer; evidence location; and retest trigger.

Decision states. Use Valid, Stale, Missing, Failed or Uncertain, and keep uncertainty visible. A valid record can still fall short for the overall risk decision if the test design or coverage is weak.

Completion Test

Create at least six records: one valid result, one changed dependency, one missing record, one failed test, one context mismatch and one uncertain dependency. Explain the release consequence of each. A reviewer should be able to reproduce the decision without your verbal explanation.

Ownership. Governance defines the evidence policy. Technical assurance records inputs and results. Compliance checks the applicable obligations, and internal audit independently tests whether the process operates.

What to submit. The matrix, a one-page explanation of your validity rules and a change log. Use synthetic data and label simulated tests clearly. Do not present a portfolio matrix as certification or legal compliance.

Project 2: Write a Change-Triggered Retesting Rule

Purpose. Rerun the checks a change affects, and stop old results from being reused without support. Start with the dependency relationships in your evidence matrix. A rule that only checks the visible model version can miss changed code, configuration, data, tools or deployment context.

Worked Change Rules

A Policy Rule You Can Adapt

When a candidate changes an input used by a mandatory control, the earlier result cannot be reused unless a documented equivalence rule shows the change cannot affect that control. When the set of affected dependencies is unknown, hold the release until it has been investigated and tested more broadly. Record the change, the controls it touches, your rationale, the test results, the reviewer and the decision.

Age matters too. An unchanged artifact does not justify reusing a result forever. Set validity periods according to risk and how fast the environment changes, along with any requirement that applies. You have to choose and justify the time limits and thresholds in this exercise yourself, because the cited sources do not supply universal values.

Harbor Services Test Case

In the fictional H-2 release, Harbor Services moves from index R-6 to R-7 and adds a document source with different access labels. Rerun the retrieval and leakage tests along with the access-control checks. Confirm whether the new source changes the intended use or how personal data is handled. Do not throw out unrelated rollback evidence automatically, but check that its own dependencies and validity period still match.

Completion test. Write six change scenarios. Include a harmless formatting change and a transitive code import. Add changed permissions and one change whose effect is unknown. For each, state which checks run and which results may be reused. Then name who approves the decision and what blocks the release.

What to submit. A change-impact table and one decision record, with a short note on the largest blind spot. Technical candidates may implement the rules in code. Others can demonstrate them with a controlled manual workflow.

Project 3: Design an AI Release Gate

Purpose. Make the deployment decision depend on current, trustworthy evidence. A release gate is a control that permits or blocks a candidate. The paper separates the planner, which only decides what to run, from the gate, which decides what may be released and never consults the planner (Section 4 of [1]). This exercise adapts that idea for a career portfolio.

Minimum Decision Logic

Plain-Language Gate Procedure

Load the approved policy and the candidate manifest. Derive the mandatory controls from the authorized classification. Check the evidence for scope, age, integrity and producer, then confirm test results and approvals. Confirm operational readiness last. Block the candidate if any mandatory condition fails. Keep the decision, the manifest, evidence references, policy version, approvers and timestamp.

Exceptions. Keep missing or failed mandatory conditions separate from optional internal checks. Any emergency route must be separately authorized, logged, time-limited and reviewed. An exception process cannot waive an applicable legal obligation.

Negative Tests

Test old model evidence, an expired result, a changed prompt, an unauthorized signer, a missing test, a failed check, changed context, an unauthorized lower risk tier and an undeclared dependency. Explain why an undeclared dependency calls for better discovery rather than stronger signatures. In the paper's own experiments, a dependency the policy never declared passed every gate design it tested (Section 7.1 and Table 3 of [1]).

Completion test. Show one allowed release decision and at least four blocked ones, with reasons. Protect the gate and its policy from the application being checked. Describe who can change the gate and who reviews those changes, and how you would detect a deployment that bypassed it.

What to submit. A decision flow, test cases, a sample manifest and a decision log. Optional code should come with a short README and reproducible synthetic fixtures. A prototype gate is not a complete production assurance system.

Project 4: Run an Agent Incident Tabletop

Purpose. Show coordinated decisions when an AI agent acts outside its approved boundaries. The FSB report says financial institutions should manage AI-related cyber and ICT risks, including, where appropriate, by incorporating AI cyber and ICT risk scenarios into tests and exercises [3]. This fictional tabletop turns that practice into a work sample.

Scenario (fictional). Harbor Services staff find tickets created in an unusual sequence. Logs suggest the assistant tried to reach a restricted document library after a retrieval configuration change. A monitor raised an alert, but the designated reviewer was unavailable. Nobody yet knows whether sensitive content was accessed or whether any outside party was affected.

Facilitation Plan

Assign the Decision Roles

Use an incident lead, a business owner, a security responder, a technical assurance specialist, a compliance or legal reviewer, a vendor liaison and a recorder. Internal audit can observe and later assess remediation. It should not take ownership of operational incident decisions.

Evidence to preserve. The incident timeline; model and application versions; prompts and configuration; relevant data permissions; tool calls and their outcomes; monitoring alerts and delivery records; approvals; and vendor communications. Restrict access and document custody. Apply proportionate retention and privacy controls.

Communication decisions. Identify the affected audiences and any contractual or legal notification questions. Record who decides applicability and timing. Do not assume a universal reporting deadline, and do not assume every agent anomaly is a legally reportable incident.

Completion Test

Produce a two-page exercise report covering facts and uncertainties, key decisions, containment steps, evidence gaps, recovery conditions, and three corrective actions with owners and dates. Record whether the team could actually pause the agent and confirm that it stopped. Separate elapsed time from time spent waiting on a vendor or an approval.

What to submit. The scenario, role list, decision log, after-action report and one revised control. Judge success by the quality of the decisions and by controls that demonstrably operated. A report that pretends the incident held no uncertainty has missed the point of the exercise.

Project 5: Assess Third-Party Monitoring Rights

Purpose. Decide whether a supplier relationship gives you enough visibility and authority to manage an AI system. The FSB report gives third-party AI risk a sound practice of its own. It says financial institutions should include contract terms with third-party AI providers that enable effective oversight, including monitoring rights. The same passage lists access, audit and information rights alongside continuity obligations and termination rights [3]. The checklist below is an editorial procurement exercise. It is not contract language, and it does not claim that every listed right is legally required. For the wider review, see what an AI vendor assessment covers.

Worked Review Checklist

Harbor Services Example

In this fictional case, the vendor offers ninety days of summary metrics but only seven days of customer-level event logs, with no export. Harbor needs to investigate delayed incident reports. Record the gap. Ask for longer event coverage that can be exported, or another way to investigate. Then weigh whether a limited pilot or a different vendor makes more sense. These durations are invented for the scenario. They are not recommended retention requirements.

Proportionate decision. Assess both the control objective and how it is delivered. A secure independent report or a supervised evidence review may meet an objective without unrestricted access to source code. Record why the alternative is adequate and what limitation remains. Name who accepted it.

Completion test. Review a fictional vendor against all eight questions. Mark each item Verified, Partial, Missing or Not applicable, with a reason. Propose three contract terms or assurance requests and one fallback option. End with a residual-risk decision. Keep confirmed evidence apart from what the supplier says.

What to submit. A completed checklist, a one-page procurement recommendation and a draft email requesting evidence. You do not need to send anything to a real vendor to finish the project.

Turn the Portfolio Into Career Evidence

Choose a Four-Week Route

Suggested README. State the problem, the fictional system, the role you played and your sources. Then list the artifacts, methods, results, limitations and improvements. Mark simulated data and manual steps. Show how a reviewer can reproduce one decision. Remove any real customer or employer information, and any personal data.

Candidate phrasing. "Designed a simulated AI evidence validity matrix and release workflow; tested stale, missing, failed and mismatched evidence; documented retesting decisions and incident recovery criteria." Use it on your resume only after you have done the work, and never describe a learning exercise as a production implementation.

Recruiter assessment. Ask the candidate to walk through a decision, then change one fact. Can they find the affected evidence and explain the release consequence, then assign the right owner and say what is still unknown? Assess the reasoning and the quality of the artifact. The number of framework names a candidate drops tells you little.

Where to Take It Next

Connect the finished work to the AI Risk Manager career guide, the AI observability guide, the AI incident response skills guide and our ISO/IEC 42001 and EU AI Act mapping. If you already know the role you want, the roadmaps for AI auditor, AI risk analyst, AI compliance specialist and model risk manager pick up where this map stops, and How to Get an AI Assurance Job covers the search itself.

For structured study, the Training and Certification Academy has 12-week readiness companions for ISO/IEC 42001 and the EU AI Act. When you are ready to apply, open AI assurance jobs show what employers are asking for. Keep legal applicability and the status of standards under review as systems and requirements change.

Primary Sources and Free Learning

[1] Atanu Mukhopadhyay and Shravya Reddy Nalla, "Reusing Assurance Evidence in AI Pipelines: Validity Conditions, a Fail-Closed Release Gate, and a Replay of 714 Public Data Version Control Pipelines," arXiv preprint 2610.27672, version 1, submitted October 20, 2026. https://arxiv.org/abs/2610.27672
A research preprint with free full text. The analysis code, the prototype, the list of repositories with the commits analyzed, the audit labels and the summary results are posted as ancillary files; the full per-commit results are available from the authors. Study the validity model and the dependency limitations. The authors note that public DVC repositories include many course and personal projects and that industrial pipelines may declare dependencies more or less carefully (Section 8.4), so the results do not establish failure rates for enterprise systems.

[2] European Commission, Shaping Europe's Digital Future library item announcing that the first harmonized standard supporting the AI Act was published in the Official Journal, published October 19, 2026, last updated October 21, 2026. https://digital-strategy.ec.europa.eu/en/library/first-harmonised-standard-supporting-ai-act-published-official-journal
A free official explanation of EN 18286:2026 and its presumption of conformity, which covers only the Article 17 requirements the standard addresses. The full standard is sold through national standards bodies. This map does no clause-level mapping of EN 18286 and does not establish conformity.

[3] Financial Stability Board, "Sound Practices for Responsible Adoption of Artificial Intelligence (AI): Final report," October 15, 2026. https://www.fsb.org/2026/10/sound-practices-for-responsible-adoption-of-artificial-intelligence-ai-final-report/
A free report setting out 12 sound practices that financial institutions are encouraged to consider. Read it for governance and lifecycle controls, then for the sections on cyber and supplier risk. The FSB says the practices are non-prescriptive and do not constitute a binding international standard.

[4] European Commission, AI Act standardization page, last updated October 19, 2026. https://digital-strategy.ec.europa.eu/en/policies/ai-act-standardisation
The Commission's overview of how harmonized standards support the AI Act. Source for the point that applying standards remains voluntary.

[5] Commission Implementing Decision (EU) 2026/2274 of October 12, 2026, published in the Official Journal on October 14, 2026. https://eur-lex.europa.eu/eli/dec_impl/2026/2274/oj/eng
The legal act that publishes the reference to EN 18286:2026 (Article 1 and Annex). Free on EUR-Lex.

[6] Regulation (EU) 2024/1689 (the AI Act), consolidated text of July 27, 2026, Article 17 (quality management system). https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27
The operative text of the quality management obligation for providers of high-risk AI systems, including its proportionality and sectoral integration provisions.

The role map and the exercises are GRC Careers editorial recommendations. Harbor Services and everything in its scenarios are fictional.

Where to go next

• Browse the jobs that use these skills
• Follow a career roadmap into the role you want
• Hiring for this? Start from a job description template
• Free certification study games, 652 practice questions

More in this series

• 9 Essential Data Governance Skills for the AI Era
• 10 Internal Audit Skills for Modern Assurance Careers
• 12 Transferable GRC Skills You May Already Have
• Technical vs. Nontechnical GRC Skills: What Employers Actually Need
• AI Governance Skills Employers Actually Hire For
• GRC Analyst Skills: What the Job Actually Requires
• Compliance Analyst Skills
• Risk Assessment Skills
• Controls Testing Skills
• Policy Writing Skills
• Regulatory Change Management Skills
• Third-Party Risk Skills
• Model Risk Management Skills
• AI Impact Assessment Skills
• AI Auditing Skills
• AI Evaluation and Testing Skills for Governance Careers
• Data Lineage Skills
• Data Quality Skills
• Privacy Engineering Skills
• AI Security Skills
• AI Incident Response Skills
• Governance Program Management Skills
• Stakeholder Communication Skills
• Executive Risk Reporting Skills
• Evidence Documentation Skills
• Control Mapping Skills
• Framework Crosswalking Skills
• Vendor Due Diligence Skills
• Responsible AI Skills
• GRC Tools and Automation Skills
• How to Build the 9 Data Governance Skills: A 12-Month Career Plan
• 10 In-Demand Compliance Skills for Today’s GRC Careers
• 10 Essential Risk Management Skills Employers Value
• 10 In-Demand Data Privacy Skills for Privacy Professionals
• 10 Cybersecurity Skills for GRC and Security Careers
• 9 AI Assurance Skills for Audit, Risk, and Governance Professionals
• Writing a Resume for AI Cybersecurity Jobs