P2 Reflection Note — What the Worked Example Revealed
Project: Pickles GmbH — AI Governance Framework Stage: Phase 2 — Worked Example Document: P2-Reflection-Note-v1.md Status: Draft Version: v1 Date: 2026-02-28 Assumptions: This document reflects on the Phase 2 worked example process. It is written in first person as a genuine methodological reflection.
Purpose
This note records what the Phase 2 worked example exercise revealed about the Pickles GmbH governance framework — what held up under realistic content, what needed adjustment, what the exercise surfaced that templates alone would not, and what would change if Vertrag.AI were a real system rather than a fictional one.
It is intended for two audiences: reviewers of this framework who want to understand how it performs under pressure, and practitioners considering adapting the framework for their own use.
What the Exercise Was Testing
Phase 1 produced 16 governance documents: inventory templates, risk classification criteria, regulatory mapping matrices, a DPIA framework, vendor risk assessment, monitoring framework, incident playbook, and commercial packaging. Every document was built on assumed characteristics of a fictional legal AI company.
The risk with any template-first approach is that it produces documents that look complete but haven't been stress-tested against real-feeling content. A risk classification framework is easy to write; applying it to a specific system, with explicit reasoning at every step, is where the gaps show.
Phase 2 asked: does this framework actually work when you put a realistic system through it?
What Held Up Well
The decision tree structure for risk classification worked. Walking Vertrag.AI through five sequential gates — prohibited practices, Annex III, GPAI, Article 50, internal criteria — produced a traceable, defensible result. The value wasn't just arriving at "limited risk." It was the documented reasoning that shows why the system doesn't fall into Annex III, which categories were seriously considered, and what design features keep it on the right side of the line. That reasoning is what a regulator or enterprise client would actually want to see.
The distinction between EU Act classification and internal classification proved its worth. Vertrag.AI came out as limited risk under the EU AI Act but Medium-High on the internal scale. This gap exists because the framework is deliberately more conservative than the regulatory minimum — and that conservatism is defensible and appropriate for a professional-context legal tool. Having two separate classification outputs, rather than collapsing them into one, gives the framework more useful precision.
The human oversight gate proved to be load-bearing across multiple documents. It appeared in the risk classification (as the factor that keeps the system out of high-risk), in the technical documentation (as a hard architectural requirement, not just a policy), in the EU AI Act mapping (as the primary control mitigating Article 50(4) uncertainty), and in the monitoring design (as the source of the acceptance rate metric). This cross-document coherence suggests the framework's architecture is structurally sound — the oversight requirement isn't just stated once and forgotten, it propagates correctly.
The assumptions discipline held. Every document flagged assumptions consistently, and the assumptions accumulated into a coherent picture of what a real company would need to do before treating any of this as operational. That's the right outcome for an open-source educational resource — the framework is useful precisely because it's honest about what it doesn't know.
What the Exercise Surfaced That Templates Alone Did Not
The GDPR/confidentiality monitoring constraint is a real design problem, not a footnote. When I reached the monitoring entry, it became clear that standard LLM monitoring approaches — logging prompt/completion pairs, reviewing output samples freely — are structurally unavailable in the legal market context. You cannot retain full contract text in monitoring logs without violating GDPR and legal professional privilege simultaneously. The monitoring document had to be redesigned around metadata-only logging, outcome proxies, and consented sampling. This constraint wasn't visible in the Phase 1 monitoring framework template, which was written at a level of abstraction that didn't surface it. A practitioner adapting this framework for a real legal AI company would hit this constraint immediately — and the worked example now flags it explicitly.
The Anthropic sub-processor relationship cascades further than expected. By the time I reached the technical documentation and monitoring entry, the Anthropic DPA gap — identified as a single compliance item in the EU AI Act mapping — had become a dependency for: the GDPR legal basis analysis, the data flow description, the log retention design, the client-facing DPA structure, and the monitoring consent framework. A single unresolved vendor relationship creates a thread that runs through almost every downstream document. The worked example makes this visible in a way that the template couldn't.
Article 50(2) — machine-readable labelling of AI-generated text — is genuinely unclear for document-based output. The EU AI Act mapping flagged this as a gap (GAP-001), and it remained open throughout the exercise. Article 50(2) requires machine-readable labelling of AI-generated content, but the practical form of that obligation for a tool that produces a DOCX file is not settled. This is a live regulatory uncertainty, not a drafting gap in the framework. Identifying it is more valuable than pretending it's resolved.
The RAG corpus is a governance object in its own right. The Phase 1 framework treats the knowledge base as a component of the system. The worked example revealed that it needs to be treated as a governed asset with its own version control, curation policy, update cadence, and quality monitoring. A knowledge base that drifts — through outdated legal sources or inconsistent curation — can degrade output quality systematically and silently, in a way that the acceptance rate metric alone might not catch quickly. This deserved more emphasis in Phase 1's technical documentation template than it received.
What Would Change With Real Company Data
The most significant changes when moving from fictional to real would fall into four areas.
Architecture verification. The entire technical documentation is built on an assumed RAG + Claude API architecture derived from Anthropic's legal productivity GitHub examples. A real implementation might differ materially — different chunking strategy, different retrieval approach, a different model, fine-tuning that fundamentally changes the training data section. The framework documents the right questions; real system data provides the answers.
Baseline establishment. The monitoring framework cannot be operationalised until production data exists. The suggestion acceptance rate target (≥60%) is a reasonable starting hypothesis, but it needs 90 days of real usage to validate. Some metrics — particularly the sampled quality review — cannot even begin until a consent model is in place with real law firm clients. Phase 2 establishes the monitoring architecture; live operation establishes the baselines.
Regulatory classification confirmation. The Annex III analysis for Categories 5 and 8 involves interpretive judgment that needs review by a qualified EU AI Act lawyer. The reasoning in the risk classification document is defensible, but "defensible" and "confirmed" are not the same thing. A real company would need external legal sign-off on this before relying on the limited-risk classification operationally.
Vendor relationships. The Anthropic DPA, the sub-processor disclosure in client contracts, and the GPAI compliance documentation request are all unresolved because they require real commercial engagement with Anthropic. In a real deployment, these would be among the first actions taken — they gate several other compliance activities.
What This Means for Practitioners Adapting the Framework
If you are using this framework as a starting point for a real legal AI product, the Phase 2 worked example suggests four prioritisation decisions:
First, resolve the model provider relationship before anything else. The DPA, data residency confirmation, and sub-processor disclosure structure underpin a large proportion of the GDPR and EU AI Act compliance work. Nothing downstream is fully settled until this is in place.
Second, treat the human oversight gate as an architectural requirement, not a policy statement. The worked example shows how much load it carries across the regulatory framework. If your product roadmap ever contemplates reducing or removing mandatory review, re-run the entire risk classification — you may be moving into a high-risk AI system category.
Third, design your monitoring framework around the confidentiality constraint from the start. Do not assume you can monitor output quality the same way you would for a general-purpose LLM product. The legal market's confidentiality requirements reshape what data you can retain and how you can use it. Build the consent and confidentiality framework for sampled review early, because without it your quality monitoring has a significant blind spot.
Fourth, get legal sign-off on the Annex III classification before you publish any compliance claims. The reasoning in this framework is educational; it is not a legal opinion.
A Note on the Framework as Methodology
The worked example validated the framework's core purpose: that it is most valuable as an embedded governance process, not as a standalone compliance exercise. Each document generated findings that shaped subsequent documents. The risk classification informed the EU AI Act mapping. The mapping identified gaps that surfaced in the technical documentation. The technical documentation's logging constraints reshaped the monitoring design. The monitoring gaps fed back into open items that a real company would need to resolve before go-live.
This is what governance documentation should do — create a coherent, interdependent record that forces each compliance question to be answered in relation to the others. Template collections that live in a folder don't do this. A worked example that runs a real system through the framework end to end does.
That is the methodological claim this project is making, and the worked example supports it.
This reflection note is part of the Pickles GmbH AI Governance Framework — a fictional demonstration case. All observations are based on the process of building and applying the framework, not on operational experience with a real AI product.