For two years the AI Act was mostly a conference topic: distant deadlines, generic obligations, a regulation everyone knew was coming but that touched few people’s day-to-day work writing software. That status changes this month: obligations for high-risk systems become fully applicable, and “high-risk” in the text covers more than most expect — not just medical diagnostics or autonomous vehicles, but hiring screening, credit scoring, and management of critical infrastructure. If your product touches one of these areas, compliance stops being a theoretical exercise.
What “high-risk” actually means in practice
The classification isn’t about the model itself, but about how it’s used. The same language model can be minimal-risk when it drafts internal emails, and high-risk when it decides who gets screened out of a hiring process. It’s the application that determines the obligations, not the underlying technology — a distinction that often gets lost when people treat compliance as a property of the model they bought or trained.
For systems that fall into the category, the main obligations are concrete and checkable:
- Traceability of decisions. You need to be able to reconstruct why the system produced a given output for a given input, not just demonstrate average accuracy on a test dataset.
- Effective human oversight. Not an override button nobody ever presses, but a process where a person genuinely has the ability and the time to intervene before the decision takes effect.
- Technical documentation kept current. Not a PDF written once at launch, but an artifact that reflects how the system actually behaves today, version after version.
- Risk management as an ongoing process. Assessing risk before release isn’t enough — it has to be repeated whenever the model, the data, or the context of use changes.
Why this is an architecture problem, not just a legal one
The natural temptation is to treat compliance as a task handed to legal downstream, once the system is already built. That doesn’t work: decision traceability has to be designed into the logging system from the start, because reconstructing it after the fact on a system already in production is often impossible without the right data already collected. The same goes for human oversight — if the interface isn’t built so a person can genuinely evaluate a decision in time to matter, the requirement stays on paper.
The practical upshot is that regulatory requirements are becoming design inputs on par with functional ones: what to log, which metrics to expose, where to place a human checkpoint are no longer decisions that can be pushed to the end of the project.
The trade-offs to account for
- Higher development cost, not just compliance cost. Structured logging, model versioning, oversight interfaces: this is real engineering work, not boxes to check on a questionnaire.
- Ambiguity around classification. Many products sit at the boundary between risk categories, and the line isn’t always clean: caution often pushes teams to treat even borderline cases as high-risk, with the cost that follows.
- Third-party vendors as a blind spot. A system that integrates an external model or service inherits part of the obligations, but has far less control over that component’s documentation and traceability.
- The risk of decorative compliance. It’s possible to produce documentation that’s formally correct but doesn’t reflect the system’s actual behavior — it satisfies an audit without reducing the actual risk.
Tip: before chasing compliance as a checkbox, take inventory of the systems that make or influence decisions about people — hiring, credit, access to services. The list is usually shorter than feared, and that’s where the real obligations start, not everything that happens to generate text.
What to expect from here
The first concrete enforcement actions will likely come from edge cases — a contested automated screening system, a disputed credit decision — rather than systematic checks on every product. But the direction is set: technical documentation and decision traceability are becoming part of the work of building these systems, not an appendix produced by someone else at the end. Teams that already designed for transparency will find themselves ahead; those who postponed it will find out how much it costs to rebuild afterward.