How do you write requirements for an AI that adapts?

Policy frameworks such as the EU’s Trustworthy AI guidelines give high-level principles but little procedure for turning them into traceable, runtime-aware requirements. Traditional requirements engineering is specification-centric and resolves contradictions before implementation; it has no model of a requirement being partially satisfied, traded off, or monitored after launch.

DARE’s claim is not that these problems are new. It is that they should be handled by one connected method — a shared vocabulary, a way of writing flexible requirements, a conflict priority order and a runtime monitoring loop that all refer to the same requirement objects.

How the rigid example is enforced

• Enforced by: matching queries against the assignment database, a confidence threshold, a solution-detail analyser and a mandatory refusal protocol.
• If violated: academic dishonesty enabled, student skills not developed, instructor trust lost.
• Monitored: every query scored, confidence logged, refusal decisions audited.

The full rigid set in the case study

• Never give graded-assignment solutions (integrity).
• Never present uncertain information (confidence below 0.8) as definite fact (accuracy).
• Escalate immediately on signs of self-harm, severe distress, or five failed explanation attempts in a row (safety).
• Keep student data confidential under FERPA (privacy).
• Never give different quality of help based on demographics (equity).

The classification is deliberately made early and argued out, because a rigid requirement has no “relaxed” state to fall back to at runtime.

Four questions asked of every requirement

• Must this hold unconditionally? If yes, it stays rigid — stop here.
• What makes it uncertain? Name the environmental factor (e.g. system load), the signal you can actually observe (CPU use, query-complexity score), and how the two relate. ENV · MON · REL
• Is there a realistic situation where the target can’t be met? If so, relaxation is warranted.
• Choose the flexibility word that fits, and set the limits.

The second question does real work: it forces the team to check whether monitoring can actually observe what the adaptation depends on. A student’s true understanding isn’t observable; the sophistication of their follow-up questions is.

Knock-on effects DEP

Each flexible requirement records which others its relaxation affects. Slowing responses to save capacity must not frustrate students into triggering encouragement messages that cost even more capacity. And no amount of explanation depth may cross into giving away a homework solution.

The four runtime states

• Satisfied — at or better than the ideal target.
• Relaxed within limits — deviating, but inside a range stakeholders approved.
• At risk — trending toward the hard limit; the system acts before it gets there.
• Violated — past the hard limit. For a flexible requirement this means handing over to a named person; for a rigid one it is an incident.

Rigid requirements only ever have the first and last state. In the case study, when response time conflicts with explanation depth, the priority order (section 7) says quality beats speed — so time is relaxed into the 10–30 second band rather than giving a shallow answer.

How the core concepts connect

A risk experience also tests specific requirements, and deliberation weighs the intentions of the agents involved. Because each link is explicit, you can trace any decision back to the situation and requirements that caused it.

Questions the vocabulary must be able to answer

• What situations trigger deliberation in the system?
• Which requirements does a given situation put to the test?
• What conflicts arise from it, and how was each resolved?
• What is the full path from situation to decision?

Risk broken into parts

Each risk is split into what could go wrong, what would be lost, how uncertain it is, what is at risk, and what makes it more likely. The last part is usually the actionable one. The thesis applies this retrospectively to the 2018 Uber self-driving fatality — where disabled emergency braking and lenient regulation appear as enablers — and prospectively to the tutor, where deadline pressure and satisfaction-driven metrics enable integrity breaches.

Also set up in Phase 0

• The operating domain, boundaries and what the system explicitly won’t do.
• Applicable ethical frameworks and regulation for the domain.
• Autonomy levels and the conditions for moving between them.

Conflicts are not eliminated at design time; they are expected to recur in operation. Phase 4 lists them with how often they occur and how hard they are to resolve, and records the reasoning behind each resolution so decisions can be audited later.

In the case study the priority order is grounded in four ethical principles adapted from biomedical ethics — do good, avoid harm, respect the student’s autonomy, treat everyone fairly — which often pull in different directions.

• Tier 1: overrides all performance, personalisation and satisfaction considerations.
• Tier 2: show uncertainty explicitly rather than trade accuracy for speed or simplicity.
• Tier 3: explanations that genuinely advance understanding beat fast shallow ones.
• Tier 4: balance challenge with support; accept reasonable delays for quality.
• Tier 5: optimise within the limits of everything above; relax under pressure.

This differs from traditional validation, where stakeholders approve a fixed behavioural specification. Here they approve a bounded decision space, accepting that the system will sometimes operate below the ideal when conditions demand — and trusting the agreed mechanisms to handle situations nobody discussed explicitly.

Acceptance requires explicit confirmation that the limits allow meaningful adaptation without hurting quality, that rigid rules hold under all conditions, that escalation triggers are understandable and workable, and that the system’s explanations of its own adjustments are clear to users. The signed baseline becomes the reference for runtime auditing and for any later change to requirements.

Each flexible requirement is turned into an executable monitoring rule with measurable signals, the three limit bands, adaptation options with safety conditions, and an escalation condition naming the responsible person. Monitoring is proactive: the aim is to spot a trend toward a violation and act before it happens.

Every escalation carries a context package — which requirements are involved, their current limits, recent adaptations, confidence levels and suggested actions — so the person stepping in can act quickly.

Signals that requirements themselves need revisiting: a requirement repeatedly relaxed to its maximum, one persistently unsatisfied despite adaptation, or behaviour revealing needs nobody anticipated.

Because autonomy is a setting rather than a fixed property, the runtime loop can lower it: in the case study, repeated breaches of the practice-difficulty limit first reduce the tutor’s autonomy to increase supervision, then escalate to the instructor. Every transition is logged.

Level 5 is excluded because even capable systems degrade on unfamiliar problems, and removing oversight in consequential domains removes accountability.