There is a reassuring phrase that appears in discussions of military AI: human in the loop.
It sounds like a safety property. Sometimes it is. Sometimes it means a person is present at the final stage of a machine-speed process, staring at a recommendation assembled by systems they cannot independently reconstruct before the decision window closes.
A human can be in the loop and still be functionally decorative.
Map the entire decision chain.
A simplified autonomous targeting system can be represented as:
sensor → detection → classification → tracking → threat assessment → target prioritization → weapon assignment → authorization → engagement → battle-damage assessment
Real systems may distribute those stages across aircraft, drones, radar networks, command systems, operators, and software services. Autonomy can exist in one stage while others remain manual. That is why arguments framed as “autonomous weapon or not?” often obscure more than they reveal.
The meaningful unit of analysis is the decision function.
Humans can control parameters without controlling moments.
One model of autonomy places human control upstream. Commanders define a target class, geographic boundary, time window, rules of engagement, and abort conditions. The system then searches and acts within those constraints.
This can provide real control if the environment is predictable and the target class is narrowly defined. It becomes more dangerous as the environment grows ambiguous, civilians mix with combatants, sensor confidence shifts, adversaries deceive the classifier, or mission conditions change after launch.
A choice made hours earlier can still be human. It is not necessarily responsive.
Humans can approve decisions they did not meaningfully make.
Now consider the opposite arrangement: software detects a target, assigns a confidence score, fuses intelligence, ranks threats, and recommends a strike. A human operator receives the recommendation and must approve or reject within seconds.
On paper, the human makes the final decision.
In practice, several pressures can turn that authority into automation bias:
• the machine sees more sensor feeds than the operator can inspect;
• the recommendation arrives with a high confidence score that appears quantitative and authoritative;
• the operator faces time pressure;
• rejecting the recommendation may require more justification than accepting it;
• repeated correct recommendations train the operator to trust the system;
• the interface may expose conclusions without exposing enough underlying evidence.
That is how a human can retain legal responsibility while losing epistemic control.
Meaningful control needs four things.
Cyberdelia's working test is brutally simple.
1. Information
The person must receive enough evidence to understand why the system is proposing force. A confidence number alone is not explanation.
2. Time
The person must have enough time to inspect, challenge, or seek additional information. If machine-speed combat reduces that window to reflex, the human becomes a biological confirmation button.
3. Authority
The person must be able to reject the recommendation without the system routing around them or institutional incentives making rejection effectively impossible.
4. Intervention
The person must possess a reliable technical mechanism to stop, redirect, or abort the action when circumstances change.
If one of those is absent, “human in the loop” becomes an incomplete safety claim.
Automation can also reduce harm.
This debate is often flattened into machines bad, humans good. Humans misidentify targets, panic, become exhausted, miss information, and commit atrocities. Automated systems can potentially improve sensor fusion, enforce geographic limits, track no-strike lists, detect friendly forces, refuse targets below confidence thresholds, or respond faster to incoming weapons.
The question is therefore not whether automation is morally contaminated by definition. The question is which functions are appropriate to automate, under what constraints, with what testing, and with what human responsibility.
That is also why systems designed to intercept incoming missiles in tightly bounded scenarios are different from systems asked to identify individual humans in complex civilian terrain. Same broad word, radically different ambiguity.
Accountability must follow the architecture.
The ICRC has repeatedly raised the problem of responsibility when autonomous systems cause unlawful harm. Traditional accountability often looks for the person who fired, the commander who ordered, or the person who planned the operation.
Autonomous systems distribute causal contribution. A classifier may be defective. Training data may exclude relevant conditions. Mission parameters may be too broad. A commander may deploy the system outside its validated environment. An operator may ignore warnings. A manufacturer may conceal known limitations.
None of that makes accountability impossible. It means investigators need logs.
A serious autonomous weapon should preserve a decision record: sensor inputs, model versions, confidence outputs, target-state history, mission rules, human actions, software changes, timestamps, and abort opportunities. Without that provenance, post-strike accountability becomes archaeology conducted by lawyers.
The arms-race problem is latency.
There is another pressure toward autonomy that moral debate cannot wish away. If one force compresses the sensor-to-engagement cycle, opponents face pressure to do the same. A slower human review process may become a tactical disadvantage.
This creates a dangerous feedback loop: automation reduces decision time, which makes human review harder, which incentivizes more automation.
Regulation therefore has to confront operational incentives rather than merely announce that humans should remain responsible. If the system architecture rewards whoever removes deliberation first, ethical language will lose to latency.
“Human in the loop” is not a sufficient standard. Meaningful human control requires information, time, authority, and a reliable intervention path. Regulation should focus on critical functions and operational context rather than treating autonomy as a binary label. The decisive question is not whether a person appears somewhere in the chain; it is whether that person can still alter the chain before force becomes irreversible.
The metric we want
Future reporting should stop asking military programs merely whether humans approve engagements. Ask instead: How much decision time do they receive? What evidence is visible? What percentage of machine recommendations are rejected? Can operators inspect uncertainty? What functions continue after communications loss? What records survive for investigation?
Those answers would tell us more about meaningful control than another thousand-word argument over the word “autonomous.”