Feature Slider

Autonomy for sale

A robot dog on a plinth at an exhibition stand, with two people standing behind it
An autonomous robot dog on show at Land Forces 2026, a system that keeps a human on the loop. Photo: Nima Monazami.

As Welsh tradition had it, a loaf of bread could soak up a dead man’s sins. Provided somebody would eat the bread and its sins. Richard Munslow, according to the BBC, the last known sin-eater, died in 1906.

Killing large numbers of civilians is considered a transgression by most moral codes and by International Humanitarian Law, but when the triggers are pulled by machines, who bears the sin?

At Land Forces 2026, a Defence Expo held in Perth in October, Perth firm Innovaero and US firm Shield AI signed a memorandum of understanding to put Shield AI’s Hivemind autonomy software into Innovaero’s loitering munitions, its drones that can hover before they strike.

Shield AI’s media release on the MOU says Innovaero’s OWL munitions are designed to identify, track and strike targets, and that Hivemind will increase their lethality. A month earlier, UN talks on autonomous weapons ended without a treaty. This week, at the Convention on Certain Conventional Weapons Review Conference in Geneva, governments are deciding whether to start negotiating one, after Russia stalled the opening session on November 16.

Screen showing red detection boxes labelled “person” over video frames of the exhibition floor
A display at Land Forces 2026 runs person-detection software over footage of the exhibition floor, boxing and numbering each person it spots. Photo: Stuart Stirling.

The law says people are responsible for how a weapon is used, according to the UN report. Which person is harder to say.

But the autonomy is already on show. International Humanitarian Law, known as the laws of war, is a set of international rules that aims to protect civilians from the effects of armed conflict. It rests on three principles: distinction, proportionality and precaution.

Visitors passing through the security gates at the Land Forces 2026 entrance
Visitors walk through security gates at the entrance to Land Forces 2026 in Perth. Photo: Nima Monazami

In 2005, the International Committee of the Red Cross published 161 customary rules that form a core baseline of protection. Proportionality bars an attack if the civilian harm it’s expected to cause would be excessive compared with the concrete and direct military advantage anticipated.

Dr Eve Massingham is a legal adviser with the Australian Red Cross, specialising in IHL.

She says although the law was written long before the rise of AI and automation, its guiding principles allow it to apply to emerging technology.

“You must use a weapon that can distinguish between a combatant and a civilian. It doesn’t matter whether that weapon is an AK-47 or…the most high-tech autonomous loitering system. It has to be able to distinguish between military and civilian or you can’t deploy it,” she says.

Dr Eve Massingham, legal adviser, Australian Red Cross.

Dr Eve Massingham, legal adviser, Australian Red Cross. Photo: ICRC.

In September, a UN group of government experts presented its report under the Convention on Certain Conventional Weapons, which bans or restricts certain weapons. Its formal name is a mouthful: The Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems.

The International Committee of the Red Cross headquarters in Geneva.
The International Committee of the Red Cross headquarters in Geneva. Photo: Torbjørn Jørgensen © 2019.

The group’s second 2026 session ran from August 31 to September 4 in Geneva. “They weren’t there to complete a treaty,” Dr Massingham says.

After three years, the group’s mandate was to agree on a set of elements a future treaty on autonomous weapons could contain and hand them to the convention’s Seventh Review Conference. It did this by consensus, as set out by the mandate. “That set of elements is by no means perfect, but it exists,” says Dr Massingham.

Agreeing on the elements is not agreeing to negotiate. The negotiations are now backed by 76 nation states, according to advocacy group Human Rights Watch, and 70 of them are High Contracting Parties, meaning they are voting members of the CCW.

From November 16, diplomats from the 128 states party to the Convention are meeting in Geneva until November 20, asked to endorse the elements. Before it opened, Dr Massingham said the conference was likely to create a mandate to negotiate a treaty. Russia blocked the agenda on the first day, and talks moved to informal consultations.

Seating chart of the 128 states parties to the Convention on Certain Conventional Weapons, with 70 flagged as backing negotiations
The CCW’s 128 states parties. A flag marks each of the 70 parties backing negotiations, according to Human Rights Watch; Australia is circled. Chart: Nima Monazami.

The elements in the UN report turn on how much control a human keeps over a weapon, before and during its use. In HRW’s 2012 report Losing Humanity, automated weapons are split into three main distinctions.

In the loop, the weapon selects targets and delivers force only with a human command. On the loop, it does so under a human operator who can override it. Out-of-the-loop weapons can identify, track and strike targets without human input.

Under the UN’s report, a system counts as a lethal autonomous weapon if it can identify, select and engage a target. Resulting in loss of life without a human operator intervening. Whether Innovaero’s technology meets the test depends on who selects targets. A human loading targets or their characteristics doesn’t take a system out of the definition. A person choosing the specific target and timing does. Control can also be exercised indirectly, the report says, through measures taken before the weapon is used.

Rows of national flags in front of a building marked “United Nations”
National flags outside a United Nations building. Photo: Xabi Oregi.

Dr Massingham says the Red Cross movement is calling for two prohibitions, it calls for a ban on unpredictable systems and on using autonomous weapons against humans. The ICRC position adds limits on targets, duration, geography and scale. It also calls for a human supervisor in case the system needs to be deactivated.

She says autonomy in weapons isn’t new; a landmine is an old-fashioned version of autonomous technology. It goes off when a sensor is triggered. A mine can’t tell who steps on it, which fails the test set out earlier. The report bans weapons that are inherently indiscriminate.

The landmine, banned by 162 countries under the Ottawa Treaty, remains static, while loitering munitions are by definition airborne, equipped with sensors to detect targets before explosive impact.

Before Hivemind, the public record on the OWL-B said little about the human. Short for One-Way Loitering, the OWL-B is an electric loitering munition built with the Australian Defence Force for medium-range strike. Defence’s Advanced Strategic Capabilities Accelerator has an 18-month, $20.8 million contract with Innovaero for the flight vehicle, which it will refine before operational use. ASCA announced a drone development project called Mission Talon Strike in May 2025, focusing on sovereign solutions for low-cost, medium-range, land-launched loitering munitions.

Shield AI’s announcement says Hivemind has already been integrated onto the OWL-B ahead of flight testing, and that strike missions are being simulated. Hivemind will enable faster mission planning and increased lethality in OWL’s attacks. It says the software reroutes mission plans, responds to unexpected conditions, and supports autonomy when communications are degraded. Innovaero’s website says the OWL-B has demonstrated the ability to execute a predefined mission fully autonomously, including changing its task in flight.

A rendered image from Mission Talon Strike.
A rendered image from the Mission Talon Strike program. Image: Advanced Strategic Capabilities Accelerator (ASCA).

Dr Emily Camins is a lecturer in International Humanitarian Law at the University of Western Australia. She says that failing to comply with IHL principles in making targeting decisions can lead to individuals or states being held responsible.

Dr Emily Camins, lecturer in international humanitarian law, UWA.
Dr Emily Camins, lecturer in international humanitarian law, UWA. Photo: UWA.

“The principles of International Humanitarian Law assume human decision-makers. When lethal autonomous weapon systems play a significant role in targeting decisions, this can obscure questions of accountability.

“Autonomous weapons systems rely on opaque algorithms and data to make targeting decisions, which can make it difficult to establish intention, knowledge or control,” she says.

Money moves first. In a June paper in the journal Defence Studies, University of Exeter academics argued that a large share of UK military AI now comes from industry. Privately funded until government buys it.

The opening paragraph of the Defence Studies paper, with estimates of military AI spending highlighted
The introduction to the Defence Studies paper estimates worldwide military AI spending rose from about US$4.6 billion in 2022 to over US$10 billion in 2025. Image: an excerpt from Defence Studies.

According to the authors, Aurel Sari and Geraint Evans, legal check may be triggered when a company is funded through a defence accelerator, under Article 36 of Aditional Protocol 1 to the Geneva Conventions (1977 version). But it may come when designs are already mature.

They say: “Assessing compliance at a late stage in the AI development cycle therefore runs the risk of creating a binary choice between either accepting costs and capability shortfalls or playing fast and loose with the rules.”

Description of article 36 on ICRC’s law database.
Description of Article 36 on ICRC’s law database. Image: ICRC.

They also point to an imbalance in the UK’s own guidance. The Ministry of Defence’s directive on responsible AI, JSP 936, devotes 47 paragraphs to ethical principles and three to legal considerations.

Sari and Evans test the problem with a hypothetical from a UK defence ethics training toolkit.

A commander launches a strike on an AI system’s recommendation. It rests on flawed data about civilian movements and civilians are harmed. The toolkit asks who is to blame, the commander or the system’s developers. Sari and Evans say the commander who relied on the recommendation without reassurance that it met the law almost certainly broke it.

The hypothetical commander could comply in two ways.

The commander can rely on a system able to verify targets itself, which developers would struggle to build because the law’s requisite certainty depends on the circumstances. Or the commander can confirm everything by hand, which brings the strike back to human speed, calling into question why the system was used at all.

The law doesn’t hold the company that built the system to the same standard.

The paper finds international law does little to enforce a company to build only systems that can be used lawfully. The UK’s own directive tells industry to get its own legal advice. The UN report says accountability cannot be transferred to machines but doesn’t mention manufacturers or developers.

In March, the Australian Government published its Defence Policy Settings for Responsible use of Artificial Intelligence. The stated purpose of the paper is to preserve individual accountability for AI-enabled decisions and outcomes and to comply with domestic and international law obligations.

But while Article 36 of AP1 mandates early legal review for military personnel acquiring new systems, it sets no obligations or compliance expectations for the private industry.

On the floor at Land Forces 2026, these policy goals hit commercial reality. The event brought senior defence figures, public servants and arms manufacturers into one room to discuss what the army buys next.

A visitor examines a robotic weapon mount at a stand on the Land Forces floor.
A visitor examines a robotic weapon mount at a stand on the Land Forces floor. Photo: Nima Monazami.

At the Lightforce Australia stand, exhibitor Benjamin Crump stands beside a weapon-mount system explaining how it aims at drones. Asked about AI, Crump says much of it is designed in-house and too expensive for rapid hardware integration.

“We’d rather try to use open-source things where we can, then develop our own models, so we can tailor them to our specific needs,” he says.

To hit a drone, the system must aim ahead of it.

“By the time the projectile gets there, there are another four levels of calculation and you’d have to do them in real time,” Crump explains. Spotting one is a separate job.

He says Lightforce the system connects to thermal cameras from Sydney company Vision Machines, which detect movement up to five kilometres away. “But they do blob detection, not drone detection,” leaving the operator to check and distinguish.

Crump explains the weapon’s sensor capabilities.
Crump explains the weapon’s sensor capabilities. Photo: Nima Monazami.

Shooting at drones sits at the safer end of autonomy. Dr Massingham says the Red Cross wants autonomous weapons kept to objects rather than people.

“The likelihood of misidentification when you’re talking about humans is greater than when you’re talking about facilities,” she says.

In 2003, three aircrew died after missile batteries classed their planes as enemy missiles. During the invasion of Iraq, US Patriot batteries brought down a British Tornado and a US Navy F/A-18. A Pentagon review found the operating protocol “largely automatic”, with personnel trained to trust the software.

C4ISREW is the army’s term for the network that carries information from sensors to commanders to weapons. The C4 stands for command, control, communications and computers. The rest covers intelligence, surveillance, reconnaissance and electronic warfare.

A fixed-wing drone on a launch rail at Land Forces.
A fixed-wing drone on a launch rail at Land Forces. Photo: Nima Monazami.

Colonel Phillip Ellsmore, Director of Land C4ISREW at Army Headquarters, spoke at a Land Forces session. The products on show have a half-life of barely six weeks at the moment, he says. Drones are countered by electronic warfare, and the counters get countered in turn.

In Geneva, states are asked only to endorse the elements without settling what form an instrument would take. The conference then decides by consensus what the group does next, so in practice a single objection can stall a mandate.

Australia, speaking for 21 states in May 2025, backed the group’s work and warned that a process outside it would set progress back. At home, the money is already committed. Defence’s 2026 investment plan sets aside $12 to $15 billion for autonomous systems over the decade.

The UN report does address development. IHL applies fully to the development of these weapons, it says, and risks to civilians must be weighed during design, development, testing and deployment. It doesn’t say by whom.

Hivemind now goes to flight testing on a munition built for the Australian Defence Force. Defence’s March policy says accountability for an AI-enabled decision stays with an identifiable Defence official.

Sari and Evans found that a commander who takes every feasible precaution has complied with the law even when civilians are harmed.

Munslow’s weathered gravestone beside a stone cross in St Margaret’s churchyard, Ratlinghope

So it seems that when it comes to automation in weapons, no one has volunteered to eat the bread.

The grave of Richard Munslow, the last known sin-eater, in St Margaret’s churchyard, Ratlinghope, Shropshire. Image: Wikimedia Commons.