AI for Defence: Insights on Strategy and National Security

Key Topics Covered

• Human judgment vs. AI autonomy — the non-delegable nature of command responsibility and lethal decision-making
• Black box problem and interpretability — the need for explainability, attribution, and observability in AI systems
• Governance and regulation — balancing rapid AI deployment with precautionary principles and international human law
• AI limitations in warfare — bounded rationality, context sensitivity, lack of abductive reasoning, training-inference gaps
• Defense supply chain modernization — demand forecasting, inventory management, route optimization using AI
• Building India's AI defense ecosystem — collaboration between military, academia, startups, and established industry
• Hardware sovereignty — dependency on GPU imports and indigenous semiconductor development
• International policy frameworks — UN conventions, confidence-building measures, and lethal autonomous weapon systems (LAWS)
• Real-world case studies — analysis of IDF's Lavender database and lessons from historical military incidents (My Lai massacre, Vietnam War)
• Responsible AI principles — accountability, testing rigor, bias mitigation, and bounded operational envelopes

---

Key Points & Insights

1. Command responsibility is absolute and non-delegable. A military commander cannot offload moral and legal accountability to an AI system, even if the system demonstrates 90% accuracy. The Lavender example (37,000 targets identified with 10% error margin = ~3,700 innocent deaths) illustrates the attribution crisis: who bears responsibility—the algorithm designer, the coder, the person who approved 10% error tolerance, or the commander?

2. AI can overcome human bounded rationality but introduces new risks. AI handles incomplete information, cognitive limitations, time pressure, and attention span constraints better than humans. However, it fails catastrophically when context shifts, lacks abductive reasoning for novel situations, and cannot perform real-time learning in battlefield conditions (training-inference gap).

3. Explainability and observability are non-negotiable. Panelists emphasized "glass box" AI over black boxes—requiring provenance (tracing outputs to sources), attribution (linking decisions to experts), and observability stacks (tracking what is delegated to GPUs vs. CPUs). Stanford's Snorkel and India's Spear platform demonstrate how programs can override ML outputs, keeping humans in control.

4. Military AI governance cannot lag behind technological advancement. Unlike civilian AI where governance can evolve gradually, defense AI demands responsibility insertion at every lifecycle stage: ideation, design, development, deployment, and destruction. Spiral development must include trust-building and bias detection continuously.

5. Testing rigor must match weapons systems standards. AI-enabled systems must undergo the same rigorous certification, red-teaming, and field evaluation as traditional weapons. Algorithms trained on clean satellite imagery fail on grainy, smoke-obscured battlefield images—a critical gap between lab and operational conditions.

6. Proportionality, distinction, and command responsibility remain with humans. These foundational principles of international humanitarian law (IHL) cannot be algorithmically determined. A commander paused a strike when learning of ongoing civilian evacuation—information the sensors hadn't captured—preventing collateral damage despite high AI confidence scores.

7. India's advantages include civilization values, military scale, and innovation ecosystem. India can lead responsible AI in defense by leveraging: (a) civilizational understanding that power must be governed by restraint (Shakti + dharma), (b) status as a major military power, and (c) vibrant startup and academic ecosystem.

8. Institutional collaboration requires embedded founders and military officers. Effective AI adoption needs bidirectional embedding: founders working within military units to understand operational constraints, and uniformed personnel within startup foundations to appreciate innovation challenges. This iterative, empathetic approach mirrors successful models like DARPA.

9. Hardware sovereignty is critical but achievable through multiple strategies. India faces GPU import dependency but can mitigate through: (a) diversifying supplier sources, (b) developing small language models runnable on CPUs, and (c) building indigenous semiconductor capabilities (long-term goal, acknowledging the 1986 SCL loss and TSMC's rise in 1987).

10. Supply chain resilience is mission-critical. Non-linear operations (jungles, oceans, long borders, multiple threat vectors) require AI-driven demand forecasting, predictive inventory management, and route optimization—not just copying global models but building indigenous, adaptive systems aligned with Atma Nirbhar Bharat.

---

Notable Quotes or Statements

Deputy Chief of Army Staff, General Shingle

"AI can inform decisions, but only humans can exercise judgment and bear responsibility for them."

"What does the machine not know?" — describing a commander's critical pause before a strike, revealing civilian evacuations the sensors had missed.

"With great power comes great responsibility... The character of war may change our conscience cannot."

"India today stands at the cusp of three powerful realities: We are a major military power. We are a rapidly growing AI country... and we are a civilization that has long understood that power must be governed by restraint."

General Rajiv Bell (DJICMR, referenced)

"It's basically you are presenting scenarios... alternatives. What AI can give you is alternatives—technology alternatives, procedure alternatives—and present with each alternative the provenance."

Prof. Ramakrishnan (IIT Bombay)

"No command center will trust AI as a black box... What AI can be used for is situation summarization, pattern detection at scale... and recommending course of action [alternatives]."

"Attributability at every level is very important."

General Anand (Defense Policy)

"When it comes to lethal autonomous weapon systems, drift detection is very important, but you also have to create certain bounded envelopes under which you use your weapon systems."

"When you use a probabilistic technology but you're getting very deterministic outcomes, there is a fundamental difference."

Vikram Jaram (Neurelix)

"ROI for defense is not profitability. ROI for defense is security... intelligence at a faster pace... effective decision support systems."

"The finest models that you see today... have gone through iterative phases... You cannot come up with something which is greatest on the first stage itself."

---

Speakers & Organizations Mentioned

Military & Government

• Deputy Chief of Army Staff, General Shingle — keynote speaker; emphasized human judgment, command responsibility, and ethical governance
• General Rajiv Bell — DJICMR (Director General Information Systems & Technology); referenced by Prof. Ramakrishnan for decision support framework
• General Anand — Defense policy expert; discussed governance, regulation, and bounded operational envelopes
• Indian Army — "Year of Data Centricity and Networking" declared; working with industry and academia on AI integration
• Ministry of Defence (UK) — supply chain digitization example
• IDF (Israeli Defense Force) — Lavender database case study (37,000 targets, 90% accuracy threshold, ~3,700 innocent casualties)

Academic Institutions

• IIT Bombay — Prof. Ramakrishnan; developing Spear platform; Bharaj Jen section for GPU-based model training
• Stanford University — Snorkel platform (open-source, human-designed programs for annotation)
• University of Oklahoma — mentioned as prior affiliation of startup founder
• IBM Research — developed annotated query language for AI with provenance tracking

Defense & Industry

• DARPA (Defense Advanced Research Projects Agency, USA) — cited as model for embedded founder-military collaboration
• Neurelix — Vikram Jaram (founder/panelist); working directly with Indian armed forces
• NATO — predictive analytics for inventory, operations, terrain, weather
• US Defense Logistics Organization — machine learning for spares failure prediction
• TSMC (Taiwan Semiconductor Manufacturing Company) — historical reference; 75% global market share, founded 1987
• Semiconductor Limited (SCL, India) — burned down 1986; marked loss of India's chip manufacturing capacity
• MD Anderson Cancer Center — referenced as parallel domain (life-and-death decisions, iterative clinical research model)
• IDEX Program — Indian startup accelerator enabling rapid innovation in defense applications

International Bodies

• UN Convention on Certain Conventional Weapons — ongoing discussions on LAWS governance
• Geneva Convention — historical example of effective IHL regulation (POWs, conduct in war)
• French President Emmanuel Macron — recent visit to Mumbai; 5 of 21 outcome statements on defense; 1 on critical/emerging technologies

---

Technical Concepts & Resources

AI Models & Frameworks

• Large Language Models (LLMs) — GPT, Gemini, Anthropic, ChatGPT; limitations: pre-trained (no real-time learning), probabilistic (not deterministic), context-sensitive, lack abductive reasoning
• Generative Pre-trained Transformers (GPT) — training cycle vs. inference cycle; produces next token probabilistically
• Small Language Models (SLMs) — runnable on CPUs (alternative to GPU-dependent LLMs)
• Mixture of Experts (MoE) — 17-billion-parameter model with 2 shared experts; faster inference at CPU level
• Snorkel (Stanford) — open-source platform for human-designed programs that can annotate data and override ML outputs
• Spear (India) — semi-supervised data programming platform; programs override ML model outputs; developed with support from DGIS (PhD student Janvi mentioned)

Technical Principles

• Bounded Rationality (Herbert Simon) — humans make satisficing (good-enough) not optimal decisions due to: incomplete information, cognitive limits, time constraints, attention span limits
• Provenance — tracing outputs to original sources (originated in database systems, SQL execution plans); critical for AI explainability
• Attribution — linking decisions to expert sources, not just data sources
• Observability Stack — tracking computation delegation across GPUs, CPUs, network switches; detecting data leakage; essential for secure, air-gapped systems
• Black Box Problem — even AI creators don't understand internal decision-making; alternatives: glass-box AI with explainability
• Context Sensitivity — AI fails when trained context shifts; requires domain knowledge
• Abductive Reasoning — reasoning with incomplete facts / novel situations; LLMs lack this capability
• Bias Mitigation — continuous detection during spiral development; not one-time activity
• Drift Detection — monitoring AI system performance degradation over time in operational conditions
• Bounded Operational Envelopes — clearly defined geographic, temporal, or contextual limits within which AI-enabled weapons operate

Datasets & Case Studies

• Lavender Database (IDF) — AI system identifying low-level Hamas operatives; 90% accuracy threshold; 37,000 targets identified; estimated 3,700 innocent casualties (10% error margin); illustrates attribution crisis
• My Lai Massacre (Vietnam War, 1968) — human-directed killing of 347–505 civilians; historical contrast with AI-enabled targeting; highlights dangers of "satisficing" decisions and psychological state in warfare

Governance & Standards

• India AI Governance Guidelines — released at summit; pragmatic approach with technical framework; identifies 6 risks including national security; addresses probabilistic, generative, adaptive nature of AI systems
• Rules of Engagement (ROE) — must keep pace with AI-enabled systems; unclear if current ROEs address AI-specific issues
• International Humanitarian Law (IHL) — proportionality, distinction, civilian protection; cannot be delegated to algorithms
• NBC Weapons Regulation — historical precedent for managing WMDs (nuclear, biological, chemical)
• Landmine Framework — international regulation example

Organizational Frameworks

• IDEX Program — Indian Defence Acceleration Ecosystem; enables rapid startup-defense integration
• System Integrators — intermediaries between startups, military, academia (often overlooked in discussions)
• Spiral Development — iterative build-test-feedback cycles; standard in successful AI teams (Gen AI models, defense applications)
• Clinical Research Model (Medical) — basic research → clinical research → FDA trials → deployment; applicable to defense AI lifecycle

Hardware & Infrastructure

• GPUs — primary compute substrate for current LLMs; India faces import dependency
• CPUs — alternative substrate for smaller models; more widely available
• Semiconductor Manufacturing — indigenous capability goal; India lost opportunity in 1986 (SCL), TSMC rose in 1987 (now 75% market share); multiple Indian agencies working on this

---

Additional Context

Policy Initiatives Mentioned

• Atma Nirbhar Bharat — self-reliant India initiative; supply chain resilience framed as operational advantage
• France-India Bilateral Outcomes — 21 outcome statements; 5 on defense, 1 on critical/emerging technologies in defense
• Year of Data Centricity and Networking (Indian Army, 2024) — institutional focus on data as strategic asset and datacentric warfare doctrine

Historical Comparisons

• Nuclear Technology Governance — few actors, slow pace, successfully regulated through international frameworks (NBC treaties)
• AI Governance Challenge — many actors, rapid pace, decentralized development; requires different regulatory approach but similar ethical commitments

---

Missing or Underdeveloped Topics

• Specific examples of AI success in Indian military operations (discussions remained largely policy/principle-level)
• Cyber vulnerabilities of AI-enabled systems in contested electromagnetic environments
• AI-enabled disinformation and information warfare (mentioned but not deeply explored)
• Legal liability frameworks (if commander follows AI recommendation that proves wrong, who is liable under military/civilian law?)
• Adversary AI capabilities and specific threat assessments (noted as "regularly surprising" but not detailed)