India AI Impact Summit Insights
All sessions

Ensuring Safe AI: Monitoring Agents to Bridge the Global Assurance Gap

Contents

Executive Summary

This panel discussion from the India AI Impact Summit addresses the urgent challenge of creating robust AI assurance frameworks for autonomous AI agents, with a critical focus on closing the global divide between well-resourced regions (Global North) and under-resourced ones (Global South). The speakers emphasize that AI assurance must evolve from theoretical frameworks to operational disciplines embedded in system development, require multilingual and culturally-contextualized evaluations, and demand collaborative, shared responsibility across governments, companies, academia, and civil society.

Key Takeaways

  1. Assurance Must Be Built Into System Development, Not Bolted On

    • Design systems for observability, auditability, and constraint from day one. This is both a technical and governance challenge, not just a compliance afterthought.

  2. Global South Countries Must Not Be Afterthoughts—They Need Co-Design Capacity

    • Success requires building regional evaluation capacity, professionalized auditor networks, and decision-making power for Global South stakeholders. Standards development (e.g., agent attribution, identity) should include them now, not in retrospective implementation.

  3. Post-Deployment Monitoring & Real-Time Detection Are Non-Negotiable for Agents

    • Unlike static models, agents demand continuous monitoring, failure detection frameworks, and clear accountability structures for interventions. This is a harder technical problem than pre-deployment testing alone.

  4. Assurance Is Shared Infrastructure, Not a Competitive Advantage to Hoard

    • No single company, government, or institution can deliver safe, trusted agentic AI. Shared evaluation infrastructure, taxonomies, benchmarks (especially in multiple languages), and investment in Global South capacity are prerequisites for success.

  5. The Next 12 Months Are Critical for Institutionalization

    • Concrete outcomes should include: changed incentives (e.g., insurance support for assurance), professionalization pathways (accreditation), operational embedding of assurance in development lifecycles, and meaningful North-South collaboration on emerging agent standards.

Key Topics Covered

  • AI Agents & Autonomous Systems: Definition, capabilities, real-world deployment challenges, and emerging safety risks
  • Global AI Assurance Ecosystem: Components (testing, standards, third-party auditors), maturity gaps, and infrastructure barriers
  • Multilingual & Contextual Evaluations: Language diversity, cultural context, and localized risk profiles across regions
  • Testing & Standards: Technical protocols (agents-to-agents communication), interoperability standards, and safety benchmarks
  • Global South Inclusion: Data extraction ethics, capacity-building, sovereignty concerns, and avoiding technological colonialism
  • Post-Deployment Monitoring: Real-time failure detection, continuous monitoring, and reversibility frameworks for agents
  • Third-Party Assurance Providers: Independence verification, auditor capacity, professionalization, and accreditation
  • Policy & Governance: Proactive vs. reactive regulation, sandbox approaches, and multilateral coordination
  • Data Supply Chains: Labor practices, consent, compensation, and dignity preservation in data annotation

Key Points & Insights

  1. AI Agents Represent a Paradigm Shift in Assurance Complexity

    • Agents are increasingly autonomous systems capable of planning, chaining actions, and adapting over time—moving beyond static model outputs to dynamic, goal-oriented behavior. This fundamentally changes assurance requirements from pre-deployment testing alone to continuous post-deployment monitoring.

  2. The Global Assurance Divide Is Severe and Widening

    • Assurance infrastructure (compute, evaluation data, third-party auditors, skilled personnel) is concentrated in the Global North. Without deliberate intervention, the shift toward agents will exacerbate this inequality, creating a two-tier AI economy.

  3. One-Size-Fits-All Testing Won't Work Across Linguistic & Cultural Contexts

    • India has 120+ languages and 19,500 dialects; Africa has ~3,000 spoken languages. System safety and performance vary dramatically across languages, cultures, and deployment contexts. Evaluation frameworks must account for local risk profiles, which differ significantly (e.g., environmental impacts matter more to Pacific island nations; healthcare/government services prioritize differently by region).

  4. Data Supply Chains in the Global South Remain Extractive

    • Much language data and annotation work happens in Global South countries with minimal local visibility, agency, or compensation. Workers and data contributors lack transparency about how their data is used, and current policy/legislation often lags behind ethical requirements.

  5. Testing & Standards Must Be Tiered by Risk & Use Case

    • Different deployment contexts demand different assurance rigor: financial services, healthcare, and government systems require higher stakes assurance than consumer applications. Assurance must also account for agent autonomy levels and reversibility of actions.

  6. Proactive Preparation Beats Reactive Regulation

    • Singapore's approach: governments must lead by testing agentic AI internally (via sandboxes, e.g., with Google) before regulating others. This builds credibility and practical knowledge.

  7. Third-Party Assurance Providers Are Essential but Underdeveloped

    • Independent verification (technical testers, auditors) provides crucial oversight and identifies blind spots that internal assessments miss. The pool of qualified assurance providers is thin globally and nearly nonexistent in the Global South.

  8. Interoperability Must Not Become a Form of Exclusion

    • Calls for "test once, comply globally" risk imposing Northern/Western assurance standards on regions with different risk profiles and needs. True interoperability requires shared guidelines and tools that remain adaptable to local contexts.

  9. Assurance Infrastructure Is the Foundational Bottleneck

    • Computational requirements for evaluation are immense (e.g., Stanford HELM used 12 billion tokens and 19,500 GPU hours). This creates barriers for Global South participation, requiring innovative approaches (cheaper models, distributed evaluation, shared infrastructure).

  10. Professionalization & Incentivization Are Missing

    • There's no clear accreditation pathway for assurance providers or enforceable standards for what "trustworthy assurance" means. Insurance and other market mechanisms could drive maturity, but institutional structures are underdeveloped.

Notable Quotes or Statements

Minister Josephine Teo (Singapore)

"There needs to be a shift...from reliance on reactive regulation to a different kind of stance which is proactive preparation."

"A company that is able to give high assurance on safety will find itself being differentiated from competitors...think of it as a strategic competitive advantage."

"Government is high-risk because the touchpoint with citizens is very sensitive. No citizen and no government wants to make serious mistakes when they interact with citizens."

"Three components [of assurance ecosystem]: testing, standards, and third-party assurance providers."

Vukosi Marivate (Masakani Research Foundation, University of Pretoria)

"It's not going to be a one-size-fits-all...this is one of the biggest challenges. When you think about localized contexts, [agents] just fail. How does that failing look? Is it jarring experience? Is it hurting people's dignity?"

"We are still having the challenge of being very extractive...people are the source of the data...they deserve to understand what they're contributing to, have their rights, be compensated, and credited for that work."

"When it comes down to the user...you're trying to think about personalization...you don't know what will happen at the individual experience...you can't model all of that."

Fred Tonucci (AI for Good / ITU)

"It's safe to say there's no shortage of high-potential AI for good use cases...but how do you turn those ambitious words and principles into actions? The devil is in the details. Standards have details."

"[Comparing to mobile payment revolution in East Africa] There may be optimism that the same could happen with AI in the global south, but I don't think we can take it for granted...It's not a guarantee. So that whole AI skilling angle is critical."

Owen Vallis (Google DeepMind)

"Agent security is something we should all be thinking about...if you're connecting increasingly autonomous systems into different accounts, different email accounts, different bank accounts, we want to be pretty careful about how we do that."

"[On access:] We have a range of models...our very quick flash models which are relatively cheap, quite efficient, very quick. These can play a really important role in powering agentic systems."

Stephanie Lutz (Partnership on AI)

"Access is something that impacts us all...the UK Department of Science Innovation and Technology has made access to models as a means to support assurance a priority for 2026."

"[On success in assurance:] How do you apply assurance based on the risks or stakes at hand? Is it financial services? Is it healthcare? So tie it as close to the use case and risks...linked to reversibility of actions and consequences."

Natasha Crampton (Microsoft, Chief Responsible AI Officer)

"Assurance really needs to move towards continuous monitoring, real-time detection, and clear accountabilities for when interventions need to take place."

"If we don't address that gap [between North and South] deliberately, the shift towards AI agents is only going to make that divide even worse rather than closing it."

"We need to treat assurance as infrastructure...that we need to build together and put into practice together."

Speakers & Organizations Mentioned

Government & Policy

  • Minister Josephine Teo – Singapore Government (led response on agentic AI governance)
  • Prime Minister Modi – India (referenced vision for building in India, delivering to world)
  • DEST (Department of Science Innovation and Technology) – UK (access to models as assurance priority)
  • NIST / CASEY (Center for AI Standards and Innovation) – US (announced agentic standards initiative)

International Organizations & NGOs

  • Partnership on AI (PAI) – Rebecca (moderator), Stephanie Lutz; released papers on global assurance divide and ecosystem strengthening
  • ITU (International Telecommunication Union) – Fred Tonucci, AI for Good initiative
  • UN Agencies – 50+ sister agencies in AI for Good network

Industry & Research

  • Google DeepMind – Owen Vallis; mentioned partnerships with UK AI Security Institute, sandbox work with Singapore
  • Microsoft – Natasha Crampton (closing keynote)
  • Virus Total – Security scanning for agentic systems
  • ML Commons – Benchmark work on multilingual evaluations

Academic & Civil Society

  • Masakani Research Foundation – Vukosi Marivate; distributed research on African language NLP
  • University of Pretoria – Vukosi Marivate
  • IIT Bombay – Partnership with Google on Indic language performance
  • Stanford – HELM evaluations mentioned
  • Lulaba AI – Vukosi Marivate co-founder (startup testing agents for local clients)
  • UK AI Security Institute – World-leading safety research on agents

Technical Concepts & Resources

AI Assurance Frameworks & Methodologies

  • AI Assurance Definition: Process of measuring, evaluating, and communicating whether AI systems are trustworthy, safe, work as intended
  • Tiered Assurance Approach: Risk-based assurance tied to use case (financial, healthcare, government vs. consumer); linked to reversibility of actions and agent autonomy levels
  • Frontier Safety Framework – Google DeepMind tool for testing models before deployment
  • Post-Deployment Monitoring Framework – Real-time failure detection, continuous monitoring (PI paper by Mardu)

Agent-Specific Standards & Protocols

  • Agents-to-Agents Protocol – Google's standardized communication protocol for agents
  • Universal Commerce Protocol – Standardized information exchange between agents and websites
  • Agent Attribution & Identity Standards – NIST/CASEY initiative (newly announced)
  • HTTP/URL Parallels – Calls for analogous foundational protocols for agent economy (early 1990s internet model)

Evaluation Infrastructure & Benchmarks

  • Stanford HELM – Foundation model evaluation (requires 12 billion tokens, 19,500 GPU hours)
  • ML Commons – Multilingual benchmark work
  • Multilingual Evaluations – Commitment from New Delhi Declaration; critical gap area
  • Usage Data Sharing Standards – Commitment to standardize how frontier AI companies share usage data

Papers & Documents Released

  • "Strengthening the AI Assurance Ecosystem" – PAI working group paper on building robust national assurance strategies
  • "Closing the Global Assurance Divide" – PAI paper identifying six challenge areas: infrastructure, skills, languages, risk profiles, capacity, documentation
  • "How We Strengthen the Global AI Assurance Ecosystem" – PAI companion paper
  • "Real-Time Failure Detection and Monitoring of Agents" – PI paper (Mardu)
  • New Delhi Frontier AI Commitments – Declaration signed at summit; Commitment 1 (usage data clarity), Commitment 2 (multilingual/contextual evaluations)
  • Singapore Model Governance Framework for Agentic AI – Live governance document for enterprises

Data & Linguistic Diversity Metrics

  • India: 120+ languages, 19,500 dialects
  • Africa: ~3,000 spoken languages
  • Global Connectivity: 2.6 billion people remain offline; many lack local language content or relevant applications

Monitoring & Accountability Tools

  • Chain-of-Thought Monitoring – Observing agent reasoning steps, not just outputs
  • Orchestration Auditing – Examining how agents coordinate multi-agent environments
  • Reversibility Frameworks – Assessing possibility of undoing agent actions and consequences

Governance Sandbox Models

  • Singapore-Google Sandbox on Agentic AI – Government-industry partnership for testing and validation

Conclusion

This summit session crystallizes a critical inflection point: as AI systems become agentic (autonomous, goal-directed, action-taking), assurance must evolve from a post-hoc compliance exercise to an embedded, continuous, globally-coordinated discipline. The overwhelming emphasis on Global South inclusion and North-South collaboration signals recognition that exclusion from assurance-building will deepen existing inequalities and undermine the legitimacy of governance frameworks.

The practical path forward hinges on: (1) building shared evaluation infrastructure and standards that are adaptable rather than monolithic, (2) professionalizing and incentivizing assurance providers regionally, (3) embedding assurance in system design from the start, (4) ensuring post-deployment monitoring capabilities, and (5) treating assurance as public infrastructure requiring collective investment—not proprietary competition.