Infrastructure & Compute

Overview

India stands at an infrastructure inflection point that will define its AI trajectory for decades. The country must scale from roughly 40,000 GPUs today to millions within a decade 1, while simultaneously solving energy supply, data governance, and workforce gaps that no amount of compute spending alone can fix. The stakes are concrete: credible projections put India's data center buildout at $90 billion or more 2, with demand potentially reaching 10–12 GW of AI compute within three years 3. What makes this moment distinctive is not just the scale of investment required but the breadth of strategic choices India must lock in now — on power architecture, sovereignty frameworks, cooling standards, and semiconductor supply chains — before infrastructure patterns harden and lock-in becomes irreversible 4. Getting these choices right matters not only for India but for the Global South more broadly, as India's hybrid public-private model is being watched as a replicable template 5.

---

Key Insights

• Compute is necessary but insufficient. The real bottlenecks holding back AI deployment are energy supply, data governance, skilled talent, and institutional capacity — not GPU count. Philanthropies and governments that overinvest in hardware while neglecting these complementary inputs will find infrastructure sitting idle 6.

• Energy must be the design starting point, not an afterthought. Building data centers first and then hunting for power is backwards engineering 7. Purpose-built AI facilities require integrated system thinking across compute, power, cooling, and telemetry from day one, with hybrid power systems — batteries, fuel cells, and storage — to manage pulsating loads that base-load generation alone cannot absorb 83.

• India's window for policy-setting is 18–36 months, not years. Establishing PUE standards, grid planning frameworks, renewable procurement requirements, long-term power purchase agreements, and state-level permitting clarity must happen now, before the infrastructure pattern is set 94. The U.S. experience — where infrastructure has become the bottleneck — is the cautionary tale 4.

• Sovereignty means verified trust and strategic agency, not autarky. Countries do not need to manufacture every chip or train every foundation model. They need cryptographically verifiable control over what runs on their infrastructure 10, the ability to steer critical stack layers (data practices, inference, safety standards), and deliberate interoperability rather than accidental lock-in 51. Hardware-rooted verification mechanisms are production-ready today, not theoretical 10.

• Distributed and edge inference, not centralized training, is India's immediate compute opportunity. With 300-plus startups building applications, variable connectivity, latency-sensitive use cases, and data sovereignty requirements, splitting AI workloads intelligently across device, edge, and cloud tiers is an economic and technical imperative 1112. Premium smartphones already run 10-billion-parameter models; AR glasses run 1–2 billion 13.

• The semiconductor feedback loop rewards early, large-scale commitment. Companies like Lam Research are using AI to design the next generation of tools that manufacture AI chips, creating a virtuous cycle 14. India's emerging design startups — Agrani (GPUs), C2I (power chips), Mangrove (SoCs) — represent a genuine architectural shift from consumption to production, but supply chain monopolies and CUDA lock-in remain hard blockers 15.

• Liquid cooling is mandatory, not optional, at modern rack densities. Any deployment above roughly 60 kW per rack requires liquid cooling; retrofitting is infeasible 16. Water-cooled systems minimize power draw but strain scarce aquifers, while air-cooled systems invert that trade-off — policy frameworks must enable evidence-based, region-specific design choices rather than uniform mandates 2.

• Open standards are sovereignty tools, not restrictions. The shift from proprietary stacks (Sun/SPARC) to open ecosystems (Linux, x86, PyTorch, RoCE-based Ethernet) creates room for regional players to build without vendor dependence 1718. Countries and enterprises that avoid CUDA lock-in through open standards can innovate faster and retain strategic autonomy 18.

• GPU product cycles (6–9 months) now outpace data center construction cycles (18–24 months). This timing mismatch means modular, retrofittable reference designs and standardized pods are competitive imperatives, not engineering preferences — they are what enable multi-generational GPU compatibility without infrastructure redesign 316.

• Cost reduction of 100–2,500x is achievable through algorithmic redesign before buying more hardware. Dynamic sparsity, pattern recognition techniques like MSET, and matching algorithm to problem type (neural networks for classification, deterministic systems for safety-critical domains) have demonstrated these gains in healthcare, aviation, and nuclear plant monitoring 19. The moral and environmental case for efficiency-first engineering is as compelling as the economic one 19.

---

Recurring Themes

• Energy as the binding constraint. Speakers across infrastructure, sustainability, finance, and geopolitics tracks independently identified power supply — not chips, not models — as the primary bottleneck. Grid interconnection delays, fragmented state policies, and the tension between data center water consumption and local aquifer stress came up repeatedly as structural risks that technical ingenuity alone cannot resolve 2789202116.

• The sovereignty-interoperability balance. A consistent tension ran through the summit between the impulse toward full-stack independence and the practical reality that autarky is neither achievable nor desirable. Speakers from government, startups, and multilaterals converged on a middle position: build verifiable control at the layers that matter most (data, inference, governance), partner deliberately at others (frontier models, leading-edge fabs), and enforce interoperability through open standards so that choice remains real 5222312425.

• Edge and distributed inference as the inclusion pathway. The claim that cloud-only AI excludes vast populations was not a fringe position — it was repeated by speakers from ITU, Qualcomm, Intel, domestic startups, and civil society alike. Offline capability, reduced latency, data privacy, and cost efficiency all converge on edge architecture as the mechanism for reaching India's next billion users 2611272813.

• The 3–5 year window is real and closing. Across compute, semiconductors, standards participation, and policy-setting, speakers from starkly different vantage points — venture capital, government labs, multilateral institutions — independently cited a 3–5 year window before current opportunities calcify into long-term dependencies 29233031. The urgency was not rhetorical; it was grounded in specific timelines: ISM 2.0 rollout, GPU scaling targets, renewable procurement cycles, and international standards processes.

• Workforce and institutional capacity as co-equal infrastructure. Hardware investment without parallel investment in skills, curriculum reform, and institutional governance capacity was flagged repeatedly as the failure mode most likely to be underestimated. India needs roughly 1 million semiconductor-trained workers in 5–7 years 14, and the gap is widest in state-funded institutions and tier-2/3 regions 32. E-waste recycling — 2 million tons currently poorly managed — was also cited as a circular economy obligation that infrastructure scaling will worsen without deliberate policy 7.

---

Open Challenges & Tensions

• Who bears the cost of grid modernization for data centers, and who benefits? As hyperscale facilities concentrate demand on state grids, the risk of stranded distribution utility assets and tariff increases for other consumers is real and unresolved 9. The question of whether data centers should provide ancillary grid services (demand response, reactive power) in exchange for long-term power commitments — and how to regulate this — has no consensus answer, and current 7-year PPA structures are too short for the 15–20 year planning horizons hyperscalers require 4.

• Open-source models and the erosion of Western closed-source dominance. Multiple speakers noted that Meta's retreat from aggressive open-source development has ceded ground to Chinese models, with Qwen cited as demonstrating superior performance for resource-constrained developers 24. This creates a genuine strategic dilemma for India: Chinese open-source models are often the most practical choice on performance and cost grounds, yet depend on actors with different geopolitical alignments. No clear policy resolution was offered.

• Space-based edge computing: serious infrastructure or speculative bet? The NeevCloud/Agnikul proposal for orbital inference infrastructure was presented as economically competitive once cooling, land, and permitting costs are factored in 33. Other infrastructure speakers did not engage with this claim, leaving it unchallenged but also unvalidated. The next 3–5 years of radiation-hardened GPU development and orbital cost curves will be determinative.

• How to fund the research and institutional capacity that no market actor will build. The gap between 99 percent of AI investment flowing to acceleration and less than 1 percent to safety and governance infrastructure 34 mirrors a parallel gap in compute research: national laboratories with living testbeds for chip-to-grid optimization exist in concept 8 but are chronically underfunded relative to commercial deployment. ANRF's ₹1 lakh crore allocation 30 is large by Indian standards but small relative to the investment being made by hyperscalers.

• Measurement without over-regulation. There is genuine disagreement on whether mandating standardized energy and water reporting through the Bureau of Energy Efficiency is sufficient 9, or whether stronger instruments like performance-based subsidy clawbacks and PAT-scheme-style targets are needed. The concern that prescriptive PUE mandates will lag technological change (liquid cooling changes the baseline) is legitimate, but purely voluntary reporting has a weak track record in infrastructure sectors globally 29.

---

Notable Examples

• India AI Mission compute buildout. The government has secured 38,000–50,000 GPUs under the India AI Mission, with targets to scale beyond 128,000 30, using a government-funded, private-sector-operated model that has compressed deployment timelines and maintained commercial incentives alongside democratic oversight 51. Real applications — farm subsidy processing, UPI fraud detection, multilingual citizen services — are already generating gigawatt-scale demand, not hypothetical future demand 3.

• ISM 2.0 and domestic semiconductor startups. India's Semiconductor Mission 2.0 is funding design and manufacturing across verticals, with Agrani (GPUs), C2I (power management chips, tape-out April 2025), and Mangrove (SoCs, generating revenue in 2024) representing the first wave of domestic AI silicon 15. The Semiverse virtual training platform, developed in partnership with Lam Research, ISM, and Indian universities, is delivering results two-plus years ahead of schedule 14.

• Bhashini and multilingual voice infrastructure. India's government-backed Bhashini platform, supporting 22 Indic languages and developed in collaboration with Intel, academia, and startups, is the clearest example of language AI treated as public infrastructure rather than a commercial product 28. It provides the foundational layer that voice-first AI agents — necessary to reach non-English-speaking populations — require.

• CAG's operational AI deployment for public audit. The Comptroller and Auditor General has moved from pilot to production with OCR-based beneficiary duplicate detection, satellite imagery analysis for scheme monitoring, and a three-year roadmap to shift model development from external contractors to 90 percent CAG-officer-led capability 35. This is among the most concrete examples of sovereign AI as a compliance and governance necessity rather than a strategic aspiration.

• Sustainable Digital Infrastructure Accord (SDIA). The Asia-Pacific industry-led initiative establishes four measurable sustainability pillars — PUE targets, clean energy coverage, water management, and circular economy practices — without the top-down regulatory structure of the EU approach 2. India's 220 GW renewable capacity (targeting 500 GW by 2030) positions it well, but fragmented state policies and slow grid interconnection are cited as the execution gaps that the SDIA framework alone cannot close 2.