India AI Impact Summit Insights
All sessions

Panel AI at Scale

Contents

Executive Summary

This panel discussion explores how AI is being scaled in emerging markets, focusing on two distinct domains: social media recommendations and telecom infrastructure. The speakers emphasize that profitability and unit economics are as critical as technical excellence, and that building sovereign AI capabilities (LLMs, recommendation systems, edge computing) is essential for countries like India and Indonesia to avoid digital monopolization by US and Chinese tech giants.

Key Takeaways

  1. Profitability beats sophistication: A cheap, efficient recommendation system serving billions at $2 ARPU is strategically superior to a marginally better system that can't achieve unit economics.

  2. Real-time infrastructure (not model architecture) is the competitive moat: The ability to train and serve models in real time, with live feature stores and low-latency pipelines, matters more than model size or novelty.

  3. Generative AI abundance makes recommendation systems more valuable, not less: As content creation explodes, the role of intelligent filtering becomes even more critical; this is an opportunity, not a threat, for platforms that have mastered recommendations.

  4. Sovereign AI (LLMs, platforms, edge compute) is a geopolitical and economic necessity: Countries cannot rely on US/Chinese tech companies for core AI infrastructure; they must invest in localized, regulated, open-source alternatives or risk long-term dependency and data loss.

  5. Emerging market unit economics ($2–10 ARPU) require fundamentally different engineering approaches: Cost efficiency cannot be an afterthought; it must drive architecture, algorithm choice, and infrastructure from day one.

Key Topics Covered

  • Cost-efficiency in AI systems: Moving from high-cost ($100M+ annual cloud bills) to profitable AI through algorithmic efficiency
  • Recommendation systems as core competitive advantage: Contrasting approaches to content and ad relevance optimization
  • Organizational transformation for AI: Culture change, horizontal working models, talent development in emerging markets
  • Sovereign AI infrastructure: LLMs, edge computing (RAN), and localized AI platforms as strategic imperatives
  • Generative AI's impact on content creation: Lowering barriers to creation and increasing algorithmic complexity
  • Telecom-as-intelligence platform: Integration of AI with connectivity infrastructure for rural and underserved markets
  • Quantum computing's future implications: Cryptographic risks and potential acceleration of AI workloads
  • Enterprise AI adoption: Current state of inferencing and deployment challenges in developing markets

Key Points & Insights

  1. Unit economics determine viability: A multitask deep learning model deployed in 2022 delivered 40–50% time-spent gains directly correlating to ad revenue. However, recommendation systems must be cost-optimized for tier-2/tier-3 markets (operating at $2–3 ARPU instead of $100+ in developed markets).

  2. Infrastructure and real-time capability trump modeling complexity: The "magic" of TikTok's algorithm lies not primarily in the model itself but in real-time training pipelines and feature infrastructure. The speaker claims they are ~90% competitive with TikTok on this dimension, not 50%.

  3. Organizational culture is prerequisite for scaling AI: Moving from vertical, siloed structures to horizontal cross-functional teams requires buy-in from 5,000+ employees and top leadership. Without cultural alignment, technical solutions fail to scale.

  4. Generative AI is reshaping content abundance and algorithmic importance: LLMs and video generation lower the barrier to content creation, moving from millions to potentially billions of daily posts. This makes recommendation algorithms even more critical because discovery becomes harder, not easier.

  5. Recommendation systems are as strategically important as LLMs but less visible: While LLMs dominate public discourse, proprietary recommendation systems are "the fundamental part of what builds the internet." Recommenders are sovereign capabilities that should not be outsourced to global platforms.

  6. Edge computing (AI RAN) is essential for serving rural/underserved populations: Deploying AI on mobile networks (edge) rather than centralized cloud allows sub-50ms latency, enabling use cases (health, education, connectivity) that require proximity to users. This positions telcos as distribution channels for intelligence, not just connectivity.

  7. Data sovereignty and localized models are existential concerns: The panel repeatedly warns against "digital colonization"—allowing foreign companies to extract local data and return only finished products. Countries must build their own LLMs, platforms, and guard rails; open-source approaches (e.g., Sahabat LLM) are framed as politically and economically preferable.

  8. Profitability in emerging markets requires extreme cost discipline: One speaker reduced cloud costs from $100M+ annually to one-third by improving algorithmic efficiency. At $2–3 ARPU, a $10 per-user AI cost model is unsustainable; targets are $1 per user.

  9. Talent development in India and Indonesia is feasible but requires time and organizational commitment: Half of the top modeling talent at one company is now India-based after 7–8 years; Indian engineers can reach parity with global talent given scale and experience.

  10. Timing and infrastructure bets are non-negotiable: Early adoption of GPU infrastructure (H100, GB200) and investment in sovereign platforms (Sahabat LLM, AI RAN) position countries and companies for dominance. Waiting for market clarity risks permanent disadvantage.

Notable Quotes or Statements

  • "You can build great tech but when you have to make it profitable you realize it's a different ballgame." — On the tension between technical excellence and commercial viability.

  • "AI is more about people than tech." — (Vikram) On organizational transformation as a prerequisite for scaling AI.

  • "The magic really was: can you make the entire system real time?" — On TikTok's competitive advantage being infrastructure, not modeling.

  • "We have to be on the lead, not in the loop" — (Vikram) On ensuring humans lead AI transformation, not merely supervise it.

  • "Think of it like connectivity plus intelligence delivered at edge in a sovereign manner is what countries like India and Indonesia need." — On the strategic value of edge AI for underserved populations.

  • "We're entering sci-fi territory" — (Ankush) On moving from retrieval (finding content) to generation (creating content) as the next frontier for social platforms.

  • "Digital colonization or digital monopoly is the biggest risk any country or company can have." — (Vikram) On data sovereignty and the dangers of relying on proprietary foreign models.

  • "From a billion [pieces of content], how do I find the 10 most relevant? The role of the recommender system becomes even more important." — On algorithmic complexity in an age of abundance.

Speakers & Organizations Mentioned

  • Ankush — Appears to lead a social media/content platform (likely ShareChat based on context); focuses on recommendation systems and content distribution in India.
  • Vikram — Senior leader at Indosat (Indonesian telecom); driving AI transformation, Sahabat LLM initiative, and AI RAN deployment.
  • References to competing platforms/companies:
    • TikTok / ByteDance (benchmark for real-time training and personalization)
    • Meta (high ARPU, set standards in recommendation systems)
    • T-Mobile US, SoftBank Japan (AI RAN leaders)
    • OpenAI, Gemini (generative AI context)

Technical Concepts & Resources

  • Multitask deep learning models: Deployed to optimize time-spent metrics and ad revenue.
  • Real-time training pipelines: Core infrastructure advantage; moving beyond static feature stores to live model training.
  • Feature stores and data pipelines: Essential infrastructure layers; real-time capability is the differentiator.
  • AI RAN (Radio Access Network): Embedding AI at the edge of mobile networks; enables low-latency, sovereign compute for telecom use cases.
  • AI Grid: Distributed edge AI deployment across 55,000 cell sites to serve rural/low-connectivity areas.
  • Sahabat LLM: Open-source language model platform developed by Indosat; supports multi-lingual/multi-dialect inference; positioned as alternative to proprietary models.
  • GPU infrastructure: H100, GB200, and upcoming Ruben deployment; essential for training and inference at scale.
  • Quantum computing: Mentioned as a future risk (cryptography) and opportunity (acceleration of matrix multiplication, core to LLMs).
  • Ad relevance optimization: Deep funnel modeling (beyond clicks); matching 1000+ product SKUs to users for e-commerce advertisers.
  • Generative AI for content: Multimodal models (text, images, video) lowering creator barriers; increases content abundance and algorithmic demand.
  • Neo Cloud (by Indosat): Regional cloud infrastructure with 28 customers (18 in Indonesia); focus on model building and inference workloads.

Additional Context

  • Financial metrics referenced:

    • 40–50% time-spent gains from multitask models
    • $100M+ → $33M annual cloud cost reduction
    • Churn reduction: 3% → 1.4% (>90 days)
    • ARPU growth: 7% industry baseline → 14% achieved
    • $2–3 ARPU (emerging markets) vs. $100+ (developed markets)

  • Timeline highlights:

    • 2018: Start of AI journey
    • 2020: Shift to building global talent teams
    • 2022: First major multitask model deployment
    • 2024/forward: Transition from retrieval to generation; edge AI at scale; sovereign LLM expansion