AI in Fintech: Solving for India at Scale

Key Topics Covered

• Infrastructure Architecture: Custom AI/ML platform design, container orchestration, LLM gateway ("Godric"), and model deployment strategies
• Agent Framework & Tools: Java-based agent framework, Agent Hub (configuration platform), and internal tools like Cleo and Sentinel
• Security & Compliance: Data sovereignty, regulatory auditing, authorization systems, and preventing data leakage outside Indian borders
• Software Engineering Adoption: AI-assisted coding, documentation generation, code review, testing automation, and design-to-code conversion
• Operational AI Use Cases: Fraud detection, merchant KYC/classification, HR automation, document processing, and investigation assistance
• Consumer-Facing Features: Intelligent search, transaction insights, credit score analysis with explainability, and AI-powered customer support
• Strategic Philosophy: Problem-first approach, avoiding over-hype, human-in-the-loop decision-making, and building for local needs
• Open Source Contributions: Container orchestrator, agent framework, Mantics security software, and various libraries made available to the community

---

Key Points & Insights

1. Scale Drives Architecture Decisions: PhonePe's infrastructure (650M users, used everywhere from hospitals to auto-rickshaws) demands custom solutions—they built their own container orchestrator instead of Kubernetes, avoiding unnecessary complexity and enabling stateless deployment patterns critical for financial systems requiring zero data loss.

2. Data Sovereignty as Non-Negotiable: The company routes API calls through a custom "Godric" gateway that enforces geo-fencing—preventing sensitive customer data from crossing borders while maintaining access to global models (Azure Foundry) for non-sensitive use cases. Compliance auditing is embedded, not bolted-on.

3. Homogeneous Tech Stack as Advantage: Rather than adopting every new tool, PhonePe intentionally maintains consistency in infrastructure, making it easier to weave AI capabilities into existing systems without disrupting workflows. This reduces organizational friction during adoption.

4. 70-75% Generic Solutions Require 25-30% Custom Work: Off-the-shelf LLM tools solve a portion of problems but demand significant tuning—prompts, parameters, observability, and domain-specific layers. PhonePe invests in proprietary tooling (e.g., Cleo for code documentation, Agent Hub for knowledge management) tailored to fintech workflows.

5. AI Unlocks Organizational Efficiency Before Consumer Features: PhonePe's adoption ladder prioritizes engineers (developer tooling), operations (merchant KYC, fraud investigation), then consumers. This allows internal learning before scaling externally and reduces toil so engineers focus on creative problem-solving.

6. Hybrid On-Device + Server Processing for Privacy: Consumer features like intelligent search combine server-side inference for complex queries with local device inference for pre-existing data. This architecture ensures sensitive transaction data never leaves the user's device while enabling personalization.

7. Problem-First, Not Technology-First: PhonePe rejects arbitrary adoption goals ("use AI for 30% of features"). Instead, teams identify genuine pain points—merchant classification, transaction insights, credit score explainability—and apply AI only where it demonstrably solves them. This prevents wasteful experimentation.

8. Explainability as Competitive Advantage: Credit score analysis includes video summaries explaining why a score is what it is and what specific actions improve it, addressing India's low financial literacy. Generic systems provide rules; PhonePe's AI provides context.

9. Trust as Core Currency: In regulated fintech, security and reliability are not features—they're prerequisites. PhonePe embeds compliance checks into Godric (authorization, quota management, audit trails), making governance scalable rather than a bottleneck.

10. India Must Solve Its Own Problems: Executives emphasize that voice-first payments, AI-native commerce, and other innovations suited to India's context require homegrown ventures to innovate. Global solutions (ChatGPT interfaces everywhere) don't reflect local needs.

---

Notable Quotes or Statements

• Shantanu (on scale): "When you have to build for something that affects everyone everywhere all the time, right? You use it in hospital, you use it in autos, you use it to order food, you use it for groceries everywhere."

• Shantanu (on avoiding premature adoption): "We did not want to set up goals right away before we ourselves understood what is possible."

• Shantanu (on custom infrastructure): "We don't use Kubernetes... we wrote a simple controller orchestrator which basically works really really fast when deploying stateless containers and scaling up."

• Puja (on engineering philosophy): "Software engineers at Phone Pay are not just coding... every feature requires engineers to go deep into the problem statement, understand business cases, analyze data, discuss architecture—and only then coding begins."

• Puja (on AI's role): "Everything that goes out in production is still human reviewed... decision-making architecture decisions are still driven by engineers... AI assists."

• Gautam (on modality innovation): "Is chat really the right interface or the right modality we don't know... I would rather get the customer to use things the way they want to use and then figure out how we can really refine it further."

• Gautam (on homegrown innovation): "India needs to be able to solve its own problems with things that it is building for itself... it has to come from homegrown ventures."

• Shantanu (on trust in fintech): "The last thing we want are inconsistent wallet balances... we don't run any stateful containers... state gets stored remotely on proper database servers... we can't have any kind of data loss anywhere."

---

Speakers & Organizations Mentioned

Speaker	Role	Organization
Shantanu Deshpande	Chief Architect	PhonePe
Puja Gupta	(Engineering focus)	PhonePe
Gautam Sharma	(Consumer/Strategy focus)	PhonePe
Anoj	Fraud Detection & Fraud Engine Lead	PhonePe
(Government/Regulatory)	Implied through UPI, RBI compliance	Indian Government

Key References:

• PhonePe: 650M+ registered users, 47M merchants, 300M monthly active users, 53B transactions (6-month period) worth ~$870B
• Indian Government Initiatives: Aadhaar (identity infrastructure), UPI (Universal Payments Interface), banking access programs
• Industry Context: Payments processed rival Visa/Mastercard volumes; governed by RBI regulations and UPI guidelines

---

Technical Concepts & Resources

Infrastructure & Deployment

• Godric: Custom LLM gateway providing unified interface (chat completions API standard), quota management, authorization, routing (local GPU vs. cloud models), experimentation, auditing, and compliance enforcement
• AIML Platform: Existing ML infrastructure extended to handle batch, real-time, and edge (on-device) model inference; handles autoscaling, deployment, observability
• Custom Container Orchestrator: Stateless-only design (state → remote databases), no Kubernetes, integrated service discovery, sidecar monitoring/authorization
• VLM / SGLang: Model serving frameworks with configuration management at the model level (tuning parameters, inference modes)
• Gluster FS: Open source distributed filesystem (PhonePe contributes maintainers); S3-compatible storage for large model weights and adapters

AI/ML & Agent Systems

• Agent Framework (Java-based): Extensible architecture with toolboxes, MCP servers, template definitions, service call composition, context window management, agent memory, agent registry (skills), multi-agent orchestration
• Sentinel: Internal name for the agent framework (open source, with QR code reference provided)
• Agent Hub: Configuration-driven agent deployment platform (derived from open source docs-gpt); manages 200-300+ configured agents; integrates knowledge sources (Slack, Confluence, compliance documents), RAG/graph RAG, MCP servers, API integrations, user permissions
• Seas: Custom storage system encapsulating RAG and graph-RAG; built-in embeddings; schema-based document ingestion
• Cleo: Tool for reverse-engineering code documentation from existing codebases (reduces manual documentation burden)

Specialized Domain Tools

• Fraud Investigation Assistant: Automative case classification, red-flag identification, preliminary investigation automation
• Merchant KYC & Classification: Document analysis, business classification, automated onboarding reduction
• HR Bot: Knowledge base for leave policies, reimbursements, HR process navigation
• Figma-to-UI Code: AI workflow converting design mockups to application code
• Code Migration Recipes (based on Logan open source): Suggests code changes for library/dependency upgrades
• UI Test Case Generation: LLM-assisted generation of test cases from natural language specifications (login screens, workflows)
• Credit Score Explainability: Video summaries (models + LLM) explaining credit scores, behavior insights, actionable recommendations

Security & Compliance

• Custom OAuth / Authorization System: Handles "insane scale" of API calls; per-container authorization; human-to-system call auditing
• Quota Management: Prevents individual teams from exhausting GPU capacity through frivolous workloads
• Audit Trails: Long-term immutable storage of who accessed what, when; queryable for regulatory compliance
• Routing Logic: Geo-fencing enforces data sovereignty—sensitive data routes through on-premises GPU infrastructure; internal/non-sensitive workloads can use Azure Foundry models
• Mantics: Security scanning software (open source) for domain vulnerability assessment

Open Source Projects Released

• Container Orchestrator (simplified alternative to Kubernetes for stateless workloads)
• AI Agent Framework (Sentinel)
• Mantics (security scanning tool)
• Supporting libraries and integration examples

Models & Services Mentioned

• Azure Foundry Models: Foundational LLMs accessed via Azure (with geo-fencing for compliance)
• On-Premises GPU Servers: PhonePe's own datacenter model deployment
• Multi-Provider Strategy: PhonePe integrates with multiple LLM providers; partnerships vary

Observability & Monitoring

• Custom Distributed Tracing (not standard distributed tracing): Traces overall flow across graph/infrastructure; avoids expense of traditional APM at PhonePe's scale
• Sidecar-Based Metrics: Request counting, CPU usage, latency monitoring; embedded in container orchestrator
• Audit Integration: Authorization and usage data fed into compliance systems

---

Strategic & Organizational Insights

Decision-Making Philosophy

• Internal-First Adoption: Infrastructure and tooling built/tested internally before external rollout
• No Arbitrary Goals: Rejection of "automate X% of processes" targets; instead, identify real pain points
• Human-Centered: All production decisions remain human-reviewed; AI is assistant, not decision-maker
• Patience with Experimentation: 70-75% generic solutions accepted; tuning and custom layers essential

Cultural / Organizational

• Homogeneous Tech Stack Preference: Consistency reduces friction; weaving AI into existing systems easier than replacing workflows
• Engineering-Centric Adoption: Engineers are first adopters (tooling, CI/CD); learnings then propagate to operations and consumers
• Compliance as Enabler, Not Blocker: Heavy auditing keeps system "super duper clean"; executives trust the architecture

Market & Vision

• India-Specific Problem-Solving: PhonePe emphasizes need for homegrown solutions tailored to India's context (voice-first, mobile-first, low financial literacy)
• Inclusion as North Star: Vision to "unlock access to money and financial services to every Indian" guides feature prioritization
• Long-Term Over Quick Wins: Differentiation comes from reliability, security, user trust—not flashy features—built over time

---

Caveats & Limitations

• Transcript Accuracy: Transcript appears to have transcription artifacts (repeated phrases, unclear sections); some technical details may be paraphrased or incomplete
• Timeline Ambiguity: Specific launch dates for consumer features are intentionally vague; some tools are still in beta
• Proprietary Details: PhonePe holds back architectural specifics (OAuth implementation, fraud detection algorithms); focus is on philosophy rather than implementation details
• Limited External Validation: Claims about scale and performance not independently verified

---

Conclusion

PhonePe's presentation articulates a disciplined, context-aware approach to AI adoption that contrasts with industry hype. By prioritizing internal mastery before external deployment, embedding compliance into infrastructure, solving specific problems rather than chasing PoCs, and maintaining human oversight, PhonePe demonstrates how a fintech incumbent can responsibly scale AI to serve financial inclusion at a billion-user scale. The emphasis on homegrown innovation and India-specific problem-solving suggests a broader argument: global AI solutions must be adapted—or reimagined—for local markets.