Gautam Gupta

Ruokai Yin, our summer intern from Yale to Microsoft Azure, worked with Xuan Zuo, @Sattwik Deb Mishra, Hokchhay T. and Preyas Shah and me. We published a paper based on his project: https://lnkd.in/eHuG3BSt Summary of paper: Finding optimal sharding is hard especially for large LLMs. A large number of devices have to be divided into TP, PP, EP dimensions. For TP, we have to decide the tensor dimension e.g. head, hidden size to shard for every op. Additionally, TP can be one-dimensional/two-dimensional etc. This makes for a large sharding space that is hard to cover exhaustively. SoTA methods use heuristics to select sharding e.g. GSPMD, Sequence Parallelism, Helix etc. There are significant time gaps, months or years, between the discovery of each of these methods. We want to fast-forward the process of discovery by 1) using a formal representation of the sharding space, and, 2) using reinforcement learning to search the sharding space.    This paper demonstrates that our method is 3.5x better than random search, and on par with hand-tuned heuristics when looking only at 1D sharding and without allowing sequence parallelism. We aim to extend the work to multi-dimensional sharding as well as sequence parallelism. Beyond that, we wish to combine the problem of sharding with fusion, tiling and collectives as well.

58

Dr.Sheth’s thoughts on building the required talent and ecosystem in India to spearhead its foray into deep tech AI products/companies is both timely and critical, for strategic autonomy over the long term. 

22

3 commenti

𝐆𝐨𝐨𝐠𝐥𝐞 𝐭𝐨 𝐈𝐧𝐯𝐞𝐬𝐭 $𝟏𝟓 𝐁𝐧 𝐢𝐧 𝐀𝐈 𝐃𝐚𝐭𝐚 𝐂𝐞𝐧𝐭𝐫𝐞 𝐢𝐧 𝐕𝐢𝐳𝐚𝐠 Google Cloud CEO Thomas Kurian announced that the tech giant will invest $15 billion to build the world’s largest AI and data centre in Visakhapatnam, Andhra Pradesh. This hub will be Google’s biggest AI facility outside the US and will feature Tensor Processing Units (TPUs) for cutting-edge AI computing. “This gigawatt-scale AI hub represents a major step in supporting India’s digital infrastructure goals, and we’re grateful for the support of the Government of India and the state of Andhra Pradesh,” said Kurian. Google Cloud CEO Kurian made this announcement during Google’s event on “India’s AI-Powered Future and Advancing the Vision of Viksit Bharat 2047,” in the presence of Ashwini Vaishnaw, Minister of Electronics & IT; Nirmala Sitharaman, Finance Minister; and N. Chandrababu Naidu, Chief Minister of Andhra Pradesh. Google Cloud Thomas Kurian Ministry of Electronics and Information Technology Via: Outlook Business | Deepsekhar Choudhury (Story link in the comment)

21

1 commento

🚀 Inside the Mind of Google's AI Visionary: Demis Hassabis' Radical Approach to Technology Unlocking the profound power of AI over mere profits. Google’s Demis Hassabis is driven by cosmic ambitions, not just business gains. As the founder of DeepMind and top AI executive at Alphabet, Hassabis has spent over a decade pushing AI boundaries—not for quick wins, but to solve humanity’s biggest challenges. While Google lags in market dominance against rivals like OpenAI, Hassabis favors a long-term vision inspired by philosophy and science fiction. Here’s what stands out to me:    - DeepMind’s journey reflects a deliberate choice to prioritize deep research over immediate commercial success, focusing on breakthroughs like AlphaFold’s Nobel-winning protein modeling.   - Hassabis turned down lucrative ventures, including partnerships with OpenAI and projects in financial trading, emphasizing AI’s safe and responsible development above profits.   - His vision for AI aims to solve “root node” problems, from curing diseases to creating a universal assistant that enhances productivity and wellbeing.   - Despite pressure, Hassabis pushed for DeepMind’s autonomy to shield AI development from corporate profit motives, highlighting his commitment to responsible innovation.   - Colleagues praise his ambitious mindset, always asking “Why not solve everything?” rather than settling for incremental gains. In the fast-evolving AI race, Hassabis reminds us that true innovation often requires patience, a scientific spirit, and a focus on impact over income. What’s your take on balancing visionary goals with business realities in AI development?  #AI #Innovation #FutureOfTechnology #DeepMind https://lnkd.in/ggS8FsgG

6

Andrew Ng reported from Davos, that AI deployment for transformative impact will require workflow redesign. It is "theory of constraints" all over again (remember Eli Goldratt's "The Goal"?), we have to understand that real business benefits come only from working across silos, from analyzing flow bottlenecks, from understanding that optimizing non-bottlenecks creates waste. As Ng noted, letting a thousand (AI) flowers bloom has failed to lead to significant payoffs. Obviously! We in the silicon industry are no strangers to this. Workflows have been stagnant for decades. Technologies for high level specification, high level synthesis, and formal verification are not yet integrated into the full silicon development flow. They are often done piece-meal, and enthusiasts' pilots are dropped at the hint of a failure or delay. People say they want formal verification in their flow, but want it yesterday and do not have the time to invest in training engineers for its rigorous usage. Ashish Darbari, responding to a question on "if formal verification is so powerful then how come it is not widely used", noted that the problem is in the name - the actual task to be done is formal analysis while the bug detection is only a corollary, but the industry puts the cart before the horse. Based on my experience with industrial practice of high level specifications and formal verification, I tend to agree with his succinct comment. Jan Rabaey, in a DAC keynote speech, said that someday he expects to see a comic book series entitled "adventures in formal verification". AI deployment faces the same problem, if not actively worked on by business leaders. To be fair, deployments of all disruptive technologies face this mental barrier, we are so used to thinking in terms of the old flow (using the old technology) that thinking of a new flow literally means us having to begin again from first principles - few leaders have the courage and the time to do that. Goldratt mentioned that he himself had faced the same mindset problem when deploying the early ERP solutions and was surprised why he wasn't getting the expected large benefits, and he said it took him many years before he could identify the core problem and define solutions for it, which he named (rather academically) as "theory of constraints". AI leaders would do well to rework the conventional workflows in a holistic manner in order to get the real benefits from this new technology. As Henry Ford had said, "If you always do what you've always done, you'll always get what you've always got".

21

Attending the Generative AI Summit in Santa Clara today, I had a moment of clarity during the talk by Tript Singh Lamba, Former SVP/CPO of consumer group, Expedia. His session on Product-led Growth in a World of AI underscored a critical truth: AI literacy is now a core hard skill across all tech industry. Whether you’re in product, engineering, or sales, understanding and leveraging AI is no longer optional. • Product leaders must know how to leverage AI to drive growth. • Architects need to navigate a growing ecosystem - Bedrock, Azure AI Studio, Vertex AI, and more - and know when to apply Generative vs. Traditional AI. • Sales teams should embrace AI-powered outreach and value-based selling. • Developers now debate Claude vs. Copilot as passionately as they once debated React vs. Angular. In this fast-evolving landscape, foundational model polygamy is the new normal, and the learning curve is steep. But that’s exactly what makes this moment so exciting. Some will exit. Some will adapt. Personally, I’m here for it. Tript Singh Lamba , thanks again for the inspiring session. And one more thing I’ve noticed - AI events content has really leveled up. Less fluff, more substance. It’s a great time to be learning.  #GenerativeAI #ProductLedGrowth #TechLeadership #AIArchitecture #SalesEnablement #AIAdoption #FutureOfWork #SantaClaraAI #VertexAI #AzureAI #AmazonBedrock #FoundationalModels #TechTrends #AIEvents

21

4 commenti

Demystifying AI at Stanford Continuing Studies & the HAI AI Index A brief recommendation for two resources: 1. Stanford Continuing Studies - several courses offered online and in person address key demand for AI education for technical leaders, investment leaders and ethicists. https://lnkd.in/gk5MJ2PC. I've attended several rewarding classes here in the past year which expanded my professional network and my knowledge of AI ethics and my ability to track the evolution of the AI industry. In my latest class Demystifying AI, Nestor Maslej treats his class to week-by-week insight at an analytical level not typically available in public news sources. 2. The HAI AI Index - https://lnkd.in/geSqc6Bs - lies at the core of the last course I attended, led by the editor of the AI index (https://lnkd.in/gXu2HkqN). Nestor and the HAI team are looking at the industry in ways that enable meta-viewpoints on progress and capability. I have no affiliation with Stanford - just a grateful review of effective curricula and for gaining the skills needed to peer into & shape our AI future. And sharing an observation on the current era of AI - this is a good time for people who have both technical and legal training to contribute to our AI future.

15

From PoC → Production is where most AI journeys fail. At MLDS 2026, I’ll talk about building real-world financial AI agents that are reliable, scalable, and trustworthy. See you in Bengaluru.

56

𝗛𝗼𝘄 𝗠𝗖𝗣 𝗦𝗲𝗿𝘃𝗲𝗿𝘀 𝗠𝗮𝗻𝗮𝗴𝗲 𝗦𝗲𝘀𝘀𝗶𝗼𝗻𝘀 𝘄𝗶𝘁𝗵 𝗟𝗟𝗠𝘀 𝗮𝗻𝗱 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗪𝗶𝗻𝗱𝗼𝘄𝘀 Managing sessions in LLMs isn’t just about keeping track of prompts. It is about orchestrating memory, context, and tools seamlessly across multiple agents. That’s where MCP servers come in. Here’s the flow I captured visually → Context Window as Working Memory: Holds system prompt, user inputs, assistant responses, tool calls, and rolling context → MCP Servers: Each server maintains its own session state, ensuring consistency across interactions → System Prompt + Tools + Code Loops: Work together to power agentic behavior while grounding outputs in real tasks → LLM Integration: The context is constantly summarized and updated before being sent to the model This structure ensures scalable, multi-session orchestration, which is critical for enterprise AI where multiple tasks and tools interact simultaneously.

149

14 commenti

🌐 Had a fascinating conversation with Sabeer Bhatia (Hotmail co-founder) at TiEcon about the future of computing—particularly quantum. The consensus? 👉 Quantum chips won’t replace digital chips. They’ll augment them—just like GPUs did for AI. We discussed emerging quantum modalities: Superconducting (IBM, Google) Trapped Ions (IonQ, Quantinuum) Photonics (Xanadu, PsiQuantum) Neutral atoms (ColdQuanta) Topological qubits (Microsoft) Some great insights from leaders in the field: 🧠 Chetan Nayak (Microsoft): "Most quantum systems today are like analog radios—fragile and noisy. With topological qubits, we’re building something closer to digital transistors: stable, scalable, and resilient." 🧠 Jay Gambetta (IBM): "Quantum won’t replace classical—it’s about expanding the computational toolbox. The future is hybrid: CPUs, GPUs, and QPUs solving what no one system can." 🚛 Arvind Ratnam (QCNTRL): "Quantum chips are already solving problems where GPS fails—underground, underwater, or in jammed environments. That’s game-changing for logistics, defense, and autonomy." 🔬 Use cases gaining traction: Drug discovery Logistics optimization Post-quantum encryption Quantum-enhanced AI It’s clear: Quantum computing is becoming a critical co-processor layer—not a replacement. The next decade of computing will be hybrid, intelligent, and cross-disciplinary. #QuantumComputing #AI #FutureOfTech #TiEcon #QuantumChips #Microsoft #IBM #QCNTRL #DeepTech #Innovation #HybridComputing

32

1 commento

There is no doubt that everyone agrees that AI is the revolutionary technology, and Jensen is strategically expanding the market of GPUs with his cash. Eventually, we may only need maybe 2 super intelligent LLMs running in 3-5 data centers to support all the applications. Who will be the winners is not settled yet. When the winner comes out, Nvidia will need the software market to make up the loss of hardware market.

3

1 commento

Purpose vs Task: A Powerful Lens for the AI Era 🚀🤖 Jensen Huang has been consistently sharing a simple but profound framework across interviews (including a viral moment on The Joe Rogan Experience): Purpose vs Task. 👨‍💻 Task: Writing code 🎯 Purpose: Discovering and solving new problems His point is clear — coding is a means, not the mission. As AI gets better at handling repetitive and mechanical tasks, humans are freed to focus on what truly matters: creativity, judgment, and problem discovery. 💡 The real leverage in tech won’t come from who writes the most code, but from who asks the best questions and defines the right problems. AI doesn’t replace purpose. It amplifies it. #AI #PurposeVsTask #FutureOfWork #SoftwareEngineering #ProblemSolving #TechLeadership https://lnkd.in/eAMkutQy

3

The paper “Challenges and Research Directions for LLM Inference Hardware” by Patterson and Ma frames what the industry is already talking about - inference is becoming constrained less by compute and more by memory bandwidth and capacity. As model sizes push toward 10 trillion parameters, an HBM-only strategy becomes economically and physically untenable. The authors argue for a tiered memory hierarchy and position High Bandwidth Flash (HBF) as a capacity-oriented extension of the memory system rather than as traditional storage. HBF borrows the 3D-stacking model from HBM but applies it to NAND flash. Instead of a discrete SSD controller connected to flash over a narrow interface, HBF integrates logic and NAND dies into a single vertical stack. A logic base die - similar in role to HBM’s base layer - implements the PHY, multi-channel flash controllers, and ECC. Vertical TSVs run through the stack, enabling thousands of parallel data paths and aggregate read bandwidths of ~1.6 TB/s per stack. A single HBF stack can provide ~512 GB of capacity, compared to 48–64 GB per HBM4 stack. The interface uses wide, synchronous, DDR-style signaling to the accelerator - similar to HBM. There are clear trade-offs, though. HBF operates at microsecond-scale latencies. Accesses are page-based (kilobytes), and small random reads quickly degrade effective bandwidth. Write endurance is limited (on the order of 10⁵ cycles), making HBF unsuitable as a general-purpose cache or a write-heavy tier. For this reason, HBM continues to remain critical for KV cache, activations, and other latency-sensitive accesses. Some systems are tiering KV cache - keeping hot KV in HBM while spilling colder data into CPU-attached LPDDR over coherent high-speed interconnects. HBF fits naturally with read-mostly data: model weights and slowly changing retrieval corpora. Weights are large, read-bandwidth-hungry, and not written during inference. Increasing per-node weight capacity by an order of magnitude reduces the number of accelerators required to fit a model, directly impacting system scale, power, and network complexity. HBF has shifted from theory to a formal industry roadmap following the 2025 SanDisk and SK Hynix alliance, with standardization underway and sampling slated for late 2026, targeting a full market debut on inference platforms by early 2027. Kioxia is pursuing an incremental path, moving multi-terabyte flash closer to accelerators over PCIe 6.0 with aggressive prefetching. Different implementations, same direction: flash is being pulled into the memory hierarchy! HBF is a serious attempt to address the capacity bottleneck. The open questions are no longer about whether this makes sense, but how to architect systems around it - software scheduling of page-based reads to hide microsecond latencies, balancing between HBM/HBF for different inference workloads, reducing tail latency, etc. Overall, I am excited to see more innovation for the memory wall problem!

156

6 commenti

YellowDog.ai just set a 10x benchmark uplift in scale computing, delivering 40,000 tasks per second (TPS) and managing 100,000 compute nodes in the cloud. What's even more interesting, that's 2x IBM Symphony and opens an intriguing pathway for these until-know closed/captive systems.

11

On the Next Phase of AI Engineering Reading the recent memos from Satya Nadella, Sundar Pichai, and even the commentary emerging from OpenAI, a consistent theme stands out: the next phase of AI is about disciplined execution, real-world impact, and sustained urgency. Satya’s “Looking Ahead to 2026” underscores something fundamental—AI’s value will increasingly be defined by how well we apply it, not just how advanced the models become. Engineering excellence, thoughtful design, and a focus on outcomes will matter more than fascination with capability for capability’s sake. It’s a reminder to stay grounded: solve real problems, create tangible utility, and translate scientific breakthroughs into engineered impact. Sundar’s internal message adds another dimension: don’t slow down. With 2026 shaping up to be intensely competitive, maintaining urgency and clarity of focus becomes essential. His stance on investment—better to over-invest than under-invest—signals the scale of transformation still ahead. Even Sam Altman’s reflections point in a similar direction. As competition accelerates, the focus sharpens on applying what is core to the business and driving value through real products. Across all these perspectives, the narrative converges: AI engineering is entering a phase where usefulness, impact, and customer outcomes are the true differentiators. In this phase, AI engineering feels less like experimentation and more like responsibility—getting the fundamentals right, at scale. For those of us building in this space, these memos don’t just set direction—they shape mindset. They remind us that what ultimately matters is empowering customers to achieve more, and ensuring the work we do moves their business forward in meaningful ways. As 2026 begins, I’m excited about the opportunity to apply AI with intention and deliver real-world impact.

38

2 commenti

Just wrapped the 7th Perplexity AI Business Fellowship session featuring Ali Ghodsi, CEO of Databricks, in conversation with Aravind Srinivas. Ali's refreshing candor about AI's enterprise impact, dropping some uncomfortable truths about enterprise AI that every engineering leader needs to hear: - "The only reason the AI bubble hasn't burst is inference-time compute" - not actual breakthroughs - When Databricks plugged advanced reasoning models into enterprise applications? "Barely any boost." The problem isn't intelligence but context - is "C" California or Celsius? - Success formula: unique data + distribution channels. Nothing else matters. - AI revolutionizes coding, which represents just 20% of engineering time. The other 80%? Human meetings and alignment work which AI can't touch. - Biggest blocker? Not capabilities but evaluation frameworks. "We have no way to measure reliability" in specialized enterprise contexts. This mirrors what we're seeing in healthcare AI - where reliable assessment frameworks are critical when patient outcomes are on the line. As I noted in my analysis of BVP's Healthcare AI Index (https://lnkd.in/gy3P7zdU), success isn't about algorithms but workflow integration and evaluation rigor. For engineering teams, the path forward isn't chasing fancier models but solving the semantic understanding and evaluation challenges that remain stubbornly human. #EnterpriseAI #EngineeringLeadership #HealthcareAI

24

Engineering teams are turning AI into real results, not just experiments. Last month at the ELC Annual in SF, Redis Principal Field Engineer Josh R. joined Vinay Perneti from Augment Code, Anand K Sankaran from Workday, and Forrest Thomas from Garner Health for a fireside chat on where AI is truly adding value. Hear how they drove adoption across their teams, improved workflows, and made AI deliver real results: https://lnkd.in/gDAjZsDF

23

The SF blackout on Dec 20, 2025, was a real-world litmus test for autonomy: Waymo vehicles stalled at dark intersections, blocking traffic and forcing a service pause, while Tesla’s FSD navigated confidently—treating dead lights as four-way stops. This highlights why probabilistic AI triumphs over rigid, rules-based systems. Trained on vast unstructured real-world data, end-to-end neural nets reason probabilistically, adapting to chaos like humans do. Rules break in edge cases; probabilities generalize and thrive. AI’s human-like uncertainty handling is the key to robust intelligence in unpredictable environments. Thoughts on the future of AV tech? #AI #AutonomousDriving #Tesla #Waymo #MachineLearning https://lnkd.in/eVR74PbZ

3