# Ramakrishnan Sivakumar **Principal ML Software Engineer at AMD - on-device AI** - Location: Seattle, Washington, USA - Email: ramkrishna2910@gmail.com - Website: https://ramkrishna.dev/ - LinkedIn: https://www.linkedin.com/in/ramakrishnansivakumar/ - GitHub: https://github.com/ramkrishna2910 - Medium: https://medium.com/@ramkrishna2910 - Google Scholar: https://scholar.google.com/citations?user=15TuaaoAAAAJ - X: https://twitter.com/ramkrishna2910 ## Summary I'm a machine learning systems engineer with over a decade of experience making AI faster on the hardware it runs on. My current focus is on-device AI - getting large language models to run efficiently on consumer hardware such as laptops and workstations with NPUs and integrated GPUs. I co-created and maintain Lemonade (5k+ GitHub stars, sponsored by AMD), an open-source local AI server that runs LLMs, speech-to-text, text-to-speech, and image generation fully locally with NPU and GPU acceleration, exposing OpenAI-, Anthropic-, and Ollama-compatible APIs. I previously co-created TurnkeyML (now in the ONNX organization) and GroqFlow. Across roles at Intel, Groq, and now AMD, I've shipped ML compilers, AI accelerator software, and developer tools spanning ONNX, ONNX Runtime, llama.cpp, and ROCm - and contributed over 2,000 models to the ONNX Model Zoo. I hold two granted US patents on thread scheduling and power management, publish research on AI accelerator evaluation, and am active in the ONNX community - presenting the relaunched ONNX Model Zoo and moderating the 2025 ONNX Community Meetup panel on interoperability and standardization in AI. ## Current Focus - **Lemonade** (Co-creator & maintainer): Open-source local AI server: LLMs, speech, and image generation on your own hardware, with NPU and GPU acceleration. https://github.com/lemonade-sdk/lemonade - **Tiered intelligence** (Building): Getting every request to the right tier of compute - NPU, GPU, or cloud - through policy-driven model routing: rule, classifier, semantic, and LLM-as-router. - **Upstream open source** (Contributor): Ongoing contributions across the local-AI stack: vLLM backends, llama.cpp enablement, and the ONNX ecosystem. ## Experience ### Principal ML Software Engineer - AMD May 2025 - Present · Seattle, WA - Technical lead for on-device AI developer experience: making large language models run efficiently on AMD consumer hardware (Ryzen AI NPUs and Radeon GPUs). - Maintainer of Lemonade, the open-source SDK for local LLM inference across ONNX Runtime, llama.cpp, and NPU-accelerated backends. - Drive open-source strategy and developer tooling that connects AMD silicon to the local-AI ecosystem. Technologies: Lemonade SDK, ONNX Runtime, llama.cpp, Ryzen AI NPU, ROCm, Python, C++ ### Staff ML Software Engineer - AMD Jan 2023 - May 2025 · Seattle, WA - Built developer tools for evaluating and accelerating ML models on AMD platforms, including the open-source TurnkeyML toolchain. - Contributed over 2,000 models to the ONNX Model Zoo, making it a practical benchmarking corpus for the ONNX ecosystem. - Worked across ONNX Runtime execution providers and quantization flows to bring LLM inference to consumer NPUs. Technologies: TurnkeyML, ONNX, ONNX Runtime, Ryzen AI, Python ### Senior ML Software Engineer - Groq Jan 2021 - Jan 2023 · Seattle, WA - Co-created GroqFlow, the automated tool flow that compiles PyTorch, TensorFlow, and ONNX models to the Groq LPU with a single function call. - Customer applications engineering for the Groq deterministic AI accelerator: ported and optimized customer models including graph neural networks. - Presented Groq's ONNX ecosystem work and GNN acceleration results at public developer events (GroqDay). Technologies: GroqFlow, Groq LPU, ONNX, PyTorch, Python ### Deep Learning Software Engineer - Intel Jan 2018 - Jan 2021 · Hillsboro, OR - Optimized transformer inference on Intel CPUs: co-authored the Microsoft Open Source blog work on accelerating BERT with the ONNX Runtime default execution provider. - Researched platform-aware resource allocation for inference on multi-accelerator edge devices (published at the EMC² workshop). - Filed multiple US patents on thread scheduling, power management, and data-flow prioritization arising from client-platform performance work. Technologies: ONNX Runtime, BERT, Intel CPU optimization, Edge inference, C++, Python ### Operating System Software Engineer - Intel Jun 2016 - Jan 2018 · Hillsboro, OR - Worked on operating system software for Intel client platforms: scheduling, power states, and platform performance characterization. - Foundation for later granted patent US 10,915,356 on augmenting thread scheduling with temporal characteristics. Technologies: Operating systems, Thread scheduling, Power management, C ## Education - Master of Science in Computer Engineering, University of North Carolina at Charlotte (2014 - 2016) ## Open-Source Projects ### Lemonade (Co-creator & maintainer) https://github.com/lemonade-sdk/lemonade Lemonade is an open-source local AI server (5k+ GitHub stars, Apache 2.0, sponsored by AMD) that runs large language models, speech-to-text, text-to-speech, and image generation fully locally, with NPU and GPU acceleration on Windows, macOS, and Linux. It exposes OpenAI-, Anthropic-, and Ollama-compatible APIs, manages GGUF and ONNX models, and integrates with Claude Code, Open WebUI, GitHub Copilot, n8n, and Dify. As maintainer I drive the roadmap - recent releases shipped the Lemonade Router (rule, classifier, semantic, and LLM-as-router policies), a server-side jobs engine, and MCP client hosting. ### TurnkeyML (Co-creator) https://github.com/onnx/turnkeyml TurnkeyML is an open-source toolchain (now part of the ONNX GitHub organization) for exploring accelerated AI: it exports, optimizes, benchmarks, and validates models across runtimes and hardware targets with a single CLI. The team used it to contribute over 2,000 models to the ONNX Model Zoo, turning the zoo into a practical, reproducible benchmarking corpus for the ecosystem. ### GroqFlow (Co-creator) https://github.com/groq/groqflow GroqFlow provides an automated tool flow that takes a PyTorch, TensorFlow, or ONNX model and produces a Groq LPU executable with a single function call - collapsing what was a multi-tool compiler pipeline into one developer-friendly step. It shipped as Groq's public on-ramp for customer models and set the developer-velocity pattern I later carried into TurnkeyML and Lemonade. ## Selected Writing - [Day 0 Support for Gemma 4 on AMD Processors and GPUs](https://www.amd.com/en/developer/resources/technical-articles/2026/day-0-support-for-gemma-4-on-amd-processors-and-gpus.html) - AMD Developer Blog, April 2026 - [Lemonade by AMD: A Unified API for Local AI Developers](https://www.amd.com/en/developer/resources/technical-articles/2026/lemonade-for-local-ai.html) - AMD Developer Blog, February 2026 - [Ryzen AI and Radeon are ready to run LLMs Locally with Lemonade Software](https://www.amd.com/en/developer/resources/technical-articles/2025/ryzen-ai-radeon-llms-with-lemonade.html) - AMD Developer Blog, November 2025 - [Run OpenAI's gpt-oss locally with Lemonade](https://lemonade-server.ai/news/gpt-oss.html) - Lemonade Blog, August 2025 - [Rethinking Local AI: Lemonade Server's Python Advantage](https://www.amd.com/en/developer/resources/technical-articles/2025/rethinking-local-ai-lemonade-servers-python-advantage.html) - AMD Developer Blog, July 2025 - [Local Tiny Agents: MCP Agents on Ryzen AI with Lemonade Server](https://www.amd.com/en/developer/resources/technical-articles/2025/local-tiny-agents--mcp-agents-on-ryzen-ai-with-lemonade-server.html) - AMD Developer Blog, June 2025 - [Unlocking a Wave of LLM Apps on Ryzen AI Through Lemonade Server](https://www.amd.com/en/developer/resources/technical-articles/unlocking-a-wave-of-llm-apps-on-ryzen-ai-through-lemonade-server.html) - AMD Developer Blog, April 2025 - [Why do companies Open Source Software?](https://ramkrishna2910.medium.com/why-do-companies-open-source-software-fc1b2c508206) - Medium, July 2024 - [GraphCast: A breakthrough in Weather Forecasting](https://ramkrishna2910.medium.com/graphcast-a-breakthrough-in-weather-forecasting-d70fae9ac365) - Medium, November 2023 - [BTS: The Marketing Masters in Disguise](https://ramkrishna2910.medium.com/bts-the-marketing-masters-in-disguise-54dea60492e4) - Medium, July 2023 - [The Power of Product Ecosystems](https://medium.com/design-bootcamp/the-power-of-product-ecosystems-3f6afd41ca51) - Medium (Design Bootcamp), February 2023 - [OpenAI's ChatGPT introduces itself](https://medium.com/@ramkrishna2910/openais-chatgpt-introduces-itself-f612e92d9be5) - Medium, December 2022 - [What, Why and How ONNX Script?](https://medium.com/@ramkrishna2910/what-why-and-how-onnx-script-74dd21ab396f) - Medium, October 2022 - [Developer Velocity You Can't Unsee: Introducing GroqFlow](https://groq.com/developer-velocity-you-cant-unsee-introducing-groqflow/) - Groq Blog, September 2022 - [The Promise of ONNX](https://medium.com/mlearning-ai/the-promise-of-onnx-d43c371f913d) - Medium (MLearning.ai), July 2022 - [Optimizing BERT for Intel CPUs using ONNX Runtime default execution provider](https://cloudblogs.microsoft.com/opensource/2021/03/01/optimizing-bert-model-for-intel-cpu-cores-using-onnx-runtime-default-execution-provider/) - Microsoft Open Source Blog, March 2021 ## Talks - [Panel: Agent Infrastructure Roundtable 2026](https://luma.com/9k0g4dqj) - Aramas AI & Berkeley Gateway Accelerator, Berkeley, CA, 2026 - [Vibe Coding with Local Models](https://www.amd.com/en/corporate/events/advancing-ai/sessions-catalog/vibe-coding-with-local-models.html) - AMD Advancing AI 2026, San Francisco & Shanghai, 2026 - [Build with local AI agents using OpenCode and Lemonade](https://build.microsoft.com/en-US/sessions/LABSP586-R1) - Microsoft Build 2026, San Francisco, 2026 - [Lemonade running local LLMs on AMD - CES 2026 booth demo](https://www.youtube.com/watch?v=-fscysymnWY) - CES 2026, AMD booth, 2026 - Moderator, [Panel: Interoperability and Standardization in AI](https://community.linuxfoundation.org/events/details/lfhq-lf-ai-data-foundation-presents-onnx-community-meet-up-1/) - ONNX Community Meetup 2025, Intel campus, Santa Clara, 2025 - [ONNX and TurnkeyML - AMD engineer interview](https://www.youtube.com/watch?v=iG6z5eaTD24) - AMD Developer Central, 2024 - [The AI Revenue Problem](https://open.spotify.com/episode/7wwkopVh05U6zcybmfcySu) - The Unenlightened Podcast, 2024 - [The AI Revolution](https://open.spotify.com/episode/35gaKkxhNQ0WO9VyWU4RJ9) - The Unenlightened Podcast, 2024 - [The Evolution of ONNX: Driving Interoperability in AI Through Open Standards](https://www.youtube.com/watch?v=N0wDg5oMHSA) - AI.dev 2023 (Linux Foundation), San Jose, 2023 - [The New ONNX Model Zoo](https://www.youtube.com/watch?v=5irQREPHm1Q) - ONNX Community Meetup 2023, NVIDIA HQ, 2023 - The Power of Graph Neural Networks - GroqDay, 2023 - The World of Ads and its evolving landscape - Industry talk, 2023 - The ONNX Ecosystem - Groq developer session, 2022 ## Patents - [Hardware and software coordinated cost-aware low power state selection](https://patents.google.com/patent/US12141015B2) - US 12,141,015 (granted), Intel Corporation, 2024 - [Technology to augment thread scheduling with temporal characteristics](https://patents.google.com/patent/US10915356B2) - US 10,915,356 (granted), Intel Corporation, 2021 ## Publications - [Exploring the Use of Dataflow Architectures for Graph Neural Network Workloads](https://link.springer.com/chapter/10.1007/978-3-031-40843-4_48) - ISC High Performance 2023 Workshops (LNCS 13999), 2023 - [A Comprehensive Evaluation of Novel AI Accelerators for Deep Learning Workloads](https://ieeexplore.ieee.org/abstract/document/10024028) - IEEE/ACM PMBS Workshop at SC22, 2022 - [Platform-Aware Resource Allocation for Inference on Multi-Accelerator Edge Devices](https://www.emc2-ai.org/assets/docs/virtual-20/emc2-virtual20-paper-11.pdf) - EMC² Workshop (Energy Efficient Machine Learning and Cognitive Computing), 2020 - [Design and implementation of a semi-autonomous waste segregation robot](https://ieeexplore.ieee.org/abstract/document/7506679) - IEEE SoutheastCon, 2016 ## Skills ### On-device & local inference - Local LLM serving (Lemonade SDK) - llama.cpp - ONNX Runtime (CPU, GPU, NPU execution providers) - AMD Ryzen AI NPU - Quantization for consumer hardware ### ML compilers & runtimes - ONNX / ONNX Script / ONNX Model Zoo - ML compiler toolchains (GroqFlow, TurnkeyML) - ROCm - Deep learning graph optimization ### AI accelerators - Accelerator software stacks (AMD, Groq LPU, Intel) - Benchmarking novel AI accelerators (Groq, Cerebras, SambaNova, Graphcore) - Platform-aware inference resource allocation ### Engineering leadership - Open-source project maintainership - Developer experience & product ecosystems - Technical talks and community building --- This document is generated from the same source as https://ramkrishna.dev/ - see also [llms.txt](https://ramkrishna.dev/llms.txt).