👋 Welcome
Hey! I’m Baptiste, a Machine Learning Engineer at Hugging Face 🤗 I made this blog to share my AI adventures and weird thoughts. Hope you find something useful here! 😵💫
🚀 My Work on Accelerating LLM Inference with TGI on Intel Gaudi
I’m thrilled to share my latest project: adding native integration of Intel Gaudi hardware support directly into Text Generation Inference (TGI), the production-ready serving solution for Large Language Models (LLMs). This integration brings the power of Intel’s specialized AI accelerators to the high-performance inference stack, enabling more deployment options for the open-source AI community 🎉 ✨ What I’ve Accomplished I’ve fully integrated Gaudi support into TGI’s main codebase in PR #3091. Previously, we had to maintain a separate fork for Gaudi devices at tgi-gaudi. This was cumbersome for users and prevented supporting the latest TGI features at launch. By leveraging the new TGI multi-backend architecture, I’ve made it possible to support Gaudi directly on TGI – no more dealing with a custom repository 🙌 ...