Enterprise AI Assistant: Open-Source Stack, Self-Deployed Models, On-Prem GPUs

This workshop presents an end-to-end approach to building an internal enterprise AI platform on dedicated on-premise hardware as a secure and cost-effective alternative to commercial SaaS licenses. Self-Hosted Infrastructure & GPU Serving focuses on running LLMs locally. It covers hardware sizing, local Nvidia workstation setups, and practical lessons from building a custom GPU cluster, including inference serving engines, quantization, memory management, and deployment realities. Open WebUI & LiteLLM Platform covers the application and integration layer. It demonstrates using LiteLLM as an API gateway for routing, governance, and access control, combined with Open WebUI as the user interface, enriched with Model Context Protocol (MCP) integrations into enterprise tools (Office, internal systems like P2K). We will cover the system architecture, hardware trade-offs, and operational lessons from deployment, followed by a live demonstration of prompt execution from the user interface down to local silicon and internal APIs.