Expo Demonstration Soft Prompts for On-Device Content Moderation

We demonstrate the first on-device integration of a safety-aligned large language model (LLM) using soft prompt distillation, powered by our proposed TV-DiSP framework. Our system showcases how a mobile device can run a quantized LLM equipped with learned soft prompts to moderate harmful or toxic content in real-time. The demo highlights the difference in LLM outputs with and without our soft prompts when subjected to adversarial or unsafe inputs, enabling efficient and safe deployment of LLMs o