Audience Q&A on validation and small models

Integrating AI into Products and Teams ✨ — with Disheng Qiu (Translated)

Topic 6 of 6

Published Feb 9, 2024 · Clip 49:21–1:01:36 (12:15)

The host reads audience questions. On whether model validation should be done on the user side or translator side, Disheng argues the golden rule is to test with the final use case, since a universal model cannot fit all brands' tone and terminology—build a representative random test set in the real context. On opportunities for small companies and open-source communities given the costs of LLM training, he says startups should target niche, specific use cases with smaller constrained models fine-tuned at reasonable cost, citing Translated's compressed model that beat its larger standard model for latency-sensitive use cases (P99 below 200ms). On whether companies should keep humans in the loop and share benefits with data creators (like Midjourney vs illustrators), he says yes: enterprises need internal professional teams reviewing LLM output continuously, because an unreviewed GPT-4 support chat can embarrass the brand, and you must find issues before customers do.