Ain’t livestreaming grand? We had a few rough-around-the-edges moments in today’s episode of Ziti TV (I worked on the demo a *little* too late last night), but the important thing is: THE DEMO WORKED.
The demo built on the previous episode’s work where we took the OpenZiti Appetizer (a service that displayed whatever text you feed it on a public web page via a secure zero-trust OpenZiti overlay) and added a RoBERTa classifier to it to prevent users from entering offensive messages for the Appetizer to display. There are lots of people out there who seem to not have aged past 12, and that’s what this is supposed to handle.
RoBERTa classifiers are good at handling profanity and direct insults like “You are too stupid to understand this”. It *will* miss more subtle jabs like “Nobody wants you here and everyone knows it”, but try saying that to a cop who’s writing you a speeding ticket, and they will classify it very differently.
So in this week’s, the classifier keeps its job, but when it isn’t sure, it asks for a second opinion. LLM Gateway, the OpenZiti-based open source OpenAI-compatible gateway, decides which LLM gives that opinion: a small model on your own machine for everyday messages, and Claude for the subtle ones. You won’t change a line of the Appetizer’s code.
I did the demo without much chance for a rehearsal, and I’ll post a GitHUb repo with all the steps and material you need to try it out for yourself. But in the meantime, catch Clint and I work our way through the demo!