Last night, we had a “standing room only” crowd at Michael Carducci’s presentation, Architecture Patterns for AI-Powered Applications, which was held jointly by Tampa Java User Group, Tampa Devs, and Tampa Bay Artificial Intelligence Meetup (which Anitra and I co-organize).
This article is a summary of the talk, complete with all the photos I took from the front row and afterparty.
The event was held at Kforce HQ, home of Tampa Bay’s meetup venue with the cushiest seats (full disclosure: I’m a Kforce consultant employee), and the food was provided by the cushiest NoSQL database platform, Couchbase!
Michael Carducci is many things: engaging speaker, funny guy, professional magician, and (of course) a software architect.
While he has extensive experience building systems for Very Big Organizations, the system-building journey he shared was a little more personal — it was about his SaaS CRM platform for a demographic he knows well: professional entertainers. He’s been maintaining it over the past 20 years, and it served as the primary example throughout his talk.
Michael’s central theme for his presentation was the gap between proof-of-concept AI implementations and production-ready systems, and it’s a bigger gap than you might initially think.
He emphasized that while adding basic AI functionality might take only 15 minutes to code, it’s a completely different thing to create a robust, secure, and cost-effective production system. That requires additional careful architectural consideration.
Here’s a quote to remember:
“architecture [is the] essence of the software; everything it can do beyond providing the defined features and functions.”
— “Mastering Software Architecture” by Michael Carducci
A good chunk of the talk was about “ilities” — non-functional requirements that become architecturally significant when integrating AI.
These “ilities” are…
- Cost – AI API costs can escalate quickly, especially as models chain together
- Accuracy – Dealing with hallucinations and non-deterministic outputs
- Security – Preventing prompt injection and model jailbreaking
- Privacy – Managing data leakage and training data concerns
- Latency & Throughput – Performance impacts of multiple model calls
- Observability – Monitoring what’s happening in AI interactions
- Simplicity / Complexity – Managing the increasing technical stack
And then he walked us through some patterns he encountered while building his application, starting with the “send an email” functionality:
The “send an email” function has an “make AI write the message for me” button, which necessitates an AI “guardrails” pattern:
And adding more AI features, such as having the AI-generated emails “sound” more like the user by having it review the user’s previous emails, called for using different architectural patterns.
And with more architectural patterns come different tradeoffs.
In the end, there was a progression of implementations from simple to increasingly complex. (It’s no wonder “on time, under budget” is considered a miracle these days)…
Stage 1: Basic Integration
- Simple pass-through to OpenAI API
- Minimal code (15 minutes to implement)
- Poor security, no observability, privacy risks
Stage 2: Adding Guardrails
- Input and output guardrails using additional LLMs
- Prompt templates to provide context
- Triple the API costs and latency
Stage 3: Personalization
- Adding user writing style examples
- Building data pipelines to extract relevant context
- Dealing with token optimization challenges
Stage 4: Advanced Approaches
- Fine-tuning models per customer
- Context caching strategies
- Hosting internal LLM services
- Full MLOps implementation
This led to Michael talking about doing architecture in the broader enterprise context:
- Organizations have fragmented information ecosystems
- Oragnizational data is spread across multiple systems after mergers and acquisitions
- Sophisticated information retrieval has to be implemented before AI can be effective
- “Garbage in, garbage out” still applies — in fact, even more so with AI
He detailed his experience building an 85-microservice pipeline for document processing:
- Choreographed approach: Microservices respond to events independently
- Benefits: Flexibility, easy to add new capabilities
- Challenges: No definitive end state, potential for infinite loops, ordering issues
- Alternative: Orchestrated approach with a mediator (more control but less flexibility)
He could’ve gone on for longer, but we were “at time,” so he wrapped up with some concepts worth our exploring afterwards:
- JSON-LD: JSON with Linked Data, providing context to structured data
- Schema.org: Standardized vocabulary for semantic meaning
- Graph RAG: Connecting LLMs directly to knowledge graphs
- Hypermedia APIs: Self-describing APIs that adapt without redeployment
He also talked about how models trained on JSON-LD can automatically understand and connect data using standardized vocabularies, enabling more sophisticated AI integrations.
What’s a summary of a talk without some takeaways? here are mine:
- Architecture is fundamentally about trade-offs! Every decision that strengthens one quality attribute weakens others; you need to decide which ones are important for the problems you’re trying to solve.
- Effective architects need breadth over depth. Instead of being “T-shaped,” which many people call the ideal “skill geometry” for individual developers, the architect needs to be more of a “broken comb.”
- AI integration is more than just functionality. It’s architecturally significant and requires careful planning
- Standards from the past are relevant again! Like Jason Voorhees, they keep coming back. Technologies like RDF and JSON-LD, once considered ahead of their time, are now crucial for AI.
- The chat interface is just the beginning! Yes, it’s the one everyone understands because it’s how the current wave of AI became popular, but serious AI integration requires thoughtful architectural patterns.
Here’s the summary of patterns Michael talked about:
- Prompt Template Pattern
- Guardrails Pattern
- Context-enrichment & Caching
- Composite Patterns
- Model Tuning
- Pipeline Pattern
- Encoder-decoder pattern
- Choreographed and Orchestrated Event-driven Patterns
- RAG
- Self-RAG
- Corrective-RAG
- Agentic RAG
- Agent-Ready APIs
And once the presentation was done, a number of us reconvened at Colony Grill, the nearby pizza and beer place, where we continued with conversations and card tricks.
My thanks to Michael Carducci for coming to Tampa, Tampa JUG and Ammar Yusuf for organizing, Hallie Stone and Couchbase for the food, Kforce for the space (and hey, for the job), and to everyone who attended for making the event so great!
