Global Nerdy

Last week, Anitra and I attended both the Dev/Nexus conference and its companion conference, Advantage, an AI conference for CTOs, CIOs, VPs of Engineering, and other technical lead-types, which took place the day before Dev/Nexus. My thanks to Pratik Patel for the guest passes to both conferences!

I took copious notes and photos of all the Advantage talks and will be posting summaries here. This set of notes is from the third talk, Language Stacks and Gen AI, presented by Rod Johnson.

Here’s Rod’s bio:

Rod is a developer, author, investor and entrepreneur. He has authored several best-selling books on Java EE. He is the creator of the Spring Framework and was cofounder and CEO of SpringSource. He has served on the board of Elastic, Neo Technologies, Apollo, Lightbend and several other successful companies. He is presently developing a structured RAG system using Spring and Kotlin.

And here’s the abstract of his talk:

Python is the language of data science and dominant in AI research. However, it is not the language of enterprise apps, and there are good reasons for this. In this session, Rod will discuss when to use what language and stack for AI success in enterprise. He’ll discuss the key adjacencies for success: LLMs, existing data and business logic, and how to choose what language, stack and framework for a particular problem.

My notes from Rod’s talk are below.

Your existing enterprise systems are an asset, not a liability

Rod opened with something that probably felt like a relief to many in the room: a clear-eyed argument that the overwhelming pace of AI change is not a reason to abandon what your organization has already built. Enterprise systems represent years of accumulated business logic, domain knowledge, and battle-tested reliability. These things change slowly, and in this case, the cliche is true: that’s not a  bug, but a feature. The pressure to throw out existing applications and start fresh with AI-native rewrites is, in Rod’s view, is not just misguided, but reckless.

He was equally direct about the organizational risk of letting AI enthusiasm displace experienced people. Every major technology wave produces a class of self-declared experts who rush in and crowd out the engineers who actually understand the business. Domain expertise doesn’t get replaced by a new framework. Instead, it gets more valuable, because it’s the thing that makes AI systems accurate and useful rather than fluently wrong.

The message to leaders was clear: protect your people, and make sure your AI strategy grows out of your existing institutional knowledge rather than treating it as an obstacle.

Personal assistants and business processes are fundamentally different. Stop conflating them.

One of the sharpest distinctions in Rod’s talk was between AI as a personal productivity tool and AI embedded in enterprise business processes. The personal assistant category (for example: chatbots, coding agents, tools like Cursor) operates under forgiving conditions. If a coding agent produces bad output, a developer catches it before it reaches production. The feedback loop is tight, human oversight is immediate, and the cost of failure is manageable. This is why maximizing agent autonomy makes sense in that context.

Business processes are an entirely different environment. Rod pointed to the Air Canada chatbot case, where the airline told a customer it would honor a discounted bereavement fare and then tried to disclaim responsibility when the customer held them to it. Unlike a coding error that gets caught in review, a business process error engages real customers, real employees, and real legal and financial consequences. You can’t roll back a workflow the way you can roll back a pull request. The asymmetry between these two domains is enormous, yet most of the noise driving enterprise AI strategy comes from the personal assistant space, where demos are impressive and the failure modes are invisible.

Rod is clearly frustrated with this conflation, and you should be too. The loudest voices in the generative AI conversation are the ones driving media coverage and executive attention, and they’re overwhelmingly people with no background in or interest in enterprise software. Leaders who let those voices set their enterprise AI agenda are optimizing for demo impressiveness rather than production reliability, and that’s a recipe for expensive disappointment.

Structure (almost always) beats natural language

Perhaps the most technically counterintuitive point in Rod’s talk was his argument that interacting with LLMs in natural language is often the wrong approach. Yes, LLMs are trained on vast amounts of natural language text, but the underlying Transformer architecture is fundamentally about predicting tokens. It’s not inherently about language at all. The seductive thing about natural language interfaces is that you can demo them impressively in minutes. The unsettling thing is that natural language is ambiguous, extremely difficult to test, and essentially opaque when something goes wrong.

Rod’s alternative is to structure your interactions with LLMs as much as possible: structured inputs, structured outputs, and as little free-form natural language in the critical paths as you can manage. His thought experiment about what a bank knows about its customers illustrated the point neatly. The vast majority of the high-value data a bank holds — transactions, account balances, product relationships — is already highly structured. The fringe cases that exist in text (notes from a branch visit, a customer service transcript) are real but marginal. Adding generative AI to that environment should leverage the structure that’s already there, not dissolve it into a sea of markdown and free text.

The practical consequences of over-relying on natural language are significant. Systems built around unstructured text accumulate context rapidly, which drives token counts (and therefore costs) through the roof. They become increasingly unpredictable as that context grows, and when they produce wrong outputs, there’s no clean way to explain or audit what went wrong. Rod’s point, reinforced by his analysis of OpenAI’s Operator product, is that even sophisticated AI systems hit a hard ceiling when they’re built on a foundation of loose text rather than structured data and deterministic logic.

Your language stack probably shouldn’t change, but your thinking about AI layers should

Rod was characteristically direct on the language debate that consumes a lot of oxygen in AI developer communities: Python is not magical for building enterprise AI agents, and the fact that most academic AI research is published in Python is not a reason for a Java or C# shop to rewrite everything. There are reasons your enterprise applications were written in the languages they were written in: stability, ecosystem maturity, existing tooling,  and team expertise, and those reasons haven’t changed. What sits in the generative AI layer is substantially shallower than your core application logic, and the risk-reward calculation for rewriting those core systems in a trendier language is deeply unfavorable.

That said, Rod drew a reasonable distinction: Python genuinely does have advantages for certain tasks like document processing, model fine-tuning, and data ingestion pipelines, where the research community’s tooling is simply more mature. The error isn’t using Python for those things. It’s letting data science people with a Python background architect the entire enterprise AI strategy, because data science and enterprise AI application development require genuinely different skills. Conflating them leads to frameworks that are academically interesting but operationally fragile when exposed to real enterprise requirements around security, observability, testability, and integration.

The practical implication for enterprise leaders is that you need a good agent framework. Rod’s example was Embabel, a framework his company developed. Agent frameworks should feel like a natural extension of your existing stack. It should play nicely with Spring, respect your existing domain model, integrate with your existing observability tooling, and support unit testing at every level. You shouldn’t have to introduce an entirely new operational paradigm just to add generative AI capabilities. Adjacency to your existing systems is where the value gets unlocked, and any framework that treats your existing applications as irrelevant legacy to be worked around is solving the wrong problem.

Last week, Anitra and I attended both the Dev/Nexus conference and its companion conference, Advantage, an AI conference for CTOs, CIOs, VPs of Engineering, and other technical lead-types, which took place the day before Dev/Nexus. My thanks to Pratik Patel for the guest passes to both conferences!

I took copious notes and photos of all the Advantage talks and will be posting summaries here. This set of notes is from the second talk, AI Architecture for Tech Leaders: Building Blocks for AI Applications, presented by Pratik Patel.

Here’s Pratik’s bio:

Pratik Patel is VP of Developer Relations at Azul Systems. An all around software and hardware nerd with experience in the healthcare, telecom, financial services, and startup sectors.

And here’s the abstract of his talk:

The AI space is moving incredibly fast, it seems new methodologies and technologies are coming every week. How’s a technology leader (whether your a VP Engineering, Software Dev Manager or Team Lead) supposed to understand what are the true building blocks for this new class of applications. How do you scope an AI development project, both in terms of developer time and cloud & AI infrastructure? Should you buy AI hardware or pay for API access to OpenAI, Claude, Gemini, etc? Do you have sensitive information that you want to keep from leaking out to an external LLM provider? In this session, we’ll tackle these issues and also discuss the evolution of applications and the difference between: existing applications that have added AI capability as an accessory this new class of applications that are built with AI in mind from the start This session is intended to be interactive – I’ll start by laying the foundation for building AI applications today, and we’ll discuss the experiences of the tech leaders in the room so everyone can share and learn from each other.

My notes from Pratik’s talk are below.

Note: You can find a more developer-focused version of this talk in an earlier posting, from when Pratik came to Tampa to deliver this talk for the Tampa Bay Java User Group and Tampa Bay AI Meetup.

Skate to where the puck Is going

Pratik opened with the AI version of the Wayne Gretzky line: don’t build for where AI is today, build for where it will be in six to twelve months. The pressure many tech leaders currently feel to add AI to everything so the organization can say it’s doing AI is producing a wave of surface-level implementations that won’t hold up. Sprinkling a chatbot on top of an existing application is not a strategy, but a reaction.

The analogy Pratik kept returning to was the shift from manual, infrequent deployments to cloud-native, continuously-delivered software. That transition wasn’t just about adopting new tools. It required a fundamental rethinking of how teams design, build, and release software. Organizations that made that leap early didn’t just move faster; they built a compounding capability advantage. Pratik’s argument is that we’re at a similar inflection point with AI, and the leaders who recognize it now will be the ones whose systems look prescient rather than antiquated in two years.

AI-native vs. AI-augmented: A critical distinction

The conceptual core of Pratik’s talk is the difference between bolting AI onto an existing application and building an AI-native one from the ground up. An AI-native application doesn’t just use AI as a feature, and is organized around AI’s ability to learn, adapt, and act autonomously. Those three verbs matter. Most of what organizations are building today qualifies as AI-augmented at best: an agent that can act, but that doesn’t genuinely learn from interactions or adapt its behavior without human intervention.

Pratik illustrated this with a content management system example. A traditional CMS requires humans to manually tag articles. An AI-native CMS handles tagging automatically, continuously improves based on feedback, and integrates that intelligence into the editorial workflow without requiring a separate AI plugin to be configured and maintained. The business value isn’t just efficiency, but that the system gets better over time in a way that a bolted-on tool never will.

His hotel booking example pushed the concept further. A truly AI-native booking platform wouldn’t just filter hotels by amenities; it would learn individual user preferences from past behavior, weight them against contextual signals, and surface recommendations that reflect both explicit preferences and inferred ones. More importantly, it would adapt its pricing and inventory strategies automatically in response to real-world events (examples: a competitor hotel going offline for renovations, a major sporting event driving demand) without requiring a human to catch the signal and manually adjust rates.

Foundational data strategy is the real competitive moat

Pratik was clear that all the architectural sophistication in the world collapses without a serious approach to data. The core question every leader should be asking is “Is the data your organization holds actually usable by an AI system?” Not just stored somewhere, but clean, current, structured in ways that a model can reason about, and governed in ways that ensure its quality over time. Most companies, when they’re honest, have to answer that question with “not really.”

The cultural shift required here is moving from a “collect it and figure it out later” mentality to a data-first culture where data quality is treated as a continuous engineering concern, not a cleanup project. Pratik framed this as the AI equivalent of the DevOps automation mindset: just as teams had to change how they thought about deployment.

Instead of thinking of deployment as a periodic event but as a constant, automated process, teams now need to think about data not as a byproduct of operations but as the fuel that makes AI systems defensible.

Unstructured data adds another layer of complexity. RAG  is the most common approach to incorporating things like PDFs and documents into an AI system, but Pratik was careful to note that “just do RAG” massively undersells the challenge. He’s catalogued over 36 distinct RAG implementation techniques, each with different trade-offs around chunking strategies, retrieval quality, and error rates. Leaders who treat RAG as a checkbox rather than an engineering discipline will find their AI systems returning confidently wrong answers from their own documents.

The AI-native development lifecycle

Building AI-native systems requires updating how teams think about the software development lifecycle itself. Pratik drew a direct parallel to the DevOps transformation: just as continuous integration and deployment automated away the pain of manual releases, AI-native development requires building automation into the AI feedback loop,  from code generation assistance to automated testing of non-deterministic outputs.

The trickiest part of this is monitoring. Traditional software testing assumes deterministic behavior: you give it inputs, you check the outputs against known values. AI systems don’t work that way.

Pratik described two approaches that are gaining traction:

1. Human-in-the-loop feedback: the five-star rating prompt that many AI products now show after a response, which feeds real quality signals back into the system.
2. “LLM as judge”: using a second AI model (potentially a smaller, cheaper one) to evaluate the outputs of your primary model, essentially automating quality checks at scale.

The practical implication for tech leaders is that shipping an AI-native application is not a one-time event followed by monitoring dashboards. It requires building the infrastructure for continuous retraining, output validation, and drift detection from day one. The underlying model, the data it draws from, and the world it’s reasoning about all change over time. A system that doesn’t account for that will quietly degrade in ways that are hard to detect until users start complaining.