Category: Artificial Intelligence

Last week, Anitra and I attended both the Dev/Nexus conference and its companion conference, Advantage, an AI conference for CTOs, CIOs, VPs of Engineering, and other technical lead-types, which took place the day before Dev/Nexus. My thanks to Pratik Patel for the guest passes to both conferences!

I took copious notes and photos of all the Advantage talks and will be posting summaries here. This set of notes is from the fourth talk, Shift to Agentic Software Engineering, presented by Dave Parry.

Here’s Dave’s bio:

David Parry is an accomplished Director of Architecture with over 20 years of experience in Software Development. It all began in 1996 when he discovered the fascinating world of programming, with a particular focus on Java applets. Throughout his illustrious career, David Parry has been involved in various noteworthy projects. He has successfully built and implemented content management systems for a wide range of clients, including the esteemed Johny Walker and its renowned keepwalking.com. Additionally, as a consultant at a Big 4 firm, David played a pivotal role in solving critical issues for numerous customers, demonstrating his expertise in handling complex and high-traffic web platforms. Never one to shy away from innovation, David Parry has expanded his skills to work on cutting-edge technologies such as mobile and embedded Android TV systems. Leveraging his expertise, he has delivered top-notch streaming services to customers, ensuring they have an exceptional viewing experience. Currently, David holds the position of Developer Advocate and Consultant overseeing strategic planning and execution of architectural designs for customers. With a deep understanding of software development principles and extensive experience in Java programming, he excels at providing valuable insights and guidance to his team. Having witnessed the evolution of Java development from its early days to its current state, David Parry’s wealth of experience and strategic perspective, combined with his consulting work at a Big 4 firm, make him an invaluable asset in any project or organization he is a part of.

And here’s the abstract of his talk:

AI is redefining how engineering organizations operate, shifting from traditional development to agentic development, where intelligent, context-aware agents partner with teams to drive measurable business outcomes. This presentation gives leaders a clear framework for understanding how agentic development improves cycle time, reduces operational risk, enhances quality, and scales organizational capacity without adding headcount. We will examine how to move beyond pilots, achieve meaningful adoption, embed governance and security controls, and connect engineering effort directly to enterprise KPIs. Leaders will leave with a strategic roadmap for guiding their organizations through this transformation with clarity, confidence, and control.

My notes from Dave’s talk are below.

The shift from AI-assisted to agentic is real, and most organizations aren’t ready

Dave opened by drawing a line between two distinct eras of AI in software development. The first era, the era of AI-assisted coding/the GitHub Copilot model, still has a human in the loop at every step. A developer reviews suggestions, accepts or rejects them, and retains full decision-making authority. This is the model most development teams have actually adopted, and it‘s valuable. The second era, agentic software engineering, is something categorically different: autonomous systems that execute multi-step workflows without continuous human supervision.

Dave was candid that most organizations are still figuring out how to use AI-assisted tools well, even as the industry conversation has moved on to agents. The gap between where the hype is and where most teams actually are is significant, and leaders who try to leapfrog directly to full autonomy without establishing the right foundations tend to end up with agents that are expensive, unpredictable, and politically toxic inside the engineering organization. The smarter path, in Dave‘s experience, is to build the scaffolding — governance, measurement, structured experimentation — before letting agents loose on anything consequential.

Governance can‘t be bolted on after the fact

The governance message in Dave‘s talk was clear: security and access controls must be architected into agentic systems from the beginning, not added as an afterthought once the agent is already running. He illustrated this with a client story about a company whose repositories were so strictly siloed that individual developers weren‘t even allowed to know other repos existed, let alone access them. An agent given broad permissions in that environment would immediately violate carefully constructed security boundaries that humans had been respecting for years, simply because nobody thought to encode those constraints into the agent‘s operating parameters.

The practical implication is that every constraint your human engineers operate under (such as access controls, data isolation, permission scoping) needs to be explicitly defined for any agent working in the same environment. Agents don‘t have professional judgment or social awareness; they will access whatever they‘re technically permitted to access. If you onboard a new human developer, you scope their access carefully before they write a single line of code. Agents require the same rigor. Dave‘s recommendation was to look for frameworks that make these governance constraints first-class concepts rather than optional configurations, and to be deeply skeptical of any agentic solution that treats security as something you layer on later.

Enterprise-readiness also extends to the technology choices themselves. Dave pushed back against agentic frameworks built in languages or runtimes that don‘t fit naturally into enterprise operational environments. A security team asked to approve an agent that spins up an npx process that re-downloads dependencies on every run is going to say no…and they should! The same agent behavior built on Spring Boot, running in a container with Prometheus observability already wired in, is a fundamentally different conversation.

Measure everything! Agents aren’t self-evidently valuable

One of Dave‘s most pragmatic points was that the business case for any given agent needs to be proven, not assumed. The pressure from above to “do AI” is real, but implementing an agent that costs more in compute and maintenance than it would cost a developer to do the same task manually is not a win — it‘s a liability that will eventually get noticed and used to discredit the entire program. Leaders who can‘t quantify what their agents are actually delivering are in a precarious position when budget scrutiny arrives.

His recommendation was to tie every agent deployment to concrete, measurable KPIs from the start. For a PR risk agent, the relevant metrics might include change failure rate, time to production, and whether bug rates are actually going down or inadvertently going up as junior developers blindly accept AI suggestions. The five-star anecdote was a useful cautionary note: some teams have discovered that their agents were actively introducing more defects than they prevented, precisely because they hadn‘t built in the measurement infrastructure to detect it early.

Dave also pushed back against the proof-of-concept mentality that treats agent work as inherently experimental. The POC era, in his view, is over. Organizations that frame every agent initiative as “let‘s see if this works” create the conditions for naysayers to kill it at the first sign of friction. His preferred framing is to pick a small, low-risk pilot, commit to shipping it to production, measure it rigorously, and use that concrete success to build momentum for the next one. Owning the conversation with data is the only reliable way to keep agentic programs alive long enough to deliver real compounding value.

Bring your existing developers into the agentic transition; don‘t route around them

A consistent thread throughout Dave‘s talk was that agentic AI is not a replacement for experienced engineers, but an amplifier of their knowledge. That amplification only works if those engineers are inside the tent. Developers who feel threatened by agents will find reasons for them to fail, and frankly, they‘ll often be right, because agents built without deep domain knowledge embedded in their prompts and tools tend to produce plausible-looking but subtly wrong outputs. The engineers who know where the bodies are buried in your codebase are exactly the people who should be shaping how your agents operate.

Dave‘s specific recommendation was that when outside expertise comes in to help stand up an agentic program, that expertise should be focused on upskilling the existing team rather than doing the work for them. An external consultant who delivers a finished agent and walks away leaves the organization with something it doesn‘t fully understand and can‘t maintain or evolve. An expert who works alongside the existing team, transfers knowledge, and helps them build the verification and governance capabilities they need to operate agents independently is creating something durable.

Dave made the point that custom MCP servers are one of the highest-leverage things an organization‘s own developers can build, because that‘s where domain-specific knowledge gets embedded in a form that agents can reliably use. A generic MCP that connects to a database and lets the LLM figure out the schema from scratch on every query is both expensive in tokens and fragile in output. A purpose-built MCP that encodes exactly what that database contains, how to query it correctly, and what the results mean — written by developers who actually know the system — is the kind of deterministic grounding that makes agentic systems genuinely trustworthy in production.

Last week, Anitra and I attended both the Dev/Nexus conference and its companion conference, Advantage, an AI conference for CTOs, CIOs, VPs of Engineering, and other technical lead-types, which took place the day before Dev/Nexus. My thanks to Pratik Patel for the guest passes to both conferences!

I took copious notes and photos of all the Advantage talks and will be posting summaries here. This set of notes is from the third talk, Language Stacks and Gen AI, presented by Rod Johnson.

Here’s Rod’s bio:

Rod is a developer, author, investor and entrepreneur. He has authored several best-selling books on Java EE. He is the creator of the Spring Framework and was cofounder and CEO of SpringSource. He has served on the board of Elastic, Neo Technologies, Apollo, Lightbend and several other successful companies. He is presently developing a structured RAG system using Spring and Kotlin.

And here’s the abstract of his talk:

Python is the language of data science and dominant in AI research. However, it is not the language of enterprise apps, and there are good reasons for this. In this session, Rod will discuss when to use what language and stack for AI success in enterprise. He’ll discuss the key adjacencies for success: LLMs, existing data and business logic, and how to choose what language, stack and framework for a particular problem.

My notes from Rod’s talk are below.

Your existing enterprise systems are an asset, not a liability

Rod opened with something that probably felt like a relief to many in the room: a clear-eyed argument that the overwhelming pace of AI change is not a reason to abandon what your organization has already built. Enterprise systems represent years of accumulated business logic, domain knowledge, and battle-tested reliability. These things change slowly, and in this case, the cliche is true: that’s not a  bug, but a feature. The pressure to throw out existing applications and start fresh with AI-native rewrites is, in Rod’s view, is not just misguided, but reckless.

He was equally direct about the organizational risk of letting AI enthusiasm displace experienced people. Every major technology wave produces a class of self-declared experts who rush in and crowd out the engineers who actually understand the business. Domain expertise doesn’t get replaced by a new framework. Instead, it gets more valuable, because it’s the thing that makes AI systems accurate and useful rather than fluently wrong.

The message to leaders was clear: protect your people, and make sure your AI strategy grows out of your existing institutional knowledge rather than treating it as an obstacle.

Personal assistants and business processes are fundamentally different. Stop conflating them.

One of the sharpest distinctions in Rod’s talk was between AI as a personal productivity tool and AI embedded in enterprise business processes. The personal assistant category (for example: chatbots, coding agents, tools like Cursor) operates under forgiving conditions. If a coding agent produces bad output, a developer catches it before it reaches production. The feedback loop is tight, human oversight is immediate, and the cost of failure is manageable. This is why maximizing agent autonomy makes sense in that context.

Business processes are an entirely different environment. Rod pointed to the Air Canada chatbot case, where the airline told a customer it would honor a discounted bereavement fare and then tried to disclaim responsibility when the customer held them to it. Unlike a coding error that gets caught in review, a business process error engages real customers, real employees, and real legal and financial consequences. You can’t roll back a workflow the way you can roll back a pull request. The asymmetry between these two domains is enormous, yet most of the noise driving enterprise AI strategy comes from the personal assistant space, where demos are impressive and the failure modes are invisible.

Rod is clearly frustrated with this conflation, and you should be too. The loudest voices in the generative AI conversation are the ones driving media coverage and executive attention, and they’re overwhelmingly people with no background in or interest in enterprise software. Leaders who let those voices set their enterprise AI agenda are optimizing for demo impressiveness rather than production reliability, and that’s a recipe for expensive disappointment.

Structure (almost always) beats natural language

Perhaps the most technically counterintuitive point in Rod’s talk was his argument that interacting with LLMs in natural language is often the wrong approach. Yes, LLMs are trained on vast amounts of natural language text, but the underlying Transformer architecture is fundamentally about predicting tokens. It’s not inherently about language at all. The seductive thing about natural language interfaces is that you can demo them impressively in minutes. The unsettling thing is that natural language is ambiguous, extremely difficult to test, and essentially opaque when something goes wrong.

Rod’s alternative is to structure your interactions with LLMs as much as possible: structured inputs, structured outputs, and as little free-form natural language in the critical paths as you can manage. His thought experiment about what a bank knows about its customers illustrated the point neatly. The vast majority of the high-value data a bank holds — transactions, account balances, product relationships — is already highly structured. The fringe cases that exist in text (notes from a branch visit, a customer service transcript) are real but marginal. Adding generative AI to that environment should leverage the structure that’s already there, not dissolve it into a sea of markdown and free text.

The practical consequences of over-relying on natural language are significant. Systems built around unstructured text accumulate context rapidly, which drives token counts (and therefore costs) through the roof. They become increasingly unpredictable as that context grows, and when they produce wrong outputs, there’s no clean way to explain or audit what went wrong. Rod’s point, reinforced by his analysis of OpenAI’s Operator product, is that even sophisticated AI systems hit a hard ceiling when they’re built on a foundation of loose text rather than structured data and deterministic logic.

Your language stack probably shouldn’t change, but your thinking about AI layers should

Rod was characteristically direct on the language debate that consumes a lot of oxygen in AI developer communities: Python is not magical for building enterprise AI agents, and the fact that most academic AI research is published in Python is not a reason for a Java or C# shop to rewrite everything. There are reasons your enterprise applications were written in the languages they were written in: stability, ecosystem maturity, existing tooling,  and team expertise, and those reasons haven’t changed. What sits in the generative AI layer is substantially shallower than your core application logic, and the risk-reward calculation for rewriting those core systems in a trendier language is deeply unfavorable.

That said, Rod drew a reasonable distinction: Python genuinely does have advantages for certain tasks like document processing, model fine-tuning, and data ingestion pipelines, where the research community’s tooling is simply more mature. The error isn’t using Python for those things. It’s letting data science people with a Python background architect the entire enterprise AI strategy, because data science and enterprise AI application development require genuinely different skills. Conflating them leads to frameworks that are academically interesting but operationally fragile when exposed to real enterprise requirements around security, observability, testability, and integration.

The practical implication for enterprise leaders is that you need a good agent framework. Rod’s example was Embabel, a framework his company developed. Agent frameworks should feel like a natural extension of your existing stack. It should play nicely with Spring, respect your existing domain model, integrate with your existing observability tooling, and support unit testing at every level. You shouldn’t have to introduce an entirely new operational paradigm just to add generative AI capabilities. Adjacency to your existing systems is where the value gets unlocked, and any framework that treats your existing applications as irrelevant legacy to be worked around is solving the wrong problem.

Last week, Anitra and I attended both the Dev/Nexus conference and its companion conference, Advantage, an AI conference for CTOs, CIOs, VPs of Engineering, and other technical lead-types, which took place the day before Dev/Nexus. My thanks to Pratik Patel for the guest passes to both conferences!

I took copious notes and photos of all the Advantage talks and will be posting summaries here. This set of notes is from the second talk, AI Architecture for Tech Leaders: Building Blocks for AI Applications, presented by Pratik Patel.

Here’s Pratik’s bio:

Pratik Patel is VP of Developer Relations at Azul Systems. An all around software and hardware nerd with experience in the healthcare, telecom, financial services, and startup sectors.

And here’s the abstract of his talk:

The AI space is moving incredibly fast, it seems new methodologies and technologies are coming every week. How’s a technology leader (whether your a VP Engineering, Software Dev Manager or Team Lead) supposed to understand what are the true building blocks for this new class of applications. How do you scope an AI development project, both in terms of developer time and cloud & AI infrastructure? Should you buy AI hardware or pay for API access to OpenAI, Claude, Gemini, etc? Do you have sensitive information that you want to keep from leaking out to an external LLM provider? In this session, we’ll tackle these issues and also discuss the evolution of applications and the difference between: existing applications that have added AI capability as an accessory this new class of applications that are built with AI in mind from the start This session is intended to be interactive – I’ll start by laying the foundation for building AI applications today, and we’ll discuss the experiences of the tech leaders in the room so everyone can share and learn from each other.

My notes from Pratik’s talk are below.

Note: You can find a more developer-focused version of this talk in an earlier posting, from when Pratik came to Tampa to deliver this talk for the Tampa Bay Java User Group and Tampa Bay AI Meetup.

Skate to where the puck Is going

Pratik opened with the AI version of the Wayne Gretzky line: don’t build for where AI is today, build for where it will be in six to twelve months. The pressure many tech leaders currently feel to add AI to everything so the organization can say it’s doing AI is producing a wave of surface-level implementations that won’t hold up. Sprinkling a chatbot on top of an existing application is not a strategy, but a reaction.

The analogy Pratik kept returning to was the shift from manual, infrequent deployments to cloud-native, continuously-delivered software. That transition wasn’t just about adopting new tools. It required a fundamental rethinking of how teams design, build, and release software. Organizations that made that leap early didn’t just move faster; they built a compounding capability advantage. Pratik’s argument is that we’re at a similar inflection point with AI, and the leaders who recognize it now will be the ones whose systems look prescient rather than antiquated in two years.

AI-native vs. AI-augmented: A critical distinction

The conceptual core of Pratik’s talk is the difference between bolting AI onto an existing application and building an AI-native one from the ground up. An AI-native application doesn’t just use AI as a feature, and is organized around AI’s ability to learn, adapt, and act autonomously. Those three verbs matter. Most of what organizations are building today qualifies as AI-augmented at best: an agent that can act, but that doesn’t genuinely learn from interactions or adapt its behavior without human intervention.

Pratik illustrated this with a content management system example. A traditional CMS requires humans to manually tag articles. An AI-native CMS handles tagging automatically, continuously improves based on feedback, and integrates that intelligence into the editorial workflow without requiring a separate AI plugin to be configured and maintained. The business value isn’t just efficiency, but that the system gets better over time in a way that a bolted-on tool never will.

His hotel booking example pushed the concept further. A truly AI-native booking platform wouldn’t just filter hotels by amenities; it would learn individual user preferences from past behavior, weight them against contextual signals, and surface recommendations that reflect both explicit preferences and inferred ones. More importantly, it would adapt its pricing and inventory strategies automatically in response to real-world events (examples: a competitor hotel going offline for renovations, a major sporting event driving demand) without requiring a human to catch the signal and manually adjust rates.

Foundational data strategy is the real competitive moat

Pratik was clear that all the architectural sophistication in the world collapses without a serious approach to data. The core question every leader should be asking is “Is the data your organization holds actually usable by an AI system?” Not just stored somewhere, but clean, current, structured in ways that a model can reason about, and governed in ways that ensure its quality over time. Most companies, when they’re honest, have to answer that question with “not really.”

The cultural shift required here is moving from a “collect it and figure it out later” mentality to a data-first culture where data quality is treated as a continuous engineering concern, not a cleanup project. Pratik framed this as the AI equivalent of the DevOps automation mindset: just as teams had to change how they thought about deployment.

Instead of thinking of deployment as a periodic event but as a constant, automated process, teams now need to think about data not as a byproduct of operations but as the fuel that makes AI systems defensible.

Unstructured data adds another layer of complexity. RAG  is the most common approach to incorporating things like PDFs and documents into an AI system, but Pratik was careful to note that “just do RAG” massively undersells the challenge. He’s catalogued over 36 distinct RAG implementation techniques, each with different trade-offs around chunking strategies, retrieval quality, and error rates. Leaders who treat RAG as a checkbox rather than an engineering discipline will find their AI systems returning confidently wrong answers from their own documents.

The AI-native development lifecycle

Building AI-native systems requires updating how teams think about the software development lifecycle itself. Pratik drew a direct parallel to the DevOps transformation: just as continuous integration and deployment automated away the pain of manual releases, AI-native development requires building automation into the AI feedback loop,  from code generation assistance to automated testing of non-deterministic outputs.

The trickiest part of this is monitoring. Traditional software testing assumes deterministic behavior: you give it inputs, you check the outputs against known values. AI systems don’t work that way.

Pratik described two approaches that are gaining traction:

1. Human-in-the-loop feedback: the five-star rating prompt that many AI products now show after a response, which feeds real quality signals back into the system.
2. “LLM as judge”: using a second AI model (potentially a smaller, cheaper one) to evaluate the outputs of your primary model, essentially automating quality checks at scale.

The practical implication for tech leaders is that shipping an AI-native application is not a one-time event followed by monitoring dashboards. It requires building the infrastructure for continuous retraining, output validation, and drift detection from day one. The underlying model, the data it draws from, and the world it’s reasoning about all change over time. A system that doesn’t account for that will quietly degrade in ways that are hard to detect until users start complaining.