Where the Tool Stops

In this digital age, it is easy to altogether avoid the organic, analog approach. Evermore there is the digital shortcut as the “force multiplier”. What was the bicycle for the mind feels like it has become the feet, legs, hands, arms, mouth, and so on for the mind ever more encroaching upon the mind’s jurisdiction itself. It can seem that more and more is delegated wholesale to the digital realm versus the digital realm being an expansion of our capabilities or who we are. As a software engineer, programmer, developer, whatever title we have bestowed upon ourselves to mean “I try and make the silicon do what us humans want it to do”, I have a decent vantage point of where this delegation to the digital is being applied, and where the digital approach has usurped the analog for better or worse. Now, with AI weaving itself into every aspect of our lives it can feel like putting on a digital exoskeleton for enhancement if not for survival. It’s becoming harder to differentiate between where the tool stops and I begin.

It’s not unprecedented to have industry shake-ups from technical advancements of the tooling utilized. Software Engineering isn’t my first career, but it is my most digital facing one for sure. My other careers have been much more rudimentary, organic: carpentry, general construction, manufacturing, it wasn’t until my stint in the US Marine Corps did I pick up a career in technology, as a Marine masquerading as the resident IT person for a few hundred Marines deployed in austere conditions responsible for hardware and software. I saw first hand how technology can be a force multiplier, and when systems we have grown to rely on fail, how it can have catastrophic consequences in very kinetic ways. While carpentry and construction are very tactile and analog, there are analogous force multipliers within its domain such as electric power tools and pneumatic nail guns. While there were some in the industry that didn’t find the overhead worth the conversion; having to have a noisy expensive air compressor, hauling the weight around of a pneumatic nail gun, and the air hose tethered to it ever sabotaging agility across a worksite, after all they were very adept at swinging a hammer and driving framing nails. But for those up and coming framers who didn’t have the existing skills, the obvious advancement of the nail gun was the “weapon of choice”. But it didn’t take long before it was a job site expectation to use the nail gun as a directive, rather than choice.

In my current profession, the onslaught of AI has been received across a spectrum: it's extremely favorable for those who it is providing opportunity they would not have pragmatically had before. I’m seeing skilled individuals with unique life experiences who are non-developers, bring their knowledge to the digital realm via generative coding and software development creating objectively helpful, and often boutique, digital solutions. To being an almost literal force multiplier when in the hands of skilled and seasoned software engineers who already know how to wield a fast moving codebase with a plethora of contributors. To downright creating AI slop. I’m not sure there’s a similar enough precedent of a technical advancement that has such a breadth of application, while also on such a spectrum of usefulness to, frankly, harm. Harm to quality for the sake of speed and quantity, harm to the sanctity of artistic human creation. Harm to our expectations, which recalibrate quietly around whatever is cheapest to produce.

Going back to construction, when having to hand cut many of the same rafters that may have an intricate seat cut, and tail shape, we would use lots of “fancy maths” and instruments to trace out the exact spots of the board to cut. This process took a while, and you had to get it right. The obvious optimization is to do this once, save the first one, and simply trace it on the next board. The uninitiated would often make the “drifting pattern” mistake where they would take each consecutive cut rafter and trace the next board with it. The first one, the one where all the time was spent getting right would be at the bottom of the stack. This created pattern drift, where slowly each trace got off a hair, but after dozens of traces would be significant enough to matter structurally. The mitigation of course was label the initial rafter measured and cut as the PATTERN, and to keep it to trace every consecutive board. Now imagine for a moment that we allowed the drifting pattern to recalibrate our expectations for tolerance?

Part of the discipline of maintaining quality is knowing what tool to use for the job. What’s interesting about the nail gun completely usurping the common nail driving responsibilities of the framing hammer, the hammer is still a staple tool in the tool belt of the framer. And is often in the hands of the framer, more than the nail gun. For a number of reasons too: the nail that didn’t set, for the angle that the bulky gun can’t get into but a swing can, for non-driver related tasks such as tapping a board into place or even in direct collaboration with a nail gun, using the hammer as a weight sink on one side of the board as you shoot a nail angled into the other side, helping to keep the board from moving off its placement at the force of the nail being shot in. The skilled framer knows when to put down the nail gun and pickup the hammer. While not a one to one comparison with the nail gun and hammer, metaphorically there is an analogous hand-off between when to apply AI and when your own strength and skill should take over, often the pragmatic approach is choosing, carefully, when generative AI completely takes over, when its used in tandem with your direct expertise, and when it just needs to be set aside.

Specifically in software engineering, I tried to adopt this perspective and doing so has resulted in a gradient of AI/human responsibility of duties. Since a lot of software engineering is cognitive rather than physical, executed in the head rather than in the hands, the demarcation zone is usually in the realm of how much I think I can trust what is produced by generative AI. Trust doesn’t come for free though, it is earned. In terms of how generative AI earns my trust is through classic quantitative analysis via instrumentation, simulation, proofs, rigorous testing harnesses, basically I have to do due diligence to carefully scope what success looks like, then come up with an adequate way to prove that the solution meets the criteria while not being over engineered. This boilerplate overhead isn’t unique to AI’s output being scrutinized, it is borrowed from things like test driven development, or critical systems acceptance criteria. The risk inherited from generative content, like code, is cognitive dissonance, creating a black box solution with expected inputs and outputs, the dissonance being expectations of those inputs/outputs vs the actual logic gates within. So whatever I can measure, and scope to a defined success criteria the more I am confident in my trust of the black box.

So here is my gradient, in descending order of how much duty I am willing to hand over. It is subjective by nature, scoped to work I am willing to claim ownership of, and calibrated to what I know I know and what I know I don’t:

Something I can trust AI to do better than myself is combing through logs for the purpose of troubleshooting something specific. Providing relevant context of the issue, access to a corpus of aggregated logs, it can quickly filter out the red herrings, draw relevant correlations between log traces and events across systems and services. I usually don’t prompt it or expect it to try and solve an open ended problem, because it doesn’t have ALL the context I do about the complete system or underlying codebase driving the logs and events to begin with. But it can provide a contextually relevant report from the noise that can come from a chatty debug trace, providing me a list of potential trees to bark up. Then I redial, recalibrate, refine the focus and prompt again eventually ending in a hand-off state that I can start to make my own discoveries and observations and draw my own conclusions now that I have a line up of suspects.

Next is using it for research in a domain I am unfamiliar, leveraging its hypersonic speed to curate and present information on a topic providing me reference and awareness on the matter. Once I zoom in enough I get into original material territory only (the references), and pivot from generated output, to source material.

Moving further on the gradient, is code generation. Of which I am comfortable relying heavily on due to the thousands and thousands of hours I have spent authoring thousands and thousands of lines of code across several programming languages and design patterns, but probably more relevant, the thousands and thousands of lines of code I have read and reviewed of others, and had to maintain, and thus am fairly confident spelunking through a code base. As anyone who’s been coding for a long time has probably experienced, the majority of the work isn’t the happy path, it's the edge cases, the error handling, knowing when not to let perfect be the enemy of good enough. I could provide a whole post on my current technique but honestly as soon as it's published it may already be outdated due to the advancements in the landscape. For example, not too long ago I could not trust myself to hand over significant enough of authorship to generative AI to be considered vibe coding. But with the advent of adversarial review baked into some of the agentic workflows, a lot of the manual review steps I would perform are effectively being self-regulated to the point I can make that black box a little larger.

Moving beyond rote work and into higher level design thinking this is where I start to reach for the hammer more, but certainly utilize AI to rapidly iterate towards a deliverable. Specifically for architecting software and system design it is very helpful for me to use AI as a sort of well informed committee making sure I am not re-inventing the wheel and instead steering me towards possible past solutions or design patterns that may play into my architecture, or considerations before investing a lot towards implementation. This use of AI is where there is the most back and forth for me, filtered through a lot of skepticism, and where I discard the most; because so many potential solution paths can start to drive the story vs maintaining focus on the core problem being solved. The outcome of such sessions is not a turn key architectural design, but instead better awareness of whole swaths of territory that have already been solved in other similar problems. And knowing where the unique parts of my problem are helps me then know where I need to focus original thought and apply my skills, that is, where I will wield the hammer.

Where I am the most distrustful or most timid to delegate any parts wholesale to generative AI is communication, both in short and long form text. I do appreciate the feedback loop it can provide. Creating written material then providing the draft to AI for its feedback on if it works for a particular audience has been the most productive use of it for me in context of writing. But even having AI produce a layout for written material starts to throw off my groove of creative writing. Just as, as a software engineer, I am more comfortable leveraging AI for generative code, a professional writer may be more comfortable utilizing generative AI for writing, through crafting with prompts then re-shaping. But I lose the initiative if I let it do that for me.

Two cases don’t fit into this gradient, at opposite ends of a different question, which isn't how much I trust the output but how much I let it run unattended. The cases being rote process management (building organizational protocols, and executing on them), and open ended, unbounded troubleshooting. I can trust administrative management to AI collaboration, because it is just executing on an organizational and administrative protocol, and is scoped to water tight success criteria. Whereas with open ended troubleshooting without clear success criteria or scoped definitions of tolerances, I run into unknown behavior often with poor outcomes. I have let it run wild before and we get a runaway minecart situation where it wants to resolve a bug that is within tolerance, but since that tolerance has not been defined the resulting refactor creates compromises in its wake that break the success criteria in other ways. Motivation without constraint is like hitting the fill button on a paint program to only find out there was an unbounded perimeter and now the whole world has been consumed by the fill color.

Determining where the tool stops and I start becomes a matter of defining bounded success and scope for the tool in a measurable and provable way. Where I can’t define it, or can’t prove it, I keep the work. The specific enumerations I stated for myself are lines drawn from my own reference points, all of it a snapshot within a moving landscape. The line moves as the measuring gets cheaper. When adversarial review made verification nearly free, the black box grew, and it will grow again the same way. The nail gun never replaced the hammer. It took over the work it could do better, and the hammer stayed where it always was, in the hand.