Improving AI Output: The XY Problem

Some time ago I came across an article talking about the XY Problem. I don’t remember the article or its specific context, but the XY problem concept is well-documented. I’ve seen it very often in requirements specifications and IT professionals asking for support on chat rooms and forums. But it’s also relevant to working with AI as well.

XY Problem in Support

The signs are unmistakable, asking how to fix X so it does Y or how to make X do Y. It affects two main areas - fixing errors and implementing functionality.

When fixing errors, the focus on X can result in incomplete error traces, not identifying earlier upstream errors, or omitting - or not identifying - key environmental information. Key environmental information may be platform information, what was occurring at the time, or what else was occurring which may be an ancillary or root cause.

When implementing functionality, the problem can be that most people don’t use X to achieve Y, they take a different, best practice approach. Because the question is phrased specific to an unconventional approach, what is actually trying to be achieved may not be clear. It may also be a case of trying to change or configure the wrong part in the process.

In both scenarios, the questioner may receive no answer or a bad answer, caused by not identifying the correct question to ask.

XY Problem with AI

The same can happen with AI as well.

It is very common when trying to solve errors. Of course there can be PEBKAC errors, where the human-in-the-loop fails to provide the right information to the agentic harness. But with a diligent human, that is not so common. Agentic harnesses, if not properly configured, can focus on the closest error message and try to fix that. Fixing the error message may be the wrong approach, the problem may be caused by bad logic upstream that results in bad inputs, which cause the error. They can also follow a scattergun approach, applying a fix for the most obvious cause and when that fails to succeed, trying a variety of options with minimal analysis. It takes careful steering, for example with a skill to encourage the harness to build and test a variety of hypotheses, and use the human-in-the-loop to validate what it cannot test. And I’ve found agentic harnesses are not good at considering that there might be other things outside their control that are causing problems, for example other conflicting processes that are and should be running. Because they do not consider it, they can fail to ask the right questions to receive the right information or offer the hypothesis so that the human considers it. Some years ago I wrote a blog post on troubleshooting support, and I’ve found that many of the lessons there are relevant to guiding agentic harnesses.

With implementing functionality, the problems are more varied. The XY problem of asking the agentic harness to use a framework or library to achieve something impossible is not uncommon. I’ve hit this in the past asking an agentic harness to configure logging file rotation in a specific, unachievable way. Some hours later, it became apparent I had not been clear enough in the scenarios and the outcomes I wanted for each. If you’re uncertain, asking if it’s possible first is a quick way to avoid frustration and embarrassment.

Similarly, asking “what is the best practice way to achieve Y” is another wise approach. It guides the model to bring together its reading of the codebase with best practice approaches. This can result in better outcomes as long as the best practice approaches are well documented on the internet, and as long as the agentic harness focuses on the specific library or framework versions. Providing clear and comprehensive information is key to avoiding bad assumptions. And asking the model to validate its assumptions can identify what they are.

Hallucinations still occur. For example, if you’re not using a framework version the model is most aware of, the agentic harness can require carefully steering to use correct version documentation. If there is a dearth of published content for what you’re looking for, but there is commonly available information that sounds semantically correct. Historically software vendors have chosen terminology that sounds familiar outside the technology area, but works in a different proprietary way. At the time they were trying to appeal to a broader audience. Inadvertently, they have created a perfect storm to confuse models. MCPs, skills or LoRAs can mitigate this - but only if they’re available.

The best solution is always specificity, clarity, comprehensiveness, and prompting the agentic harness to grill you thoroughly to clarify assumptions or gaps. Alongside the human-in-the-loop needs to bring careful comprehension skills to identify misunderstandings or gaps in the responses from the agentic harness. This can be challenging with the natural verbosity and “AI-speak” of outputs. But we have control of the agentic harness to steer models to outputs that are easier to work with. A good example is the unslop skill.

Summary

The need for clarity of thought and communication is not new in agentic engineering. And it’s relevant for much more than this. But focusing on actual outcomes, not specific technical implementation approaches can avoid wasted time and effort - and of course money.