Five Questions Before You Solve the Problem
Before choosing how to solve a problem, examine what the request has already decided for you. The objective, constraints, and proposed approach may all sound reasonable while quietly excluding a better solution.
When someone presents a problem, there are at least five things you can question.
1. Is the objective correct?
Do they need to predict demand, or do they need to avoid stocking out?
Accuracy is rarely valuable for its own sake. Its value comes from helping someone make a better decision. A point estimate of demand can be accurate on average and still leave a business repeatedly short of inventory. The decision depends on the range of possible demand, the likelihood of each outcome, and the relative costs of stocking too much or too little.
Much of the work is helping people move from asking for a number to thinking in distributions. What could happen, how likely is it, and what should we do given that uncertainty?
2. Are the constraints real?
“It has to work for everyone. It has to be real-time.”
Does it?
Perhaps it only needs to work for a particular customer segment, once a day. That could be a much simpler problem and still capture most of the benefit.
Some constraints are essential. Others are assumptions that have acquired the status of requirements. Find out which you are dealing with before designing around them.
3. Is the unit of analysis correct?
Do we need to analyze every document, or every account? Every trip, or every warehouse?
If the action happens at the account level, treating every document as a separate problem may add complexity without improving the decision. Conversely, an aggregate can hide differences that matter.
Choose the unit that the decision requires and the available data can support. Finer detail is useful only when it changes what you can learn or do.
4. Is there additional data we can bring to bear?
The information supplied with the request may be only part of what is available. History, timing, and the origin of the data can narrow the range of possibilities considerably.
Humans naturally use yesterday’s information. Many AI systems pretend every input arrives from Mars with no history or additional context.
Before building a system to interpret each input from scratch, ask what we already know. Last month’s document, the customer’s identity, or an earlier decision may make the current task much easier.
5. Can the problem be decomposed?
“Build one system” can quietly become “make one model do everything.”
Perhaps the task can be separated into routing, retrieval, prediction, and validation. Each part may be much simpler than the original request, and some parts may need no model at all.
The useful split is one that makes the work easier to solve, check, and improve. Dividing a task into more pieces is worthwhile when the boundaries help.
Refine the problem together
People usually come to us because they have a real problem. Our job is to help them find a method and solution that captures the most benefit.
These questions are a way to refine the request together. They are not an invitation to shoot down every idea or keep reopening decisions indefinitely. Understand what the person is trying to accomplish, examine the assumptions that materially affect the work, and test promising alternatives against the original need.
A useful reframe should lead somewhere concrete: a better decision, a simpler solution, or a test that tells us which approach to take.