Introduction
Every so often you will see a post that concludes that “AI cannot be trusted since the result of their optimisation dwarfs the results of my manually handwritten optimisation”.
The flavor of such a post will go something along these lines:
I, an expert in my field having spend a decade or so working on these type of problems, create a baseline for an agent to optimize, e.g. with a starting result of 1s runtime or whatever.
The agent optimized the baseline and got it down to 100ms runtime.
I am able to write a solution that produces a 1ms runtime, ergo, AI outputs cannot be trusted since humans are still superior.
The purpose of this article is to convince you that such conclusions are quite meaningless, because they don’t account for the human time of searching for a good solution.
What Makes A Good Solution?
It can be argued that everything in life is a search problem.
We are always “looking” for the best solution. Even in entrepreneurship, “move fast, break things” is just another way of saying that we need to iterate along the solution space.
Movement / iteration is an act towards getting better; and the goal is to find the best solution. The globally best solution allows you to become a monopolistic behemoth, and the locally best solutions allow you to improve until you “get stuck” in that solution space and need to look elsewhere.
Everything in life is about searching for a good solution, ergo, everything in life is a search problem, that is, an algorithm that allows you to move and land on good solutions.
The hard part of a search algorithm is what steps to take next. When you take a step, you get new information (you end up on a different point of the search space), and given that new information, you need to decide where to go next.
A perfect search algorithm will bring us towards the globally optimal solution with every passing step, but humans (and AI) do not (yet) have perfect search algorithms.
Our experience, talent, tacit and implicit learning allow us to refine our search algorithm to improve it. A child will rarely be able to tell you what is the next best step in a complex field, but an expert who has made that field his life will often have a pretty good guess.
We call that many things: “experience”, “intuition”, etc, but it’s really just his fine-tuned search algorithm.
Experts And Their Great Solutions
When an expert says that they can out-optimize an agent at a particular solution, what they really mean to say is that they found a globally more optimal point in the search space than the agent.
What they have missed out, however, is how long they have spent fine-tuning their search algorithm and/or exploring the search space.
The agent has days, sometimes just hours, of computational time, and has refined a search algorithm capable enough to actually land in a local optima; whilst the human is comparing the agent against his decade(s) of experience and search.
I’ll take the human’s words that AI output is doomed when he pits himself against the AI in a field where he has like a child and still out-optimizes the AI.
How To Get AI To Give You Better Solutions?
This, ultimately, is the crux of most agentic engineers concerns.
Time Is The Ultimate Arbiter
A human expert has had plenty of time (years, decades) to explore their search space and tune their search algorithms. For really hard problems, you need to give the AI plenty of time to iterate, and to “institutionalize” its learning so that it may continue to tweak its search algorithm.
Confine The Search Space
In an infinite search space there are an infinite amount of points to explore. That takes more time. If you know where not to look, or even better, where to look, you should make that clear to your agents.
Guide The Search Algorithm
Defining the objective score (what is a good direction) is a large part of improving the search. Giving the agents clear instructions on how to iterate, if you have a view formed of experience, will also short-circuit the time needed for AI to learn those points on their own.
Conclusion
AI is getting frighteningly more intelligent with every passing iteration. AGI is also but a point in the AI search space, and we seem to be heading in that direction quite convincingly.
Current levels of AI have pretty good intuition in just about every technical field, and will likely continue to get better. Their short-ranged (e.g. horus/days) optimisations already produce something that is to be lauded, and they have proven that over a long enough horizon of searching (computational time), they can solve problems that no humans have recorded to have been able to do so -- even humans who have spend decades on that problem.