We Don’t Compute Everything
Combinatorial explosion, correlation, and structure — the foundational logic of HIPLE
Years ago, back when I was studying CRM software, I was deep in the Apriori algorithm — association rule mining that finds, from sales data, “what tends to be bought together with what.” It taught me something simple that has stayed with me ever since.
You cannot compute every possible relationship.
With n items, the number of possible subsets is 2ⁿ − 1. Add just a few more items and the number of combinations explodes. So Apriori itself doesn’t blindly compute every combination — it prunes candidates it can already rule out, shrinking the search space.
The question it left me with was more fundamental than the algorithm:
Is intelligent computation about computing more — or about figuring out what you don’t need to compute?
Decades later, looking again at artificial intelligence, I came back to the same question.
Correlation is not logic
What association rules find is, fundamentally, patterns of co-occurrence. That A and B often occur together is useful information — in recommendation, marketing, or demand forecasting, that alone carries real value.
But correlation is not, in itself, causation or logical proof. Think of the famous “beer and diapers” story. Even if a statistic shows the two are bought together, the statistic itself doesn’t explain why the phenomenon occurred.
There is a more important problem. As the number of relationships you search grows enormous, so does the chance of finding a relationship that looks strong purely by accident. In other words, a large search space creates two problems at once:
Large Search Space → Combinatorial Explosion + Spurious Association
Correlation can be excellent evidence. But correlation alone cannot establish truth.
This difference matters especially where a wrong answer is costly — medical diagnosis, industrial safety. The fact that a symptom frequently co-occurs with a disease in the data is not the same as the judgment that this patient’s evidence actually supports that disease.
So such a system needs not only the ability to find patterns, but a separate structure that judges: “Can this pattern be trusted?”
Modern AI computes a staggering number of relationships
The Transformer showed a very powerful answer to this problem. A typical dense self-attention layer computes broad pairwise interactions among tokens.
This does not mean it literally enumerates every possible subset. Apriori’s 2ⁿ combination space and a Transformer’s attention computation are not the same problem mathematically. But between them there is a shared design question that I find important:
If you can’t be sure in advance which relationships are relevant, how wide a space of relationships must you compute first?
Today’s large models have pushed this approach to the extreme with enormous data and compute — and the result is astonishing. Language, code, images, reasoning — problems that once required different systems can now be handled by a single giant model.
Yet, quite apart from that success, I wanted to ask a different question:
Even in problems where we already know the structure, must we compute every relationship broadly?
Is a bigger model the answer to every problem?
It is not accurate to explain the LLM problem simply as “hallucination happens because attention computes too many correlations.” Hallucination involves many causes — training data, generation method, distribution shift, calibration, retrieval, verification.
What I take issue with in HIPLE is a little different. A large statistical model is superb at generating plausible relationships, but the procedure for verifying whether such a relationship is true in the external world is not the same as the model’s basic generative process. That is,
Plausibility ≠ Truth
Making a model larger lets it learn more patterns and obtain better representations. But scale itself does not automatically answer the question, “Has this relationship been verified?”
So I began to think in a different direction.
So, HIPLE — we don’t compute everything
HIPLE stands for Hierarchical Independent Parallel Learning Engine. The starting point is simple:
Use the structure we already know to shrink the space we have to compute from the outset.
Take medicine. When a patient says “my chest hurts,” a human doctor does not search the whole of medical knowledge with equal weight. They use the large structure first.
Patient │ ├── Head ├── Shoulder ├── Chest ├── Abdomen ├── Pelvis └── Cross-cutting systems
And within Chest, they divide again.
Chest │ ├── Cardiovascular ├── Pulmonary ├── Pleural ├── Musculoskeletal └── Other / Cross-system
Instead of computing the entire unlikely world at equal depth, computation is moved to the part relevant to the current evidence. This is HIPLE’s Hierarchical.
Turning Apriori’s pruning into architecture
Here the idea I learned long ago from Apriori returns. Instead of building the whole search space first and discarding the unneeded parts later, we don’t build the space we never needed to look at in the first place.
Conceptually, consider a system with N features in total. Without structure, the potential combination space grows very quickly. If we divide it into k relatively independent regions, each with nᵢ features, the conceptual search space changes from one vast global space to a sum of small local spaces:
2ᴺ (one vast global space)
↓
Σ (i=1..k) 2^(nᵢ) (many small local spaces)
This does not mean HIPLE actually enumerates these subsets. The expression is a conceptual analogy for the difference between a structure-free global interaction space and a structure-limited local interaction space.
For example, treating 60 elements as one space gives about 2⁶⁰ ≈ 1.15 × 10¹⁸ subsets. Dividing them into six independent regions of ten each gives, in the same conceptual comparison, 6 × 2¹⁰ = 6,144.
Structure is not merely an optimization that makes computation a little faster. Structure itself changes the search space.
But dividing alone is not enough
If it were only a matter of splitting a problem into several small models, it wouldn’t need to be HIPLE. There is Mixture-of-Experts, there are ensembles, there is sparse attention.
The difference I care about in HIPLE is putting knowledge into the very process that decides what to compute. Routing is not determined by learned scores alone; known structure and verified rules constrain which paths are computable. That is, rather than finding the answer directly in the full correlation space,
full correlation space → knowledge-permitted search space → local computation → verification
it proceeds this way. So in HIPLE, the reduction of the computation space is itself a representation of knowledge.
Separating correlation from logic
The second problem is separating the roles of correlation and logic. In HIPLE, the neural representation can express complex, flexible patterns in the data. But important knowledge can be expressed in a separate, growable symbolic concept structure. The two worlds play different roles.
Neural representation
│ discovers / represents patterns
▼
Correlation
│ evidence + verification
▼
Symbolic knowledge
│
▼
Auditable reasoning
We are not asking the neural model to stop finding correlations — that is what it does best. What HIPLE adds is the next step:
Of the relationships discovered, which do we accept as knowledge?
A candidate relationship can be verified through external evidence and outcomes, and once it earns enough confidence it enters the system’s knowledge structure. Conversely, when new evidence conflicts with an existing rule, that rule must be open to re-evaluation.
So knowledge is neither a fixed rulebook nor mere neural weights. It is a structure that grows as it is verified, and is revised when necessary.
Think independently, integrate at the end
Dividing hierarchically creates another problem: the world is not perfectly separable. Chest pain can be a heart problem, but it can also be a lung problem. An abdominal condition can present as pain in the chest or shoulder, and cancer can cut across several anatomical regions.
So HIPLE’s Independent does not mean “they don’t look at each other.” It is closer to this: each specialist domain keeps its own knowledge and parameters and judges independently, and consults other domains when needed.
Case
│
┌─────┼─────┐
▼ ▼ ▼
Engine A Engine B Engine C
│ │ │
└─────┼─────┘
▼
Integration
│
┌─────┴─────┐
▼ ▼
Decision Abstain
In this structure, retraining one domain does not require retraining all of them. If new knowledge affects only a particular subtree, only that part can be revised. This is why Independent and Parallel matter in HIPLE.
And when it doesn’t know, it doesn’t answer
I regard this as a very important ability in HIPLE. If the engines’ judgments don’t agree well enough, or the necessary evidence is missing, or the input lies outside the domains the system knows, the system does not have to force a conclusion.
Instead it can ask for more evidence, route to another specialist domain, escalate to a human, or withhold judgment. That is, “I don’t know” must also be a normal output of the system.
Intelligence is not only the ability to always produce an answer. The ability to judge that the current evidence does not permit an answer is also part of intelligence.
An honest boundary
Of course this approach has an important premise: there must be structure.
In fields like medicine, with long-accumulated physiological and anatomical structure, it is easy to build a starting point. Fields like industrial equipment, law, finance, and science — where one can define at least some verifiable structure — are also possible. Conversely, for problems whose structure is entirely unknown, broad relational search is still needed.
HIPLE’s goal is not to eliminate that. Rather, I think of it this way:
Use what is known as structure, explore what is unknown — but do not treat a discovered relationship as truth right away.
The initial structure need not be perfect either. Verified knowledge can be a seed; relationships not yet certain can remain candidates. As new external evidence accumulates the structure can grow, and when evidence shows existing knowledge to be wrong it can be revised.
So HIPLE’s structure is not a wall but a growing boundary.
In the end, the question is not about compute
At first I thought it was a matter of compute. The thought began watching combinations explode in Apriori. But the problem I now want to address in HIPLE is a little larger:
- What to compute.
- What not to compute.
- Which correlations to accept as knowledge.
- When to request another specialist’s judgment.
- And, most importantly, when to say I don’t know yet.
So HIPLE’s basic flow is:
Structure → Selective Local Computation → Independent Reasoning
→ External Verification → Integration or Abstention
What Apriori taught me long ago was not merely a data-mining algorithm. Computing every possibility is not intelligence. Perhaps an important part of intelligence lies in the opposite.
Intelligence does not come from computing everything. It comes from the structure that knows what not to compute.
— Kyewook Lee
HIPLE — Hierarchical Independent Parallel Learning Engine