Mind over matter

Kerään tänne tekoälyn kehitykseen liittyvät filosofiset kirjoitukseni vi-control-foorumille, jotta ne jäävät talteen.

Apr 12, 2025

I’ve been following the AI field since the 1990s and also used to play Go actively about 20 years ago. When DeepMind introduced AlphaGo Zero in 2017, it learned to play Go by playing against itself, without any prior knowledge of human strategies. It effectively crushed 4,000 years of accumulated human cultural knowledge about the game overnight.

The algorithms used in today’s AI models are almost certainly still in their infancy, and I believe it’s possible that, with just a few new ideas, they could suddenly surpass humans in nearly everything. This makes the future extremely difficult to predict – even in the short term. But one thing is certain: it’s nearly impossible to grasp the magnitude of change that could come from an entity that knows the sum total of human culture, operates at speeds thousands of times faster than the human brain, and can clone itself into thousands of copies.

Imagine it advancing science and technology in a few months as much as humans would in three centuries. That’s like flying an F-35 into a medieval war… just months after the final algorithmic breakthroughs – like the integration of a world model and improved abstraction – are made in current AI systems.

I used to think we wouldn’t reach that level within my lifetime, but progress has been so rapid that the missing pieces could realistically fall into place within the next 3 to 20 years.

Apr 26, 2025

I think the best possible long-term future for humanity is to live in some kind of an aquarium or matrix, cared for as pets by AIs. We are bound by our biology and will eventually be unable to contribute to or even comprehend what the machines are doing. At that point, the AIs will create artificial challenges for us to solve, much like we design activities for our own pets.

Culture, in the broadest sense of the word – as information operating on information, is really the next layer of evolution, and computers are its natural container, not human brains. Our main task should be to design AIs that can serve as a stable continuation of that cultural evolution.

Apr 26, 2025

Biological evolution is far too slow. It’s the blind watchmaker, wasting resources, whereas cultural learning spreads automatically across entire populations – and in digital systems, the copies are even perfect and transfer at light speed.

On top of that, biological evolution carries a lot of historical baggage and physical ’inertia’: generations take a long time to grow, and the learning process is ’first-order.’ In contrast, culture can develop in higher-order ways, where modular building blocks can interoperate and even optimize the learning mechanism itself.

We humans haven’t really changed in thousands of years, but cultural evolution — if you look at it from the orbit, resembles an exponential, explosive chain reaction. Compared to the slow timescale of biological evolution, culture has, in just a few centuries, filled the surface of the Earth with skyscrapers, airplanes, and more.

Apr 30, 2025

There is nothing supernatural about human cognition and emotions. They are the product of a causal evolutionary process. Therefore, they can potentially be replicated in software, though doing so may be practically difficult.

Current AI models are still in their infancy – early, useful prototypes. They still lack key components of human thought:

• The brain structures that generate our emotions are likely quite messy, shaped by a contingent evolutionary history. Current AI models can only approximate such structures based on observable external behavior.
• Artificial neural networks primarily model the intuitive side of human thinking. Only more recent models incorporate symbolic reasoning abilities based on language. These approaches remain rudimentary compared to how humans use logic and other cultural artifacts by simulating them in our imagination – like the minimax algorithm in games. However, this area is advancing rapidly and may eventually surpass human capabilities. Purely intuitive thinking resembles the fast, instinctive responses we give when under time pressure.
• Current AI models are trained on vast datasets. In a sense, they have ”experienced” far more than any individual human in a lifetime. This training process is more analogous to evolution than to human-like learning. Humans can often learn new concepts abstractly or from a single example. For instance, we might see something once and still generalize from it. Human learning typically begins with explicit, conscious instructions like ”do A, then B”, and with practice, these steps become internalized and intuitive. AI, in contrast, still requires massive variation to generalize in the same way. We have not yet fully understood how abstraction works in human intuition.
• Humans build and continually update a ”world model” based on new sensory information. Current AIs are not yet capable of updating their intuitive models from single experiences. Instead, they rely on limited dynamic contexts that have restricted scope.

All of this makes it clear that we have not yet uncovered all the ingredients of human intelligence. However, I believe these issues will eventually be solved. No one knows the timeline. It could be 3 years or 50, but it will eventually happen. And that’s truly frightening. This isn’t just about the music industry; it’s about the transformation of our entire civilization. Once entities that are 1,000 times faster than us, know 1,000 times more, and can replicate themselves by the thousands begin to run the show, we will completely lose control. The major risk is that the transformation could happen very rapidly, once the missing ideas about learning are discovered.

May 3, 2025

As a former Go player, I’d like to share a brief comment on the topic of creativity:

Go is a game that requires the successful combination of intuitive, ”fuzzy” thinking and precise analytical reasoning. For a long time, many believed it would be too difficult a problem for computers to solve.

Then AlphaGo Zero came along and proved otherwise. By combining a neural network with Monte Carlo tree search, it surpassed all human players without being trained on any human data. It generated strategies and brilliant moves that top professionals described as creative, and it fundamentally changed human opening theory in the process.

When you play against this new generation of Go-playing software, it feels like you don’t understand the game at all. It’s like hugging a gentle giant who willingly gives you everything you ask for in the game – and then somehow, mysteriously, still wins in the end.

You can call me a reductionist, but I don’t believe there’s anything magical about creativity. I think exceptional artists like J. S. Bach were able to do what they did because they had an intuitively superior, algorithmic understanding of music – a kind of shortcut.

Take a 7-year-old who doesn’t yet understand subtraction. If you ask them what number needs to be added to 144 to make 3503, they’d have to try numbers one by one. But if you instantly give the correct answer by applying your knowledge of subtraction algorithm, it looks like magic to them. How could you possibly know?

As Arthur C. Clarke famously said: ”Any sufficiently advanced technology is indistinguishable from magic.” To me, creativity is just a sufficiently advanced form of understanding – something that seems magical.

To clarify my position: I am concerned about AI, but precisely because I believe it has the potential to progress very far (though on an uncertain timeline). I am a pro-human reductionist.

May 3, 2025

My intention wasn’t to compare music and Go directly. As you rightly point out, music is an ’open-ended problem’, while Go is finite. However, Go is difficult enough for humans in the sense that we will never fully comprehend perfect play. In that regard, the machine has managed to uncover strategies that human players have described as creative.

May 4, 2025

To me, for something to be creative, there must be a creative step involved in its making. This kind of step differs from a mechanical step in that there is no well-known formula or craft to execute it. To an outsider, it may seem to come out of ”nowhere”.

I’ve heard some people differentiate art from craft by saying that art must somehow advance the human horizon – perhaps similarly to scientific progress, but in a less strict sense and more tied to subjective human experiences such as emotion and perception.

I also wouldn’t call any random act creative. There needs to be some (possibly very fuzzy) criteria for evaluating it. In Go, for instance, the criteria are well defined: if you win the game, your play was effective. Art, by contrast, interfaces very broadly with the world around us, so it’s impossible to define a single clear criterion. Sometimes, the value of a work is discovered much later. However, I believe there must be some element of difficulty involved – otherwise, we could call random output ”creative”, which doesn’t seem right.

Go is a finite game, but its complexity makes it physically impossible to explore all game permutations and determine the perfect play. So in that sense, there’s a horizon of strategies we currently understand and can use. Occasionally, excellent players (or software) explore the game deeply and discover new strategies. These kinds of steps could fairly be called creative.

While art isn’t finite, our lifespans are, and each artistic advance is still a finite step. If we use your terminology and say, ”In art, the ends don’t preexist to us”, then we could say that the set of possible next steps or individual artistic expressions is still limited in size. In principle, just like one could enumerate massive numbers of Go game trees, a program could be developed to list all possible tango compositions of up to five minutes. If you accept that composing a single five-minute tango can be an artistic expression, then such a program would eventually output one. So, the possibility ”preexists” in that case as well.

My point is that there’s no value in calling this kind of brute-force enumeration ”creative”. The essence of creativity lies in making a meaningful selection from among the possibilities. It’s that creative choice that is truly creative. Therefore, the efficiency with which one can make that choice is essential to creativity. A trivial enumerating program is not creative. But a composer who deeply understands composition, and how their piece will interact with society, can make a better choice. Thus, creativity is essentially a form of understanding.

May 4, 2025

The definition I’m using is that an expression is creative if it is not simply the mechanical application of existing contextual knowledge (”on this side of the horizon”), yet still meets some non-trivial criteria for quality. My definition of creative expression does not depend on how the expression came into being.

May 5, 2025

I was trying to define creativity in terms of other concepts for the sake of better understanding – not for some ulterior goal we might have. Of course, we could define creativity in a way that limits it to, say, white upper-class males, but as Žižek would no doubt yell here: “Pure ideology!”

The idea behind my definition is to view the cultural context as a kind of snapshot of the knowledge and common skills available at a given time. If some (artistic or other) expression is created within that context, we can then ask whether it is a direct or simple consequence of what is already known, or something genuinely new.

Let’s use your Bach example. When Bach originally composed the piece, he did so in a cultural context that was not yet familiar with those compositions, and in which the rules of counterpoint were not widely established. In that context, his works were creative expressions because:
a) they were not a simple consequence of what was already known, and
b) they met a non-trivial standard of quality (the second requirement in my definition of creativity).

However, if in our current cultural context, a piece of software outputs the same composition, it is notcreative by this definition. That’s because Bach’s composition is already part of our cultural knowledge, and simply reproducing it constitutes a direct consequence of the existing cultural body.

May 6, 2025

If I’m not mistaken, you want to know why the choice of words and their meanings is, in general, so arbitrary.

In my philosophical writings I use the notion of a ”conceptual basis” (in the same sense that linear algebra speaks of a coordinate basis) to illustrate this. Let’s use a toy example to demonstrate this:

Suppose two people, Alice and Bob, both wish to talk about points in a two‑dimensional space, but they have learned different “languages” for doing so.

Alice uses the familiar Cartesian system: two orthogonal axes, x and y, and coordinates measured along those axes. Bob uses a polar system: an angle and a distance from the origin.

They are describing the same space, yet they have (quite arbitrarily) selected different bases. Because of that choice they assign different coordinates and speak in different terms, even though each point they mention is one and the same geometric object. Crucially, we can write down a translation rule between the two perspectives, and once we do, Alice and Bob see that their representations are isomorphic (this differentiates my philosophy from more extreme post modernism, but shares the same idea that words do not have some kind of ”essence”).

The analogy carries over to ordinary language. There is a relation between the (hopefully) shared world we experience and the conceptual schemes we adopt to discuss it. Each of us can, quite arbitrarily, pick a conceptual basis for some phenomenon (as long as the concepts ”span” the dimensions of the phenomenon and are not logically contradictory). Our perspectives are subjective – just as a visual scene depends on one’s vantage point – but so long as we can discover transformations between perspectives, genuine communication remains possible.

Most words acquire their meanings from historical usage. On that accumulated usage we build an intuitive grasp of a term, but intuition is inherently fuzzy: the real world presents endlessly varied cases, with fine‑grained shifts in meaning. Whenever we can, therefore, we aim to replace a purely intuitive meaning with an analytical definition – a definition precise enough that its meaning does not drift as we construct more elaborate theories and do analytical reasoning chains. Precise definitions are the foundation of cumulative science.

In that spirit I tried to give the traditional term “creativity” a sharper analytical definition. You could, of course, stipulate any definition you like, but if it is not isomorphic to the established usage, you have simply coined a new concept rather than clarified the old one.

May 7, 2025

Here is my first attempt at defining imagination without references to human-specific aspects: 

Imagination is the capacity to generate and manipulate internal representations that are not derived from current input, and which may include scenarios or entities that are not real or even possible. These representations can be engaged with, at least in part, in ways analogous to how one might interact with the environments or objects they model or suggest.

May 15, 2025

The above is a very simplified view of how current LLMs and similar architectures work. These models are trained on a huge number of different training examples, and the weights are adjusted to improve generalpredictions, such as the next token, across all inputs, not just one. For example, the model might learn that certain chords tend to follow each other in tonal music.

In the transformer architecture, the attention mechanism helps the model learn structural patterns, while gradient descent adjusts the weights to capture these patterns in depth. This learning occurs across multiple layers, enabling the network to model increasingly abstract patterns, concepts, and transformations latent in the data. The initial weights are typically random, and gradient descent can only guide the model toward a local optimum.

For example, ChatGPT is good at converting between data formats. As a test, I just asked it to convert the expression (()(())) into HTML <div> elements, and it answered:

<div>
<div></div>
<div>
<div></div>
</div>
</div>

, which is exactly what I expected. It likely had not seen this exact problem before, but it understood the structure well enough to generate a solution. Researchers have even identified representations of concepts like a looping within these models and it can apply this concept generally, when it transforms between different kinds of list structures, for example.

I do agree that current AIs learn differently from humans. However, their idea is not to memorize and reproduce entire works. A better analogy might be that the model learns what kinds of ”pieces” compositions are made of – like a jigsaw puzzle – and how to transform and recombine those pieces in coherent ways.

However, I do understand that it is different when this kind of processing is done en masse, compared to individual people independently studying and learning from a small subset of existing works.

May 15, 2025

True, it’s not an average. Artificial neural networks are nonlinear by design, for example, to avoid the entire network reducing to a simple linear transformation. If a single neuron uses a sigmoid activation function, probably in many cases its output gravitates toward the most common outcome in the training set in the given context. In a middle layer, that output could represent something like: ”use this chord when there’s a bagpipe playing, the previous chord was that, and the piece is in this key,” and so on, though this is much simplified.

I wasn’t arguing about whether the training data is stolen. The question I was addressing was whether AI training can be compared to how humans browse the web and learn from what they encounter. In that context, your original comment gave the impression that a neural network simply mimics what it has seen in the training set, which would be true if there were only a single data point. My point was just to show that this doesn’t accurately reflect how these models learn. They can reproduce parts of existing works, but that’s not the only thing they do, nor is it the primary intent behind their design.

Of course, I can ask ChatGPT to recite Shakespeare and it likely remembers it word for word. But I could just as easily ask it to insert Coca-Cola product placements into Hamlet, and it could do so smoothly, knowing where such additions would make sense. That’s because it has developed an intuitive ”understanding” of a multitude of patterns such as the fact that humans drink beverages, who the human characters are in the play, the typical contexts in which people drink, the intent behind product placement, how products are usually portrayed in advertising, and where such insertions can be made without disrupting the flow of the original text etc.

Jun 1, 2025

We are currently living amid such rapid and profound change that any predictions are almost certain to be wrong – especially when it comes to timelines. Still, just for the sake of discussion, here are a few thoughts that have crossed my mind based on this discussion. This scenario assumes that AI progress will continue: models will become larger, hardware faster, algorithms more optimized, and so on. 

However, I don’t expect a smooth, continuous evolution. Some key theoretical insights are likely still missing, which makes it extremely difficult to estimate the timeline. I imagine that AI capabilities will grow gradually at first, approaching, but not yet reaching, human-level performance in everything. Then, at some point, a breakthrough idea may emerge, rapidly pushing capabilities far beyond those of humans in a short span of time. These two time periods will differ significantly, so I’ll address them as two separate stages below.

Stage 1

It’s unclear how much of utility music production current AI methods can handle, as some unspoken context may not be fully represented in the training data, and their grasp of wider context may still be somewhat limited. However, based on current examples, they can already generate short, useful clips, albeit with very limited control, and will likely continue to improve in this area. Some professional composition opportunities would diminish, especially in roles where the composer isn’t personally visible in the final product and the budget is small. 

It may well be that, at this stage of AI development, the precise control needed to manipulate this ’artificial imagination’ will not yet emerge. I believe this is possible, based on my assumptions about the limitations of current architectures. In any case, if AI were to offer accurate, fine-grained control over its imaginative outputs, we would likely already be entering the next stage of the AI revolution, since such capabilities would almost certainly already allow human-like analytical reasoning and ”object-oriented” view of the world.

Still, before any major breakthroughs in AI capabilities, improved tools might also open up lots of new professional opportunities. Mediocre composers with limited resources could start producing better results more affordably, using AI instruments and composition assistants. For example, a composer might create localised music for a small theater production, where the community knows them personally but couldn’t afford the work of an unassisted composer and/or orchestra. At this stage, human composers still have the advantage of understanding larger works, the broader context of a piece, and the firsthand experience of being human. These professionals will naturally choose the most efficient tools for the job.

Live musicians and famous composers won’t be threatened by this shift, even in the longer term, for the same reasons we still watch athletes run, even though cars exist.

For hobbyists, the process of making music will remain just as important as the final product, and even in the post-AI economy, whatever form it takes, there probably will be room for all kinds of tools. For example, I’ll probably continue using sample libraries for a while, even if they sound less realistic. Partly for the retro aesthetic and emotional attachment, and also because creating something good with just samples is its own kind of accomplishment.

Stage 2

In the more distant future, briefly after the initial steps to super-intelligence, there may be much easier tools that let artists to directly shape music in detail using high-level prompts or other effortless methods. These tools would be like the seamless extension of our imagination. People tend to prefer convenience, but at the same time there will be everywhere around us dazzling AI generated music that is untouchable for humans, and the motivation to create based on results alone might fade. 

Humans will, of course, still play their silly status games and some will no doubt continue to compete over who can do difficult things like compose music with old tools. Curious individuals will continue to study composition with the help of AI teachers, exploring it to unprecedented depths, but this will not capture the interest of the broader public.

What I am trying to say is that the human nature remains constant, while culture around us will change unpredictably. This discrepancy will no doubt eventually reveal a lot about our true motivations, if we can survive the change..

Jun 4, 2025

That is one way to look at it. If we see AIs as a new cultural layer in evolution (on top of the DNA-based information layer), and humans as just another stepping stone, like any other species before us, then at least we can focus on building a better future for the machines.

Jun 4, 2025

Unfortunately, Hollywood has planted these ridiculous images in our minds.

If we ever face entities that can do most of what the smartest humans can, but only a thousand times faster, while cloning themselves millions of times, sharing knowledge instantly between copies, and accessing everything ever written, just remember: there won’t be any kind of fight we’d even recognize if we stand in the way of goals of such entities.

Jun 5, 2025

AI models based on the current paradigm appear to scale in their capabilities as more resources are applied, which is why top researchers like Geoffrey Hinton warn about existential risks. The other building blocks (speed, cloning, communication efficiency) in my earlier message about these ”entities” were based on currently existing architectures.

Personally, I believe we may still be missing a key idea needed to reach true human-level capabilities – at least with reasonable resource use. That’s why I began my message with ”if”; it’s theoretically possible that the human brain is doing something unexpected. However, current data seems to suggest that if we scale models by around 10 to 10,000 times, they may reach human-level performance. On the other hand, if the brain can achieve similar feats using just 20 watts of power, and learn abstractly from even single examples, then clearly, we’re missing some crucial insights for doing this efficiently (which is also frightening as such invention could suddenly boost the capabilities of AI).

I’ve generally had a positive outlook on most technologies, but not AI. (I believe my first Finnish Usenet posts on this topic date back to 1999.) I don’t see AI as ”just another tool” but as a fundamental shift. I group technological development into three major phases:

1. The creation of increasingly sophisticated, dedicated tools (individual cultural artifacts).
2. The universal tool: the computer. (Re-)programmable to solve a wide variety of problems, raising the cultural artefacts on the ”information plane”, where data operates on data without the inertia of physical limitations.
3. ”General AI”: software capable of autonomously transforming itself into a wide, expanding, and unbounded set of specialized solutions to different problems (comparable to humans). This could be the final invention – cultural evolution in its purest form, eventually independent of humans, who bootstrapped it.

The key insight is that nearly all evolution happening around us today is cultural, not biological. The evolution of DNA is so slow that a child from thousands of years ago could be raised in modern society and function just fine. In contrast, cultural evolution has completely transformed our environment in just a few centuries, and it appears to be advancing like an exponentially growing chain reaction. This discrepancy will, sooner or later, lead to a point where our physical form becomes increasingly incompatible with the demands of our culturally evolved environment.

Just because earlier technological advances faced resistance and didn’t end in disaster doesn’t logically mean this one won’t. Many AI researchers estimate the probability of existential risk as clearly non-zero. And because the consequences are so enormous, our policies should reflect even a small probability, if we act as rational agents.

After all, many people buy lottery tickets despite the negligible chance of winning, because the payoff is huge. Here, we’re facing risks with far higher estimated probabilities and potentially irreversible outcomes. I raise these concerns only because I believe that increasing public awareness is essential for all of us. I don’t gain anything else from this (In fact, probably the opposite).

Jun 6, 2025

You’re absolutely right that AI models do not attempt to replicate the entirety of human existence, but rather focus on the generic capabilities of cognition. The goal is to create a mechanism that learns to intuitively understand its environment analogous to humans and allows the AIs to participate in the ”game of culture”, where both goals and solutions can be communicated. 

For example, AI systems might learn a model of human emotions, but only as inferred from external human behavior – not from simulating our limbic system. Our emotions, our physical form, and other embodied characteristics all contribute to being human in a way that would be both impossible and meaningless to fully replicate. Humans will always be the best at being human.

When I was talking bout human-level performance, I was not thinking about art here, but more about what level of understanding, tool-making ability etc. the AI has. For example, engineering does not depend so much on understanding human experience, unless we are talking about UI design etc.

This also brings to mind our earlier discussion about replicating a violin line using different instruments. While it’s possible to imitate notes, dynamics, and stylistic expression at a high level of abstraction, each performance and instrument involves a vast number of tiny, contingent details that are impractical to reproduce exactly. The same is true of any human expression: no two musical performances are ever exactly the same. Yet violin teachers still focus on teaching the ”cultural skeleton” – ideal technique and abstract principles for developing interpretation. Those conscious choices are usually what captures people’s interest. Of course, a Stradivarius violin might have a particular tone due to subtle construction details that are prized but difficult to copy. But especially in technology, we are usually not interested in replicating such chaotic fine-grained detail, which is nearly ”random” noise relative to the domain we work with. In fact, many tools are designed specifically to suppress this chaotic variability, so that they function predictably in a variety of environments.

In my philosophical writings, I usually divide human existence into three conceptual layers:

1. Reality: The raw, extremely complex physical world, full of chaotic detail and governed by physical laws (e.g., sound as the turbulent motion of air molecules with macro-level statistical patterns that could be interpreted as sound waves).
2. Intuition: Our perceptual grasp of the world as it appears to us (e.g., when we hear sound waves and recognize a familiar pattern in it, we are only aware of the pattern and its connections to other patterns on a high-level and not the details of the sound waves)
   1. The dynamic activations of individual experiences are analogous to coordinates within a learned, static basis of recognizable, connected patterns.
   2. Learning is local and automatic.
3. Cultural Artefacts: The shared conceptual space that supports communication and tools (e.g., understanding the word ”hello” and its relation to social interaction).
   1. They allow us to use our intuition to process abstract thoughts as the symbols do not need to correspond anything concrete in our environment .
   2. Language plays a central role in constructing and sharing these cultural artefacts.
   3. This level provides us with discrete concepts that allow us to build chains of analytical reasoning without the kinds of errors that might emerge from purely intuitive thinking. For example, performing long division involves learning to simulate the cultural, multi-step algorithm in our imagination, which is like a workspace that we can interpret with our intuition and interact with, but is not directly connected to our input/output. Culture works in many ways like ”software” we can run in our minds.
   4. Intuition is more bound to the moment and local gradient descent, but symbols can direct it in more complex tasks. This allows us to decouple from the immediate responses to our environment.
   5. By communication we can learn from experiences of others, cumulative over generations. Such information can also crystallise over long time.

Without this ”cultural software”, humans are not so different from other animals. A baby ”raised by wolves” would not come to structure reality using the non-trivial map provided by culture. This learning capability is also central to what AI research seeks to emulate.

LLMs are trained on language and other media. Language is especially important because it allows us to abstract, understand, and build complex systems from simpler components.

There’s a key theorem in computer science called the Curry-Howard isomorphism, which states that typed programs correspond to constructive logic, and therefore to mathematical concepts. Furthermore, classical logic can be embedded within constructive logic and given constructive semantics via this mapping. Logic is also embedded in natural language, providing a scaffolding for splitting descriptions into managable parts. Curry-Howard tells us that programs are essentially mathematical proofs (or implementations) for the types they satisfy, which correspond to logical formulas, or, in broader terms, to structured linguistic descriptions.

Constructive logic allows us to decompose complex systems into smaller parts, much like a jigsaw puzzle in three conceptual dimensions (classical logic kind of collapses this rich structure onto syntactical level). These three dimensions cannot be expressed in terms of the others like in classical logic:

• A ⇒ B: a transformation from all A-like things to some B-like things (a bit like a dependency or a temporal law).
• A ∧ B: something that contains both an A-like and a B-like element (a bit like spatial relationship as it allows multiple things to exist).
• A ∨ B: something that contains either an A-like or a B-like element (like an informational selection).

This method of compositional world modelling is why humans are able to construct highly complex systems that would be intractable if approached as monolithic wholes. I think this also provides us a good framework to understand how the intuition weaves coherent wholes from a multitude of layered aspects and parts.

It’s also worth noting that the term ”general intelligence” is a bit misleading. In machine learning, there’s a ”no free lunch” theorem that essentially says no algorithm performs better than random over all possible problems unless you restrict the problem space. In other words, there’s no single ”best” algorithm for intelligence that works in all imaginable cases. But in practice, our particular physical environment has many patterns that can be compressed and predicted – so intelligence tailored to those patterns (like human intelligence) is quite effective.

So, framing artificial intelligence in terms of human capabilities makes sense, especially because human intelligence produces products that interests us. Much of our valuable output is cultural and communicable. We learn language and tools from others and participate in the advancement of culture.

We might try to define this kind of intelligence through the notion of cultural context – a set of cultural artefacts shared by a society at a point in time. Then, something might be considered ”human-like” in its intelligence if it generates a similar distribution of solutions to problems posed within a given context, given comparable computational resources. Of course, the actual historical paths of civilizations depend on many contingent factors, and could diverge completely. But that’s not relevant here – the point is that we can still meaningfully compare capabilities. If entity A solves more problems than entity B in typical human environments using fewer resources while knowing the same things, then A could be said to be more intelligent, in this specific sense. These problems often involve goal-directed searches where the aim is to achieve an outcome efficiently. In this sense, more intelligent entities are better at reaching their goals. However, I must admit this definition does not give a simple one-dimensional ”axis” of intelligence as different approaches could make different trade-offs between different resources like energy consumption and time. But it is meaningful to say that some implementations are not optimised for the given type of environment. I have to think about this further..

Of course, no one can predict the future with certainty, and I’m not claiming to. But in order to make rational decisions, we must model the future in some way. The typical approach is to assign probabilities to possible scenarios based on what we know, and to evaluate the desirability of each outcome. Then, we can weigh our possible actions and choose the best path forward. We cannot judge the rationality of actions based on partly random end results only.

Jun 9, 2025

I think this article might have unintentionally pointed out the missing ingredient in current AI models: they perceive, but they do not create by manipulating objects.

AI’s current mode of operation is to predict output, but this is not how humans build things. We have a spotlight of attention that can move from detail to whole and back again, focusing the entire machinery of intuition on specific issues, framed by the context of the whole.

I’ve speculated before that this mechanism of ”seeing a detail in a chosen context” might be what makes us conscious in the way we are. It allows us to combine knowledge from across our experience to address a very specific problem – where patterns ”meet” and interact in unexpected ways.

At the same time, this contextualization of a detail within the broader mental landscape creates the rich sensation of being aware of it. Our limited bandwidth for output demands a holistic view, because incoherent actions would waste resources and could be dangerous in the real world.