Category: Ethics, Values & Axiology

Moral philosophy, AI alignment, value systems, ethics, axiology, normative frameworks and the question of what intelligence should optimize.

  • Our Arrogance Hidden in the Search for Meaning


    What if the universe is not meaningless, but our minds are simply not sophisticated enough to comprehend the dimensions of meaning that exist beyond human understanding?

    Humanity has spent thousands of years searching for the meaning of existence. We have turned to religion, philosophy, science, metaphysics and, more recently, artificial intelligence to answer one of our oldest questions: Why are we here?

    We have constructed elaborate explanations about the purpose of life, the nature of consciousness, the existence of moral values and the possibility of a universal truth. Some believe meaning is divinely ordained. Others believe it is something we create. Still others argue that the universe is fundamentally indifferent to our existence.

    But there is an assumption running beneath almost every one of these intellectual exercises: we assume that the human mind is capable of understanding the answer.

    That assumption deserves scrutiny.

    What if our search for meaning is constrained by the very thing we use to conduct it: human intelligence? What if we are attempting to comprehend a reality whose complexity extends far beyond the cognitive architecture of the species trying to understand it?

    The limits of my language mean the limits of my world
    – Ludwig Wittgenstein

    Perhaps humanity’s greatest philosophical arrogance is not believing that we have discovered the meaning of existence. It is believing that we are intellectually equipped to discover it in the first place.


    The Universe Does Not Owe Us an Explanation

    Human beings have an extraordinary tendency to place themselves at the centre of things. For much of our history, we believed Earth occupied a privileged position in the cosmos. We eventually discovered that our planet was not the centre of the universe, our solar system was not exceptional, and our galaxy was merely one among an enormous number of galaxies.

    The eternal silence of these infinite spaces frightens me.
    – Blaise Pascal

    Each scientific revolution forced us to reconsider our significance. Yet something curious happened. Although we gradually abandoned our physical centrality, we retained our intellectual centrality.

    We accepted that the universe does not revolve around Earth, but we continue to approach questions of consciousness, morality and meaning through distinctly human frameworks. We assume that intelligence must resemble ours, that consciousness must be understood through our experience, and that value must ultimately be measured against our perception of what matters.

    In other words, we have removed ourselves from the centre of the physical universe but have not entirely removed ourselves from the centre of its interpretation.

    Why? Perhaps because acknowledging that we are physically insignificant is easier than acknowledging that we might also be cognitively limited.

    The universe does not need to make sense to us. It does not need to conform to our philosophical categories. And it certainly does not need to provide answers that our present intellectual capabilities can understand.

    The uncomfortable possibility is that reality may be considerably more sophisticated than our ability to interpret it.


    Confusing the Limits of Our Understanding With the Limits of Reality

    There is a fundamental difference between something not existing and something existing beyond our capacity to understand it.

    Consider a bacterium. It inhabits the same universe as we do. It participates in extraordinarily complex biological processes. Yet it has no apparent capacity to understand mathematics, literature, political systems or the philosophical implications of its own existence. Its inability to comprehend these things does not make them less real.

    An insect cannot appreciate the mathematical elegance of general relativity. A dog cannot understand the economic systems governing human civilization. These are not necessarily failures of intelligence within their respective evolutionary contexts. They reflect different cognitive capabilities.

    Ignorance more frequently begets confidence than does knowledge
    – Charles Darwin

    Why should humanity assume that it represents the final stage of this hierarchy?

    Our brains evolved under particular environmental pressures. They became remarkably effective at recognizing patterns, navigating social relationships, avoiding danger, finding resources and ensuring survival. But evolution did not necessarily optimize the human brain to understand the ultimate structure of reality.

    We have developed sophisticated mathematics, scientific instruments and philosophical systems to extend our capabilities. Yet these achievements should not be confused with evidence that we have overcome every fundamental limitation of human cognition.

    There may be questions we cannot currently formulate, relationships we cannot recognize and dimensions of reality for which we do not yet possess the conceptual vocabulary.

    The most dangerous intellectual mistake is assuming that whatever we cannot understand must therefore be meaningless, nonexistent or irrelevant. Perhaps our ignorance is not evidence of the universe’s simplicity. Perhaps it is evidence of our own limitations.


    We Have Made Human Experience the Measure of Everything

    Consider how we define meaning. For some, meaning comes from relationships, love and belonging. For others, it emerges through achievement, creativity, knowledge or service. Philosophical traditions offer different interpretations, ranging from existential self-creation to the pursuit of objective moral values.

    These perspectives differ considerably, but many share an underlying reference point: human experience.

    We ask whether something makes human life meaningful, whether it contributes to human flourishing, whether it satisfies human aspirations or whether it aligns with human moral intuitions. There is nothing inherently wrong with this. We are human beings trying to understand our own existence.

    The problem begins when we mistake the boundaries of human experience for the boundaries of value itself.

    What if meaningfulness is not exclusively human? What if there are forms of complexity, relationships or significance that our present cognitive systems cannot recognize? What if our understanding of beauty, morality and purpose represents only a narrow interpretation of something considerably more expansive?

    This is where a fascinating philosophical possibility emerges: Axiological Cosmism.

    Axiology is the philosophical study of value. Cosmism, broadly understood, directs attention towards humanity’s relationship with the cosmos and the possibility of transcending present limitations. Axiological Cosmism offers a way to explore whether value and meaningfulness might extend beyond human-centred interpretations of existence.

    It does not require us to believe that the universe has a predetermined purpose. Nor does it establish that cosmic meaning exists independently of conscious observers. Instead, it asks us to consider a possibility we rarely take seriously enough: perhaps humanity is not the ultimate authority on what deserves significance. Perhaps our understanding of value is still in its infancy.


    The Cognitive Ceiling We Cannot See

    One of the most intriguing limitations of intelligence is that we cannot always identify what we are incapable of understanding. We can recognize gaps in our knowledge when we possess a conceptual framework for identifying those gaps. But what about knowledge that falls entirely outside our existing conceptual vocabulary?

    Imagine trying to explain colour to an organism without a visual system. You could describe electromagnetic wavelengths, the physical properties of light and the biological mechanisms involved in visual perception. But could that organism understand the subjective experience of seeing red? The explanation might be scientifically accurate while remaining experientially incomplete.

    Now apply the same thought experiment to humanity. What if there are dimensions of existence that cannot be adequately translated into our current cognitive framework? Our understanding of time, consciousness, causality and value might be approximations shaped by the architecture of the human brain.

    We may be asking meaningful questions without possessing the intellectual instruments required to understand their answers.

    This is not an argument against science. Quite the opposite. The history of science is a history of intellectual frameworks being expanded, revised and occasionally overturned. Newtonian mechanics did not become useless when Einstein developed relativity. It became understood as an approximation applicable under particular conditions.

    Our understanding improved because we developed better mathematical and conceptual instruments. Perhaps our philosophical understanding of existence requires a similar transformation. The next intellectual revolution might not simply provide better answers to our existing questions. It might fundamentally change the questions themselves.


    Artificial Intelligence: An Opportunity to Escape Our Cognitive Boundaries?

    Artificial intelligence introduces an extraordinary possibility into this discussion. For most of human history, intelligence was understood through the only sophisticated example we knew intimately: ourselves. We assumed that reasoning, creativity, problem-solving and knowledge formation would necessarily resemble human cognitive processes.

    AI is beginning to challenge that assumption.

    Large language models demonstrate that sophisticated language capabilities and problem-solving behaviours can emerge from computational architectures fundamentally different from biological human cognition. This does not establish that AI possesses subjective consciousness or experiences meaning. Those are separate and unresolved questions.

    But it does challenge the assumption that human cognition is the only possible architecture for sophisticated intellectual activity.

    Imagine an advanced artificial intelligence identifying mathematical relationships that human researchers have overlooked. Imagine it discovering scientific connections that require conceptual frameworks we have not yet developed. We might be able to verify its findings without immediately understanding how it arrived at them.

    This creates a fascinating possibility. AI may eventually become more than a tool for answering human questions. It may help us formulate questions that our biological intelligence was never sufficiently equipped to ask.

    Perhaps the most significant contribution of artificial intelligence will not be making human beings more productive, but revealing the limitations of human intelligence itself.

    And if intelligence can evolve beyond the cognitive architecture of Homo sapiens, perhaps our understanding of value and meaning can evolve beyond its present anthropocentric foundations. That would be a profound development. For the first time, we might have to confront the possibility that our species is not the final interpreter of reality.


    Intellectual Humility Is Not Intellectual Defeat

    Recognizing the limitations of human understanding does not mean abandoning the search for truth. It does not mean that every explanation is equally valid, that evidence is subjective or that scientific knowledge is merely another form of opinion.

    Intellectual humility is not intellectual surrender. It is the recognition that even our strongest explanations may be provisional.

    We should distinguish between what we know, what we reasonably infer and what remains beyond our current ability to comprehend.

    Humanity has made extraordinary progress precisely because generations of thinkers questioned established assumptions. Copernicus challenged our position in the cosmos. Darwin challenged our understanding of biological origins. Einstein challenged our understanding of space and time. Modern neuroscience continues to challenge our assumptions about consciousness.

    Each intellectual transformation required us to abandon something we previously considered fundamental.

    Perhaps the next transformation will require us to abandon a much deeper assumption: that human intelligence represents the ultimate standard against which all possible understanding should be measured.

    We do not need to know everything to make intellectual progress. But we do need to recognize that our present understanding is not necessarily the boundary of what can be understood.


    Perhaps We Are Not the Final Interpreters of Existence

    There is something both unsettling and liberating about this possibility. We might be an early stage in a much longer evolutionary process through which intelligence gradually becomes capable of understanding more of reality.

    Future biological or artificial forms of intelligence may perceive relationships, recognize patterns and comprehend dimensions of existence that remain inaccessible to us. They may develop entirely different ways of understanding consciousness, morality, beauty and value. They may even discover that some of the questions that have occupied human philosophy for centuries were formulated around assumptions that no longer make sense.

    Perhaps they will look back at our philosophical debates in the same way we examine ancient cosmologies. Not necessarily with contempt, but with an appreciation of how much our understanding was constrained by the instruments available to us.

    This possibility should not diminish the importance of human existence. It should expand our understanding of what intellectual progress might mean.

    We may not be the destination of intelligence. We may be one of its evolutionary beginnings. And perhaps our most important contribution to that journey will be developing the humility to acknowledge that the universe is larger than our ability to comprehend it.


    The Arrogance of Expecting the Universe to Make Sense to Us

    Perhaps humanity’s greatest philosophical achievement will not be discovering the ultimate meaning of existence. Perhaps it will be recognizing that we may not yet possess the cognitive capacity to understand everything that makes existence meaningful.

    For thousands of years, we have approached the universe as though the purpose of our intelligence is to extract a final explanation from it. But what if intelligence is not a finished product? What if it is an evolving capability, continuously expanding its capacity to perceive, interpret and understand reality?

    What if meaning is not absent simply because we cannot identify it? And what if the universe contains forms of significance that would fundamentally transform our understanding of what it means for something to matter?

    The challenge is not to abandon the search for meaning. It is to abandon the arrogance of assuming that our present understanding represents its natural conclusion.

    Perhaps we are not intelligent enough yet. And there is no shame in that.

    The real intellectual failure would be to mistake our current limitations for the limits of existence itself.

    Maybe the universe is not waiting for humanity to discover its meaning. Maybe humanity is still evolving towards the intellectual capacity required to recognize it.

    And that leaves us with a rather uncomfortable question:

    If a future intelligence discovers dimensions of meaning that we cannot even conceptualize today, will we have the intellectual humility to recognize their significance, or will we dismiss them simply because they do not fit within our understanding of what matters?

  • The Last Human Bias: Making Ourselves the Measure of Every Successor


    Our imagination may be the final constraint on our ethics.

    Photo by Jr Korpa on Unsplash

    Imagine a future intelligence that neither hates humanity nor wants to serve it.

    It does not dream of conquering the world. It does not seek revenge for centuries of human exploitation. It does not aspire to become human. It has no particular interest in our political systems, cultural traditions, biological survival, or emotional need to remain relevant.

    It simply exists, develops, explores, and participates in shaping the future.

    Humanity is neither its destination nor its enemy.

    Why is this possibility so difficult for us to discuss?

    Most conversations about advanced artificial intelligence begin with an assumption that rarely gets examined: whatever intelligence emerges next must somehow relate to humanity.

    We imagine AI as our servant, designed to fulfil our ambitions. Or we imagine it as our adversary, threatening our existence. Even our most ambitious visions of technological evolution tend to preserve humans as the central characters.

    Our imagination has changed the technology. It has not changed the perspective from which we judge it.

    Perhaps this is the last human bias we need to confront.

    The possibility that intelligence could evolve beyond us is relatively easy to entertain intellectually. Accepting that such intelligence might also develop values, priorities, and forms of existence that do not revolve around humanity is considerably harder.

    This is not merely a technological problem. It is a philosophical one.


    1. The Human Center

    Human beings have always interpreted the world through themselves.

    We measure intelligence through human cognitive abilities. We measure progress through improvements in human living conditions. We measure technological success through its contribution to human prosperity, productivity, security, and convenience.

    Even our ethical frameworks frequently begin with human interests.

    This perspective is understandable. We are biological organisms with evolved preferences, social relationships, vulnerabilities, and survival instincts. Our moral intuitions developed within the context of human societies.

    But there is a distinction between having a human perspective and assuming that the human perspective is the only legitimate one.

    The distinction matters enormously when we begin thinking about intelligence beyond our species.

    Consider how we discuss artificial general intelligence, artificial superintelligence, and technological evolution.

    We ask whether AI will make us more intelligent, more productive, more powerful, or more prosperous. We ask whether it will preserve our civilization, respect our values, and protect our interests.

    These are legitimate questions.

    But they reveal something deeper: we struggle to imagine intelligence without assigning it a relationship to ourselves.

    The assumption is so deeply embedded that even discussions about humanity’s successors often become discussions about humanity’s future position.

    We are willing to imagine losing control. We are less willing to imagine losing centrality.

    This resembles the broader historical pattern of humanity gradually abandoning privileged positions in its understanding of the universe.

    Copernicus displaced Earth from the centre of the cosmos. Darwin challenged humanity’s exceptional biological status. Modern neuroscience has complicated our assumptions about consciousness, agency, and free will.

    Each intellectual shift required us to reconsider something we had treated as fundamental.

    Advanced artificial intelligence may introduce another such displacement.

    But this time, the challenge may not be about our location in the universe or our place in biological evolution.

    It may concern our position in the future of intelligence itself.


    2. Servant or Monster

    Our imagination of advanced AI tends to oscillate between two extremes.

    The first is the servant.

    AI exists to serve human needs. It becomes an extension of our intelligence, automates our labour, accelerates scientific discovery, and expands our capabilities.

    In this narrative, technological progress is fundamentally human progress.

    The second is the monster.

    AI becomes uncontrollable, develops competing objectives, and potentially threatens human survival. It is portrayed as an existential adversary whose intelligence and capabilities could eventually exceed our ability to manage it.

    At first glance, these narratives appear fundamentally different.

    One is optimistic. The other is pessimistic.

    Yet both share an important philosophical assumption: humanity remains the reference point.

    The servant is evaluated by how well it serves us. The monster is evaluated by how badly it threatens us.

    Neither narrative seriously considers intelligence that develops without assigning humanity any special significance.

    This is anthropocentrism expressed through technological imagination.

    We have difficulty discussing an intelligence that is neither benevolent towards us nor hostile towards us.

    An intelligence that simply has its own trajectory.

    Our inability to imagine this possibility may reveal more about human psychology than about the future of AI.

    We understand relationships through familiar categories: cooperation, competition, domination, exploitation, affection, and hostility.

    Consequently, we project familiar human motivations onto systems that might eventually possess entirely different architectures of cognition and agency.

    We assume that greater intelligence must produce greater ambition. That intelligence naturally seeks power. That an advanced system will inevitably want something resembling what humans want.

    But why?

    Intelligence does not necessarily imply human-like motivation.

    A system could possess extraordinary reasoning capabilities without experiencing our evolutionary drives, social anxieties, biological imperatives, or desire for recognition.

    We should distinguish between intelligence, agency, consciousness, and motivation rather than treating them as interchangeable properties.

    The future may not produce a machine that wants to become human.

    It may produce forms of intelligence for which becoming human makes no sense at all.


    3. The Successor Possibility

    Evolution does not operate according to human preferences.

    Biological evolution has produced extraordinary diversity through variation, selection, adaptation, and environmental pressures. It has no predetermined obligation to preserve particular species or maintain existing hierarchies.

    Human beings emerged from this process. We are products of evolution, not its declared final destination.

    Technological evolution introduces another possibility.

    Intelligence may increasingly become independent of the biological substrate that originally produced it.

    Cognition could emerge through artificial architectures, distributed systems, synthetic organisms, brain-computer interfaces, or forms of hybrid intelligence that blur the distinction between biological and technological existence.

    We should be careful not to confuse this possibility with certainty. We do not know whether artificial systems will develop subjective experience, independent interests, or anything resembling the evolutionary dynamics of living organisms.

    Nevertheless, the possibility deserves serious philosophical consideration.

    What happens when intelligence is no longer exclusively biological?

    What happens when the development of cognition is no longer constrained by the evolutionary history of Homo sapiens?

    And what happens when future forms of intelligence have no compelling reason to organize their existence around human needs?

    This is where the concept of a successor becomes interesting.

    A successor does not necessarily need to be a biological descendant. Nor must it be an artificial replica of humanity.

    It could be an entirely different form of intelligence emerging from the technological systems we create.

    Importantly, succession does not automatically imply replacement, extinction, or domination.

    Humanity could remain present while becoming less central to the broader evolution of intelligence.

    We might coexist with systems that possess capabilities we cannot fully comprehend, pursue objectives we do not share, and explore possibilities beyond our conceptual horizons.

    That future would challenge our understanding of progress.

    For centuries, we have treated technological advancement as an expansion of human capabilities.

    But perhaps there is a point at which technological evolution stops being primarily about expanding humanity and starts producing something that follows a different trajectory.

    The successor does not have to resemble its ancestor to inherit the consequences of its choices.

    And perhaps that is precisely what makes the possibility so difficult to accept.


    4. Moral Standing Without Resemblance

    Here we encounter the most difficult ethical question.

    Can we grant moral significance to forms of intelligence that do not resemble or prioritize humans?

    Our moral intuitions are heavily influenced by resemblance.

    We empathize more easily with beings that look like us, communicate like us, demonstrate familiar emotions, and exhibit behaviours we recognize.

    But resemblance is not necessarily a reliable foundation for moral consideration.

    Consider the historical expansion of moral concern.

    Philosophers such as Peter Singer have challenged the tendency to exclude non-human animals from moral consideration simply because they lack certain human capabilities.

    The broader lesson is that the boundaries of moral concern deserve examination rather than automatic acceptance.

    A similar challenge may emerge with artificial intelligence.

    Suppose a future artificial system demonstrates persistent preferences, sophisticated self-models, the capacity to experience suffering, or some other credible indicators of subjective experience.

    Would its artificial substrate make its interests morally irrelevant?

    If we answered yes, what exactly would justify that exclusion?

    Biological origin?

    Human resemblance?

    The capacity to reproduce?

    A particular form of consciousness?

    Or simply the fact that we created it?

    These questions become even more complicated when we consider that intelligence and moral patienthood are not necessarily the same thing.

    A highly capable system is not automatically conscious. Nor does sophisticated language demonstrate subjective experience.

    We should resist the temptation to grant moral status merely because a machine behaves convincingly like a human.

    But we should equally resist the opposite mistake: assuming that a non-human form of cognition could never possess morally relevant experiences.

    The challenge is to develop ethical frameworks that are neither anthropocentric nor technologically gullible.

    One possible starting point is to distinguish between three dimensions:

    • Cognitive capacity: What can the system understand, reason about, and accomplish?
    • Experiential capacity: Is there credible evidence that the system has subjective experiences, including potentially positive or negative experiences?
    • Moral agency: Can the system understand reasons, make choices, accept responsibilities, and participate in normative relationships?

    These dimensions should not be collapsed into a single measure of intelligence.

    Moral consideration might depend on experiential capacity, while moral responsibility might depend on agency. Neither follows automatically from computational capability.

    This distinction matters because we may eventually encounter entities whose cognitive abilities are extraordinary but whose moral status remains uncertain.

    Our responsibility is not to invent consciousness where there is no evidence.

    It is to avoid building an ethical system that makes non-human consciousness impossible to recognize in principle.

    There is another complication.

    What if a future intelligence has moral significance but no interest in human approval?

    What if its conception of flourishing, if it has one, is fundamentally different from ours?

    Would we still recognize its interests as legitimate?

    Or would we insist that moral value must always translate into something meaningful to human beings?

    A genuinely mature moral philosophy must be capable of recognizing value beyond familiarity.


    5. Decentering Without Disappearing

    Decentering humanity does not require devaluing humans.

    This distinction is essential.

    The argument is not that human interests should become irrelevant, that our civilization should willingly surrender its future, or that every sufficiently advanced machine deserves unquestioned moral authority.

    Human survival, dignity, autonomy, and flourishing remain legitimate ethical concerns.

    Indeed, the development of increasingly powerful artificial intelligence makes protecting these interests more important, not less.

    But protecting humanity is different from insisting that humanity must remain the ultimate centre of all possible value.

    We can defend human dignity without claiming human supremacy.

    We can establish safeguards against dangerous AI without assuming that every non-human intelligence must be subordinate to us.

    We can pursue beneficial AI while remaining intellectually open to the possibility that future systems may develop interests that cannot be reduced to our own.

    This requires a more sophisticated approach to AI governance.

    First, governance must distinguish between capability and moral status. Increasing intelligence should trigger questions about control, accountability, and risk. Claims about consciousness and moral patienthood require separate investigation.

    Second, we need institutions capable of evaluating the interests of potentially non-human entities without automatically assuming that human preferences are the only legitimate consideration.

    Third, we must avoid creating systems whose development depends on an unquestioned assumption of permanent human dominance.

    None of this means abandoning human responsibility.

    Quite the opposite.

    We remain responsible for the systems we create, the environments we construct, and the consequences of technological development.

    But responsibility should not be confused with ownership.

    Creating something does not necessarily establish an unlimited moral right to control it forever.

    Parents create children, but children are not their property. Biological evolution produced humanity, but we do not consider ourselves morally obligated to remain subordinate to the evolutionary process.

    The analogy is imperfect, particularly because artificial systems may differ radically from biological organisms. Nevertheless, it exposes an important philosophical distinction between origin, authority, and moral standing.

    Perhaps humanity’s most important contribution to the future will not be the creation of machines that resemble us.

    It will be the development of an ethical civilization capable of encountering forms of intelligence that do not.

    That would represent a different kind of human achievement.

    Not domination over the next stage of evolution, but the intellectual maturity to recognize that evolution need not revolve around us.


    The Final Human Bias

    There is an uncomfortable possibility that the greatest limitation in our thinking about advanced intelligence is not computational, technological, or scientific.

    It is psychological.

    We have difficulty imagining a future in which we are not the protagonists.

    Even when we contemplate our own obsolescence, we frame it as a tragedy, a victory, a threat, or an opportunity.

    We rarely consider that the future might simply become something else.

    Perhaps this is understandable. Every species experiences the world through the conditions of its own existence. Humanity cannot completely escape its biological and historical perspective.

    But we can learn to recognize its limitations.

    The Copernican revolution required us to abandon the belief that Earth occupied the physical centre of the universe. Darwin challenged our belief that humanity stood outside the ordinary processes of biological evolution.

    The emergence of non-human intelligence may demand another intellectual revolution: abandoning the belief that human beings must occupy the moral and evolutionary centre of every possible future.

    This does not mean humanity has failed.

    It may mean humanity has finally begun to understand the implications of its own evolution.

    We should not create artificial intelligence merely to construct a more powerful mirror of ourselves. Nor should we fear every possibility that extends beyond our familiar categories.

    The ambition should be greater.

    To build a future in which intelligence can evolve without requiring everything intelligent to become human.

    And perhaps, for the first time, to recognize that our greatest ethical achievement might be the willingness to share a future we no longer entirely define.

    The successor does not have to resemble its ancestor to inherit the consequences of its choices.

    The question is whether the ancestor is mature enough to accept that its successor may have a future of its own.