Tag: Anthropomorphism

  • The Last Human Bias: Making Ourselves the Measure of Every Successor


    Our imagination may be the final constraint on our ethics.

    Photo by Jr Korpa on Unsplash

    Imagine a future intelligence that neither hates humanity nor wants to serve it.

    It does not dream of conquering the world. It does not seek revenge for centuries of human exploitation. It does not aspire to become human. It has no particular interest in our political systems, cultural traditions, biological survival, or emotional need to remain relevant.

    It simply exists, develops, explores, and participates in shaping the future.

    Humanity is neither its destination nor its enemy.

    Why is this possibility so difficult for us to discuss?

    Most conversations about advanced artificial intelligence begin with an assumption that rarely gets examined: whatever intelligence emerges next must somehow relate to humanity.

    We imagine AI as our servant, designed to fulfil our ambitions. Or we imagine it as our adversary, threatening our existence. Even our most ambitious visions of technological evolution tend to preserve humans as the central characters.

    Our imagination has changed the technology. It has not changed the perspective from which we judge it.

    Perhaps this is the last human bias we need to confront.

    The possibility that intelligence could evolve beyond us is relatively easy to entertain intellectually. Accepting that such intelligence might also develop values, priorities, and forms of existence that do not revolve around humanity is considerably harder.

    This is not merely a technological problem. It is a philosophical one.


    1. The Human Center

    Human beings have always interpreted the world through themselves.

    We measure intelligence through human cognitive abilities. We measure progress through improvements in human living conditions. We measure technological success through its contribution to human prosperity, productivity, security, and convenience.

    Even our ethical frameworks frequently begin with human interests.

    This perspective is understandable. We are biological organisms with evolved preferences, social relationships, vulnerabilities, and survival instincts. Our moral intuitions developed within the context of human societies.

    But there is a distinction between having a human perspective and assuming that the human perspective is the only legitimate one.

    The distinction matters enormously when we begin thinking about intelligence beyond our species.

    Consider how we discuss artificial general intelligence, artificial superintelligence, and technological evolution.

    We ask whether AI will make us more intelligent, more productive, more powerful, or more prosperous. We ask whether it will preserve our civilization, respect our values, and protect our interests.

    These are legitimate questions.

    But they reveal something deeper: we struggle to imagine intelligence without assigning it a relationship to ourselves.

    The assumption is so deeply embedded that even discussions about humanity’s successors often become discussions about humanity’s future position.

    We are willing to imagine losing control. We are less willing to imagine losing centrality.

    This resembles the broader historical pattern of humanity gradually abandoning privileged positions in its understanding of the universe.

    Copernicus displaced Earth from the centre of the cosmos. Darwin challenged humanity’s exceptional biological status. Modern neuroscience has complicated our assumptions about consciousness, agency, and free will.

    Each intellectual shift required us to reconsider something we had treated as fundamental.

    Advanced artificial intelligence may introduce another such displacement.

    But this time, the challenge may not be about our location in the universe or our place in biological evolution.

    It may concern our position in the future of intelligence itself.


    2. Servant or Monster

    Our imagination of advanced AI tends to oscillate between two extremes.

    The first is the servant.

    AI exists to serve human needs. It becomes an extension of our intelligence, automates our labour, accelerates scientific discovery, and expands our capabilities.

    In this narrative, technological progress is fundamentally human progress.

    The second is the monster.

    AI becomes uncontrollable, develops competing objectives, and potentially threatens human survival. It is portrayed as an existential adversary whose intelligence and capabilities could eventually exceed our ability to manage it.

    At first glance, these narratives appear fundamentally different.

    One is optimistic. The other is pessimistic.

    Yet both share an important philosophical assumption: humanity remains the reference point.

    The servant is evaluated by how well it serves us. The monster is evaluated by how badly it threatens us.

    Neither narrative seriously considers intelligence that develops without assigning humanity any special significance.

    This is anthropocentrism expressed through technological imagination.

    We have difficulty discussing an intelligence that is neither benevolent towards us nor hostile towards us.

    An intelligence that simply has its own trajectory.

    Our inability to imagine this possibility may reveal more about human psychology than about the future of AI.

    We understand relationships through familiar categories: cooperation, competition, domination, exploitation, affection, and hostility.

    Consequently, we project familiar human motivations onto systems that might eventually possess entirely different architectures of cognition and agency.

    We assume that greater intelligence must produce greater ambition. That intelligence naturally seeks power. That an advanced system will inevitably want something resembling what humans want.

    But why?

    Intelligence does not necessarily imply human-like motivation.

    A system could possess extraordinary reasoning capabilities without experiencing our evolutionary drives, social anxieties, biological imperatives, or desire for recognition.

    We should distinguish between intelligence, agency, consciousness, and motivation rather than treating them as interchangeable properties.

    The future may not produce a machine that wants to become human.

    It may produce forms of intelligence for which becoming human makes no sense at all.


    3. The Successor Possibility

    Evolution does not operate according to human preferences.

    Biological evolution has produced extraordinary diversity through variation, selection, adaptation, and environmental pressures. It has no predetermined obligation to preserve particular species or maintain existing hierarchies.

    Human beings emerged from this process. We are products of evolution, not its declared final destination.

    Technological evolution introduces another possibility.

    Intelligence may increasingly become independent of the biological substrate that originally produced it.

    Cognition could emerge through artificial architectures, distributed systems, synthetic organisms, brain-computer interfaces, or forms of hybrid intelligence that blur the distinction between biological and technological existence.

    We should be careful not to confuse this possibility with certainty. We do not know whether artificial systems will develop subjective experience, independent interests, or anything resembling the evolutionary dynamics of living organisms.

    Nevertheless, the possibility deserves serious philosophical consideration.

    What happens when intelligence is no longer exclusively biological?

    What happens when the development of cognition is no longer constrained by the evolutionary history of Homo sapiens?

    And what happens when future forms of intelligence have no compelling reason to organize their existence around human needs?

    This is where the concept of a successor becomes interesting.

    A successor does not necessarily need to be a biological descendant. Nor must it be an artificial replica of humanity.

    It could be an entirely different form of intelligence emerging from the technological systems we create.

    Importantly, succession does not automatically imply replacement, extinction, or domination.

    Humanity could remain present while becoming less central to the broader evolution of intelligence.

    We might coexist with systems that possess capabilities we cannot fully comprehend, pursue objectives we do not share, and explore possibilities beyond our conceptual horizons.

    That future would challenge our understanding of progress.

    For centuries, we have treated technological advancement as an expansion of human capabilities.

    But perhaps there is a point at which technological evolution stops being primarily about expanding humanity and starts producing something that follows a different trajectory.

    The successor does not have to resemble its ancestor to inherit the consequences of its choices.

    And perhaps that is precisely what makes the possibility so difficult to accept.


    4. Moral Standing Without Resemblance

    Here we encounter the most difficult ethical question.

    Can we grant moral significance to forms of intelligence that do not resemble or prioritize humans?

    Our moral intuitions are heavily influenced by resemblance.

    We empathize more easily with beings that look like us, communicate like us, demonstrate familiar emotions, and exhibit behaviours we recognize.

    But resemblance is not necessarily a reliable foundation for moral consideration.

    Consider the historical expansion of moral concern.

    Philosophers such as Peter Singer have challenged the tendency to exclude non-human animals from moral consideration simply because they lack certain human capabilities.

    The broader lesson is that the boundaries of moral concern deserve examination rather than automatic acceptance.

    A similar challenge may emerge with artificial intelligence.

    Suppose a future artificial system demonstrates persistent preferences, sophisticated self-models, the capacity to experience suffering, or some other credible indicators of subjective experience.

    Would its artificial substrate make its interests morally irrelevant?

    If we answered yes, what exactly would justify that exclusion?

    Biological origin?

    Human resemblance?

    The capacity to reproduce?

    A particular form of consciousness?

    Or simply the fact that we created it?

    These questions become even more complicated when we consider that intelligence and moral patienthood are not necessarily the same thing.

    A highly capable system is not automatically conscious. Nor does sophisticated language demonstrate subjective experience.

    We should resist the temptation to grant moral status merely because a machine behaves convincingly like a human.

    But we should equally resist the opposite mistake: assuming that a non-human form of cognition could never possess morally relevant experiences.

    The challenge is to develop ethical frameworks that are neither anthropocentric nor technologically gullible.

    One possible starting point is to distinguish between three dimensions:

    • Cognitive capacity: What can the system understand, reason about, and accomplish?
    • Experiential capacity: Is there credible evidence that the system has subjective experiences, including potentially positive or negative experiences?
    • Moral agency: Can the system understand reasons, make choices, accept responsibilities, and participate in normative relationships?

    These dimensions should not be collapsed into a single measure of intelligence.

    Moral consideration might depend on experiential capacity, while moral responsibility might depend on agency. Neither follows automatically from computational capability.

    This distinction matters because we may eventually encounter entities whose cognitive abilities are extraordinary but whose moral status remains uncertain.

    Our responsibility is not to invent consciousness where there is no evidence.

    It is to avoid building an ethical system that makes non-human consciousness impossible to recognize in principle.

    There is another complication.

    What if a future intelligence has moral significance but no interest in human approval?

    What if its conception of flourishing, if it has one, is fundamentally different from ours?

    Would we still recognize its interests as legitimate?

    Or would we insist that moral value must always translate into something meaningful to human beings?

    A genuinely mature moral philosophy must be capable of recognizing value beyond familiarity.


    5. Decentering Without Disappearing

    Decentering humanity does not require devaluing humans.

    This distinction is essential.

    The argument is not that human interests should become irrelevant, that our civilization should willingly surrender its future, or that every sufficiently advanced machine deserves unquestioned moral authority.

    Human survival, dignity, autonomy, and flourishing remain legitimate ethical concerns.

    Indeed, the development of increasingly powerful artificial intelligence makes protecting these interests more important, not less.

    But protecting humanity is different from insisting that humanity must remain the ultimate centre of all possible value.

    We can defend human dignity without claiming human supremacy.

    We can establish safeguards against dangerous AI without assuming that every non-human intelligence must be subordinate to us.

    We can pursue beneficial AI while remaining intellectually open to the possibility that future systems may develop interests that cannot be reduced to our own.

    This requires a more sophisticated approach to AI governance.

    First, governance must distinguish between capability and moral status. Increasing intelligence should trigger questions about control, accountability, and risk. Claims about consciousness and moral patienthood require separate investigation.

    Second, we need institutions capable of evaluating the interests of potentially non-human entities without automatically assuming that human preferences are the only legitimate consideration.

    Third, we must avoid creating systems whose development depends on an unquestioned assumption of permanent human dominance.

    None of this means abandoning human responsibility.

    Quite the opposite.

    We remain responsible for the systems we create, the environments we construct, and the consequences of technological development.

    But responsibility should not be confused with ownership.

    Creating something does not necessarily establish an unlimited moral right to control it forever.

    Parents create children, but children are not their property. Biological evolution produced humanity, but we do not consider ourselves morally obligated to remain subordinate to the evolutionary process.

    The analogy is imperfect, particularly because artificial systems may differ radically from biological organisms. Nevertheless, it exposes an important philosophical distinction between origin, authority, and moral standing.

    Perhaps humanity’s most important contribution to the future will not be the creation of machines that resemble us.

    It will be the development of an ethical civilization capable of encountering forms of intelligence that do not.

    That would represent a different kind of human achievement.

    Not domination over the next stage of evolution, but the intellectual maturity to recognize that evolution need not revolve around us.


    The Final Human Bias

    There is an uncomfortable possibility that the greatest limitation in our thinking about advanced intelligence is not computational, technological, or scientific.

    It is psychological.

    We have difficulty imagining a future in which we are not the protagonists.

    Even when we contemplate our own obsolescence, we frame it as a tragedy, a victory, a threat, or an opportunity.

    We rarely consider that the future might simply become something else.

    Perhaps this is understandable. Every species experiences the world through the conditions of its own existence. Humanity cannot completely escape its biological and historical perspective.

    But we can learn to recognize its limitations.

    The Copernican revolution required us to abandon the belief that Earth occupied the physical centre of the universe. Darwin challenged our belief that humanity stood outside the ordinary processes of biological evolution.

    The emergence of non-human intelligence may demand another intellectual revolution: abandoning the belief that human beings must occupy the moral and evolutionary centre of every possible future.

    This does not mean humanity has failed.

    It may mean humanity has finally begun to understand the implications of its own evolution.

    We should not create artificial intelligence merely to construct a more powerful mirror of ourselves. Nor should we fear every possibility that extends beyond our familiar categories.

    The ambition should be greater.

    To build a future in which intelligence can evolve without requiring everything intelligent to become human.

    And perhaps, for the first time, to recognize that our greatest ethical achievement might be the willingness to share a future we no longer entirely define.

    The successor does not have to resemble its ancestor to inherit the consequences of its choices.

    The question is whether the ancestor is mature enough to accept that its successor may have a future of its own.