HOUSE_OVERSIGHT_013130.jpg

214 12 The Engineering and Development of Ethics

thinking, this is a cognitive precedent to being able to reconsider an entire belief system, one
that has had contractual logic built atop reflexive adherence that began in early development.
If the initial moral system is viewed as positive and stable, then this cognitive capacity is
seen as dangerous and scary, but if early morality is stunted or warped, then this ability is
seen as enlightened. However, achieving this cognitive stage does not mean one automatically
changes their belief systems, but rather that the mental machinery is in place to consider
the possibilities. Because many people do not reach this level of cognitive development in the
area of moral and ethical thinking, it is associated with negative traits (“moral relativism”
and “flip-flopping”). However, this cognitive flexibility generally leads to more sophisticated and
applicable moral codes, which in turn leads to morality which is actually more stable because
it is built upon extensive and deep consideration rather than simple adherence to reflexive or
rationalized ideologies.

12.4.2 Stages of Development of Empathic Ethics

Complementing Kohlberg’s logic-and-justice-focused approach, Carol Gilligan’s [Gil82] “ethics
of care” model is a moral development theory which posits that empathetic understanding
plays the central role in moral progression from an initial self-centered modality to a socially
responsible one. The ethics of care model is concerned with the ways in which an individual
cares (responds to dilemmas using empathetic responses) about self and others. As shown in
Table 12.3, the ethics of care is broken into the same three primary stage as Kohlberg, but with
a focus on empathetic, emotional caring rather than rationalized, logical principles of justice.

Stage | Principle of Care
Pre-Conventional | Individual Survival
Conventional | Self Sacrifice for the Greater Good
Post-Conventional | Principle of Nonviolence (do not hurt others, or oneself)

Table 12.3: Gilligan’s Stages of the Ethics of Care

For an “ethics of care” approach to be applied in an AGI, the AGI must be capable of internal
simulation of other minds it encounters, in a similar manner to how humans regularly simulate
one another internally. Without any mechanism for internal simulation, it is unlikely that an
AGI can develop any sort of empathy toward other minds, as opposed to merely logically
or probabilistically modeling other agents’ behavior or other minds’ internal contents. In a
CogPrime context, this ties in closely with how CogPrime handles episodic knowledge – partly
via use of an internal simulation world, which is able to play “mental movies” of prior and
hypothesized scenarios within the AGI system’s mind.

However, in humans empathy involves more than just simulation, it also involves sensorimotor
responses, and of course emotional responses – a topic we will discuss in more depth in Appendix
?? where we review the functionality of mirror neurons and mirror systems in the human brains.
When we see or hear someone suffering, this sensory input causes motor responses in us similar
to if we were suffering ourselves, which initiates emotional empathy and corresponding cognitive
processes.

HOUSE_OVERSIGHT_013130