Lecture 11: Ethics of CAI: Values and Norms

What is Ethics?

The field of ethics involves systematizing, defending, and recommending concepts of right and wrong behavior.

Ethical dilemmas are scenarios where there is no obviously good choice.

Values

Values: What is important to us in life

Features of values:

- Priorities guide Actions

- Beliefs linked to Affect (e.g. emotions)

- Refer to Goals

- Transcend Contexts (values span more than 1 context)

- Standards of criteria

- Ordered by Importance

Preferences: Values vs Interests

A preference means a more positive attitude (leaning) toward one alternative over another alternative

Preferences over values vs Preferences over interests:

- An interest is manifested as sustained attention involving cognition of the interest object, accompanying positive impact

- A value is manifested as affective valuation

- Both values and interests influence behavior

- When judgement in preference is based on liking (i.e. attraction), it is an interest; when the basis is importance (i.e. significance or meaning), the preference is a value

Value-Sensitive Design (VSD)

VSD methods are not standalone but are intended to be integrated with other methods and processes for technical design and development

Focus: Value identification, value preference elicitation (want to elicit value for each stakeholder)

Value Identification

Use an existing model of values

E.g. Schwartz Value Model - 10 universal values

Not all values are relevant to all contexts: Context-Specific Values

The meaning of a value depends on the context

Value Identification:

We need to identify and define values in a context: context-specific values

We use Axies, a hybrid methodology for identifying context-specific values.

To be effective for collaborative agents, the methodology needs to be:

- Democratic → Based on population input

- Hybrid → AI + human intuition

- Unbiased → Performed by a group of annotators

- Simple → On a web platform

Axies Overview

Start with value-laden opinion corpus. (No labels)

Methodology:

- involves value annotators (people who can abstract opinions into values)

Exploration: look at different opinions, see which value is applicable to the context

Consolidation: Put them together, ouput a list of values applicable

AI in Axies: help value annotators to efficeintly explore what's in the corpus

Axies input - Word Embeddings

The input of the methodology is population feedback on a context (e.g. tweets).

Words and sentences are represented as points in a n-dimensional space where semantic similarities are preserved.

Axies Methodology - Exploration

In the exploration phase, each annotator independently develops a value list by analyzing the population input.

All input is embedded through sentence embeddings, the next opinion shown to the annotator is selected with the Farthest First Traversal algorithm:

- Start with a random sample.

- Measure distances with other samples to distance sample and choose the one with largest distance.

Then again, find distances with other samples

→ Try to explore the space

→ Even only small samples, cover large areas

Given V as the set of visited points and N the set of novel points, FFT selects the next point as follows:

5 and 11, so it's assigned to the 5.

Axies Methodology - Consolidation (굳히다)

In the consolidation phase, the annotators in a group collaborate to merge their individual value lists

How to know if I know we covered large proportion of compass: 새로운 anotated value들이 안 생길 때까

Example of Values Identified by Axies

We compare the values we elicited (values identifeid by the annotators) with the Schwarz (x axis).

For self-direction, it can be identified to many values for this specific context (need more fine-grained vocabulary for the context).

Value Elicitation

The result of Axies is a list of values relevant to a context.

A concrete application is the elicitation of value preferences through natural language.

Estimating values from natural language allows:

- Humans to express values naturally

- Agents to have meaningful conservations with us

- Agents to learn a user's value profile by aggregating values estimated from sentences

Value Classification

We can treat value classification as a supervised classification.

We need:

- Labelling schema → Output of Axies

- Data → Tweets, surveys, etc

- Labels → Crowd annotations

Then we can just perform a regular supervised classification → Wrong

Label Subjectivity

The subjectivity of the interpretation of values must be considered, especially in collaborative settings.

Textual inputs must be annotated by multiple annotators.

Then, we select the majority annotation to perform supervised classification. → Wrong

That would result in the tyranny of the majority.

We must consider a plurality of opinions, so we must build uncertainty in the model.

Data Representation

Example: three annotators have annotated one datapoint:

A1: [0, 1, 0, 0]

A2: [0, 1, 1, 0]

A3: [0, 0, 0, 1]

Examples of data representation are:

- Majority agreement aggregation: [0, 1, 0, 0]

- Repeated lables: [0, 1, 0, 0], [0, 1, 1, 0], [0, 0, 0, 1]

- Distribution of labels: [0, 0.66, 0.33, 0.33]

Evaluation of Supervised Classifiers

Supervised classifiers are typically evaluated with the F1 score, assuming that a prediction is either correct or wrong.

P = Precision

R = Recall

Evaluation Metrics

Examples of evaluation metrics are:

- Accuracy and (class-weighted) F1 score

- Disagreement-weighted F1 score (items with a lot of disagreement are weighted less than "easy" items)

- Cross-entropy (to compare annotated and predicted distributions of labels)

Takeaways

Artificial agents must understand values operate among humans

Values represent what humans find important in life

Values are subjective and context dependent

Values can be identified and elicited via systematic AI (NLP) techniques

Social Norms

A social norm is a directed social expectation between principals (stakeholders)

Formal representation: Norm_Type(Subject, Object, Antecedent, Consequent)

Subject and Object: The two principals involved in the norm

Antecedent: The condition that brings a norm into force

Consequent: The outcome determining the satisfaction or violation of a norm

Why social norms?

Accountability - A principal can call another to account for its actions

Flexibility - A principal can violate norms (but bears the consequences)

Explainability - Provides an opportunity for principals to explain their actions

Types of social norms