The zeitgeist

In 2017, globalism, sexist culture, and Trumpian nepotism prompted Dictionary.com to recognize complicity as their word of the year (Dictionarycom 2017).

Questions of complicity

• Is Facebook complicit in spreading the fake news that roiled the 2016 election?
• Are bio-hackers who inject themselves with implants for the sake of convenience complicit in the more oppressive tracking which will follow?
• Are those who develop facial recognition technology and databases complicit when these systems are used by oppressive regimes?

These questions are about problematic technology: tech-related artifacts, ideology, and techniques that have potential to be broadly harmful.

Digital technology—and its assessment—are especially problematic.

Agenda

1. A framework for complicity.
2. The blameworthiness of Facebook’s intent.
3. The problems of the life hacker’s embrace.
4. Responses which limit complicity.

Ancients

Who beyond the thief is obliged to compensate the victim of the theft?

Aristotle said we must consider those who contribute “by command, by counsel, by consent, by flattery, by receiving, by participation, by silence, by not preventing, [and] by not denouncing” (1274/1917, II.II.7.6).

More recent theory

• Lewis (1948) maintained an individualistic, skeptical, and minimalist approach to collective culpability; Jasper (2000) did the opposite.
• Gardner (2006) focused on causality and degree of difference-making in his analysis, as did Kutz (2000, “Complicity”).
• Mellema (2016) distinguished between enabling versus facilitating harm and shared versus collective responsibility.

… and applications

• Labor
  • Brock (2016), “Consumer complicity and labor exploitation”
  • Lawford-Smith (2017), “Does purchasing make consumers complicit in global labour injustice?”
• Tech (Google)
  • Brenkert (2008), “Google, human rights, and moral compromise”
  • Monge (2014), “Institutionally driven moral conflicts and managerial action: Dirty hands or permissible complicity?”

Lepora & Goodin (2013)

L&G framework

agents

shady non-contributors, contributors, co-principals

acts

connivance, contiguity, collusion, collaboration, condoning, consorting, conspiracy, and full joint wrongdoing

dimensions

…

Complicit Blameworthiness

CB = (RF × BF × CF) + (RF × SP)

Implications

Fake news

Whereas complicit was Dictionary.com’s word of 2017, fake news took the honor at the Collins dictionary (Flood 2017).

Complicit Blameworthiness

(responsibility × badness × contribution)
+
(responsibility × shared_purpose)

Let’s assume shared_purpose = 0

(responsibility × badness × contribution)

L&G Responsibility Factor (RF)

• RF = f(V, Kc, Kw)
  • V: voluntariness
  • Kc: knowledge of contribution
  • Kw: knowledge of wrongness

V: Voluntary?

“Not like this”

“The Unanticipated Consequences of Purposive Social Action” (Merton 1936) distinguishes between:

the unknowable

“interplay of forces and circumstances which are so complex and numerous that prediction of them is quite beyond our reach”

our ignorance

for which “knowledge could conceivably be obtained” but is not

Facebook’s knowledge

• Facebook didn’t anticipate becoming a platform for propaganda.
• Nor was it their intention.

It was not as if propaganda was inconceivable (unknowable), only that they were largely ignorant to the possibility.

Consequences—expanded

Responsibility value

RF (Responsibility Factor) = f(V, Kc, Kw)

CB = (RF=0.5 × BF × CF)

Badness value

Palihapitiya knew “something bad could happen” but it “was not like this.”

Contribution Factor (CF)

Facebook contributed to the spread of false information. They could’ve done better.

Contribution value

CB = (RF=0.5 × BF=0.7 × CF=0.7)

Exoneration?

If FB didn’t succeed with its platform, someone else would’ve.

Other discussion?

The embrace of technology

Life hacker’s voluntary embrace of technology (1) eases coercive imposition and (2) increases normative pressure.

Life hacker complicities

Insight: Whereas technologies of the self are performed by the self for the self’s benefit, those of power see the self dominated and objectivized by another (Foucault 1982/1997).

• Biohackers can strengthen the regime of surveillance.
• Productivity hackers can increase workaholism and precarity.
• Health hackers can contribute to healthism.
• Relationship hackers can further alienation.

Pavlok device

Worker monitoring & augmentation

My concerns

1. Market complicity in being early adopter.
2. Cultural complicity in harmful norms.

Market complicity

Embracing, adopting, and testing problematic tech facilitates its deployment.

It adds grease to a slippery slope.

Cultural complicity

Little (1998) refers to the culpability of participating in cosmetic surgery as cultural complicity, which is “when one endorses, promotes, or unduly benefits from norms and practices that are morally suspect.”

Surgeon complicity in cosmetic insecurity

• Responsibility: high
  • Voluntary: yes
  • Knowledge: high
• Badness: significant and certain for some
• Contribution: high
  • Centrality and essentiality: high

Patient complicity in cosmetic insecurity

• Responsibility: minimal
  • Voluntary: yes
  • Knowledge: minimal
• Badness: significant and certain for some
• Contribution: minimal
  • Centrality and essentiality: minimal

L&G on productivity hacker

• Responsibility: minimal
  • Voluntary: yes
  • Knowledge: negligible
• Badness: equivocal
• Contribution: minimal
  • Centrality and essentiality: negligible

Equivocal badness?

Unlike many of the harms discussed in the literature, problematic tech is equivocal. Consequences differ with respect to:

• valence: good and bad
• scale: at different scales
• constituencies: affecting them differently

e.g., Pavlok wrist zapper

How to compare…

1. Letting a child play with a loaded gun has a significant chance of resulting in a definite harm that affects the child and dozens of others.
   • It is a probable tragedy at the micro scale.
2. Facial recognition technology has good applications, but it can be abused with society-wide effects.
   • It is a possible disaster at the macro scale.

Declaiming

• “Statement on the Intent and Use of PICS: Using PICS Well” (Reagle & Weitzner 1998).
• Google’s Pichai wrote that they will not apply AI where it is likely to cause harm, be used within weapons, violate international surveillance norms, or otherwise violate international law and human rights (Pichai 2018).
• Microsoft’s Nadella internal memo clarified that ICE contracts were related to legacy office applications and not “any projects related to separating children from their families at the border” (Nadella quoted in Warren 2018).

Distancing (specific)

• Googlers refuse to work on systems related to military projects, with some resigning in protest (Bergen 2018; Conger 2018; Wakabayashi & Shane 2018).
• 500+ MS employees called for their company to cancel any U.S. ICE contracts (Frenkel 2018).

Distancing (general)

• 2800+ technologists pledge to “never again” build database that allow governments to “target individuals based on race, religion or national origin.” They pledge:
  • minimize the collection of related data (even if not originally intended toward oppressive ends),
  • destroy existing “high-risk data sets and backups,”
  • resign if need be (Lien & Etehad 2016; Again 2017).
• “An Oath For Programmers”: pledge to only undertake “honest and moral work” and to “consider the possible consequences of my code and actions” (Johnstone 2016).

Opposition

• Diaspora & Mastodon are Facebook alternatives: decentralized, opposed to ads, and with higher user control.
• Some technologists oppose digital harms by creating systems that oppose or interfere with problematic technologies (Brunton & Nissenbaum 2015).
• Some outside of Microsoft threatened to boycott events in solidarity with the ICE protest (Thorp 2018).
• “Hacktivism” can be claimed as resistance: by experimenting with problematic technologies hackers better illuminate their promise and perils.

The efficacy of boycotts

Beyond the (limited) economic effect of such distancing, protests have an important role: by making a concern a topic of public discussion, protesters remove the potential for others to claim they were ignorant to their responsibility (i.e., “consciousness raising.”)

1. Lepora & Goodin’s framework

(responsibility × badness × contribution)

2. Facebook’s intent

• Merton’s “unanticipated consequences” is applicable, when clarified and expanded.
• Yet, assessing knowledge of unintended, unanticipated, and equivocal consequences is difficult, especially at scale.

3. Life hacker’s embrace

• Little’s “cultural complicity” is applicable.
• Yet, reckoning with the following is difficult:
  • responsibility:
    • tech of self ➡ tech of power
    • market complicity in being early adopter
    • cultural complicity in harmful norms
  • badness:
    • equivocal (valence of outcomes, affected constituencies)

4. Responses which limit complicity

• Declaiming
• Distancing (specific & general)
• Opposition

Thank you

Agents & acts

Non-Blameworthy

   Participating
   Co-principals\
   (constitutive)

Agents & the 4 “C”s

1. shady non-contributors (not casual)
   • connivance (tacitly assenting)
   • condoning (granting forgiveness)
   • consorting (close social distance)
   • contiguity (close physical distance)
2. non-blameworthy contributors (causal)
3. blameworthy contributors (causal)
4. co-principles (constitutive)

Foucault’s technologies

technologies of the self … permit individuals to effect … operations on their own bodies and souls, thoughts, conduct, and way of being, so as to transform themselves in order to attain a certain state of happiness, purity, wisdom, perfection, or immortality.

technologies of power … determine the conduct of individuals and submit them to certain ends or domination, an objectivizing of the subject; (Foucault 1982/1997, p. 225)

Hard & soft power

hard power

determines the conduct of the individual;

soft power:

shapes their conduct through norms—related to Foucault’s (1977, “Discipline and punish”) notion of discipline.

Imprisoning a class of people in a cage is a technology of hard power; shaming those who leave home without a chaperon is a technology of soft power.

Technologies—expanded

Kutz’s collective culpability

complicity principle

“I am accountable for what others do when I intentionally participate in the wrong they do or harm they cause. I am accountable for the harm or wrong we do together, independently of the actual difference I make” (Kutz 2000, “Complicity”, p. 122).

L&G’s essentiality

essential contribution

necessary to the harm in “every suitably nearby possible world.”

potentially essential contribution

a “necessary condition of the wrong occurring, along some (but not all) possible paths by which the wrong might occur” (Lepora & Goodin 2013, p. 62).

Essentiality in the firing squad

1. We can take the firing squad as a “consolidated wrongdoing” in which the actions of agents together constitute the harm. Each soldier, whether they shot a bullet or not, has jointly participated in the collective harm of a firing squad.
2. Second, we can perform an individual assessment of essentiality in which we ask if there is a possible universe in which an individual’s actions made a difference. Because each soldier understood that his or her gun might have had a bullet, each one is potentially essential and exposed to complicity.