Binary Governance: Lessons from the GDPR’s Approach to Algorithmic Accountability (Q2401)
Revision as of 12:53, 9 December 2019 by Podehaye (talk | contribs) (Created claim: Property:P126: because the GDPR relies on collaborative governance, its attempts at individual transparency must serve a dual role. Absent policy changes, the GDPR’s individual transparency provisions will need to ser...)
No description defined
| Language | Label | Description | Also known as |
|---|---|---|---|
| English |
Binary Governance: Lessons from the GDPR’s Approach to Algorithmic Accountability
|
No description defined
|
Statements
Secret profiles and decisions made based on secret profiling can threaten personhood and thus dignity by proscribing active individual involvement in the construction of this objectified version of the self.
0 references
the “ ‘data shadows’ . . . threaten to usurp the constitutive authority of the physical self despite their relatively attenuated and often misleading nature”
0 references
Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.
0 references
Limiting the choices we see—whether by failing to show opportunities or by offering only bad options—limits our freedom to make choices.
0 references
Failing to be transparent about the fact that individuals are being targeted or the reasons why they are targeted itself may threaten autonomy. Secret profiling and decision-making can lead to manipulation. Without knowing how we are being targeted or why, we can be manipulated into making choices that are not autonomous at all.
0 references
Concerns about autonomy and the potential for manipulation, to a great degree, motivated the indignation around Cambridge Analytica’s targeted manipulation of U.S. voters prior to the 2016 election (and motivated the California legislature to enact the California Consumer Privacy Act in 2018).
0 references
the dominant rationale for regulating algorithmic decision-making is an instrumental (or consequentialist) rationale. We should regulate algorithms, this reasoning goes, to prevent the consequences of baked-in bias and discrimination and other kinds of error.
0 references
The instrumental rationale for regulating algorithmic decision-making counsels that regulation should try to correct these problems, often by using systemic accountability mechanisms, such as ex ante technical requirements, audits, or oversight boards, to do so.
0 references
The other two rationales for regulating algorithmic decision-making, however, suggest that systemic oversight is not enough. Both dignitary and justificatory reasoning point towards including individual rights.
0 references
The dignitary argument posits that an individual human being should be respected as a whole, free person. Being subjected to algorithmic decision-making threatens individuals’ personhood by objectifying them. Objectification defeats autonomy: the freedom to make choices, be offered opportunities, or otherwise move freely through the world.
0 references
The first [version of the dignitary argument], largely European, criticism of algorithmic decision- making is that allowing a decision about humans to be made by a machine inherently treats humans as objects, showing deep, inherent disrespect for peoples’ humanity.
0 references
A second type of dignitary concern appeals, however, across cultural divides. Automatically making decisions based on what categories an individual falls into—that is, what correlations can be shown between an individual and others—can fail to treat that individual as an individual.
0 references
[The third type of] dignitary concerns include concerns (more familiar to Americans) about individual autonomy. Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.
0 references
The third category of concerns about algorithmic decision-making, justificatory concerns, aims to ensure the legitimacy of a decisional system. Justificatory concerns resonate strongly with calls for rule of law.
0 references
Human decision makers have the capacity to expand decisional context when it seems unfair to ignore information a machine might not know is relevant (“You are speeding on the way to the hospital”)
0 references
Collaborative governance is, at best, a highly tailored, site-calibrated regulatory system that aims to pull inputs from, obtain buy-in from, and affect the internal institutional structures and decision-making heuristics of the private sector, while maintaining the legitimacy, efficacy and public-interest orientation of public governance.
0 references
individual narratives about discrimination or bias may be more palatable to the public than agency-produced reports or statistics, and could feed back into collaborative governance by contributing to ongoing policy conversations about the broader governance regime.
0 references
Intriguingly the GDPR’s absence of public-facing and stakeholder-facing accountability suggests that individual transparency rights may have to serve a crucial accountability role in its system of collaborative governance. Thus, even for those focused on instrumental [] goals, individual rights in the GDPR may be necessary for producing effective systemic regulation, too
0 references
The GDPR for the most part envisions collaboration as taking place between regulators and regulated private parties, to the exclusion of third parties, such as civil society or external experts. This threatens both the substance and legitimacy of the regime. To some extent, this design flaw may reflect the relative weakness of civil society in the EU.
0 references
An impact assessment, in other words, is supposed to be a tripartite conversation between a regulated entity, the regulator, and third parties such as impacted persons or civil society organizations. In the GDPR, it is largely used internally or, at most, in conversation with regulators.
0 references
It is possible that despite the GDPR’s lack of public transparency and input by third parties, impact assessments and maybe draft codes of conduct will be made available to the public through other means. Freedom of information law might be used to obtain public disclosure.
0 references
[high-quality collaborative governance in the absence of true public transparency] will depend on the ability of NGOs and other policy advocates to harness the GDPR’s system of individual transparency rights coupled possibly with freedom of information laws to obtain both enforcement and transparency.
0 references
To accomplish meaningful oversight, both NGOs and the press will need to link individual disclosures into politically effective group narratives, revealing what is going on over an algorithmic decision-making system as a whole. This will be costly and time- consuming and will involve much coordination. Still, it may be possible over time.
0 references
because the GDPR relies on collaborative governance, its attempts at individual transparency must serve a dual role. Absent policy changes, the GDPR’s individual transparency provisions will need to serve both individual dignitary and justificatory ends and as a crucial element of structured accountability in its collaborative governance regime.
0 references