Difference between revisions of "Binary Governance: Lessons from the GDPR’s Approach to Algorithmic Accountability (Q2401)"

From Wikibase Personal data
Jump to navigation Jump to search
(‎Created claim: comment (P126): Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.)
(‎Created claim: comment (P126): Governing algorithmic decision-making is hard. [] in significant part, solving the governance problem is hard because we cannot agree on why to regulate. A growing literature now focuses on regulating alg...)
 
(25 intermediate revisions by 2 users not shown)
Property / comment
 +
Limiting the choices we see—whether by failing to show opportunities or by offering only bad options—limits our freedom to make choices.
Property / comment: Limiting the choices we see—whether by failing to show opportunities or by offering only bad options—limits our freedom to make choices. / rank
 +
Normal rank
Property / comment
 +
Failing to be transparent about the fact that individuals are being targeted or the reasons why they are targeted itself may threaten autonomy. Secret profiling and decision-making can lead to manipulation. Without knowing how we are being targeted or why, we can be manipulated into making choices that are not autonomous at all.
Property / comment: Failing to be transparent about the fact that individuals are being targeted or the reasons why they are targeted itself may threaten autonomy. Secret profiling and decision-making can lead to manipulation. Without knowing how we are being targeted or why, we can be manipulated into making choices that are not autonomous at all. / rank
 +
Normal rank
Property / comment
 +
Concerns about autonomy and the potential for manipulation, to a great degree, motivated the indignation around Cambridge Analytica’s targeted manipulation of U.S. voters prior to the 2016 election (and motivated the California legislature to enact the California Consumer Privacy Act in 2018).
Property / comment: Concerns about autonomy and the potential for manipulation, to a great degree, motivated the indignation around Cambridge Analytica’s targeted manipulation of U.S. voters prior to the 2016 election (and motivated the California legislature to enact the California Consumer Privacy Act in 2018). / rank
 +
Normal rank
Property / comment
 +
the dominant rationale for regulating algorithmic decision-making is an instrumental (or consequentialist) rationale. We should regulate algorithms, this reasoning goes, to prevent the consequences of baked-in bias and discrimination and other kinds of error.
Property / comment: the dominant rationale for regulating algorithmic decision-making is an instrumental (or consequentialist) rationale. We should regulate algorithms, this reasoning goes, to prevent the consequences of baked-in bias and discrimination and other kinds of error. / rank
 +
Normal rank
Property / comment
 +
The instrumental rationale for regulating algorithmic decision-making counsels that regulation should try to correct these problems, often by using systemic accountability mechanisms, such as ex ante technical requirements, audits, or oversight boards, to do so.
Property / comment: The instrumental rationale for regulating algorithmic decision-making counsels that regulation should try to correct these problems, often by using systemic accountability mechanisms, such as ex ante technical requirements, audits, or oversight boards, to do so. / rank
 +
Normal rank
Property / comment
 +
The other two rationales for regulating algorithmic decision-making, however, suggest that systemic oversight is not enough. Both dignitary and justificatory reasoning point towards including individual rights.
Property / comment: The other two rationales for regulating algorithmic decision-making, however, suggest that systemic oversight is not enough. Both dignitary and justificatory reasoning point towards including individual rights. / rank
 +
Normal rank
Property / comment
 +
The dignitary argument posits that an individual human being should be respected as a whole, free person. Being subjected to algorithmic decision-making threatens individuals’ personhood by objectifying them. Objectification defeats autonomy: the freedom to make choices, be offered opportunities, or otherwise move freely through the world.
Property / comment: The dignitary argument posits that an individual human being should be respected as a whole, free person. Being subjected to algorithmic decision-making threatens individuals’ personhood by objectifying them. Objectification defeats autonomy: the freedom to make choices, be offered opportunities, or otherwise move freely through the world. / rank
 +
Normal rank
Property / comment
 +
The first [version of the dignitary argument], largely European, criticism of algorithmic decision- making is that allowing a decision about humans to be made by a machine inherently treats humans as objects, showing deep, inherent disrespect for peoples’ humanity.
Property / comment: The first [version of the dignitary argument], largely European, criticism of algorithmic decision- making is that allowing a decision about humans to be made by a machine inherently treats humans as objects, showing deep, inherent disrespect for peoples’ humanity. / rank
 +
Normal rank
Property / comment
 +
A second type of dignitary concern appeals, however, across cultural divides. Automatically making decisions based on what categories an individual falls into—that is, what correlations can be shown between an individual and others—can fail to treat that individual as an individual.
Property / comment: A second type of dignitary concern appeals, however, across cultural divides. Automatically making decisions based on what categories an individual falls into—that is, what correlations can be shown between an individual and others—can fail to treat that individual as an individual. / rank
 +
Normal rank
Property / comment
 +
[The third type of] dignitary concerns include concerns (more familiar to Americans) about individual autonomy. Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.
Property / comment: [The third type of] dignitary concerns include concerns (more familiar to Americans) about individual autonomy. Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have. / rank
 +
Normal rank
Property / comment
 +
The third category of concerns about algorithmic decision-making, justificatory concerns, aims to ensure the legitimacy of a decisional system. Justificatory concerns resonate strongly with calls for rule of law.
Property / comment: The third category of concerns about algorithmic decision-making, justificatory concerns, aims to ensure the legitimacy of a decisional system. Justificatory concerns resonate strongly with calls for rule of law. / rank
 +
Normal rank
Property / comment
 +
Human decision makers have the capacity to expand decisional context when it seems unfair to ignore information a machine might not know is relevant (“You are speeding on the way to the hospital”)
Property / comment: Human decision makers have the capacity to expand decisional context when it seems unfair to ignore information a machine might not know is relevant (“You are speeding on the way to the hospital”) / rank
 +
Normal rank
Property / comment
 +
Collaborative governance is, at best, a highly tailored, site-calibrated regulatory system that aims to pull inputs from, obtain buy-in from, and affect the internal institutional structures and decision-making heuristics of the private sector, while maintaining the legitimacy, efficacy and public-interest orientation of public governance.
Property / comment: Collaborative governance is, at best, a highly tailored, site-calibrated regulatory system that aims to pull inputs from, obtain buy-in from, and affect the internal institutional structures and decision-making heuristics of the private sector, while maintaining the legitimacy, efficacy and public-interest orientation of public governance. / rank
 +
Normal rank
Property / comment
 +
individual narratives about discrimination or bias may be more palatable to the public than agency-produced reports or statistics, and could feed back into collaborative governance by contributing to ongoing policy conversations about the broader governance regime.
Property / comment: individual narratives about discrimination or bias may be more palatable to the public than agency-produced reports or statistics, and could feed back into collaborative governance by contributing to ongoing policy conversations about the broader governance regime. / rank
 +
Normal rank
Property / comment
 +
Intriguingly the GDPR’s absence of public-facing and stakeholder-facing accountability suggests that individual transparency rights may have to serve a crucial accountability role in its system of collaborative governance. Thus, even for those focused on instrumental [] goals, individual rights in the GDPR may be necessary for producing effective systemic regulation, too
Property / comment: Intriguingly the GDPR’s absence of public-facing and stakeholder-facing accountability suggests that individual transparency rights may have to serve a crucial accountability role in its system of collaborative governance. Thus, even for those focused on instrumental [] goals, individual rights in the GDPR may be necessary for producing effective systemic regulation, too / rank
 +
Normal rank
Property / comment
 +
The GDPR for the most part envisions collaboration as taking place between regulators and regulated private parties, to the exclusion of third parties, such as civil society or external experts. This threatens both the substance and legitimacy of the regime. To some extent, this design flaw may reflect the relative weakness of civil society in the EU.
Property / comment: The GDPR for the most part envisions collaboration as taking place between regulators and regulated private parties, to the exclusion of third parties, such as civil society or external experts. This threatens both the substance and legitimacy of the regime. To some extent, this design flaw may reflect the relative weakness of civil society in the EU. / rank
 +
Normal rank
Property / comment
 +
An impact assessment, in other words, is supposed to be a tripartite conversation between a regulated entity, the regulator, and third parties such as impacted persons or civil society organizations. In the GDPR, it is largely used internally or, at most, in conversation with regulators.
Property / comment: An impact assessment, in other words, is supposed to be a tripartite conversation between a regulated entity, the regulator, and third parties such as impacted persons or civil society organizations. In the GDPR, it is largely used internally or, at most, in conversation with regulators. / rank
 +
Normal rank
Property / comment
 +
It is possible that despite the GDPR’s lack of public transparency and input by third parties, impact assessments and maybe draft codes of conduct will be made available to the public through other means. Freedom of information law might be used to obtain public disclosure.
Property / comment: It is possible that despite the GDPR’s lack of public transparency and input by third parties, impact assessments and maybe draft codes of conduct will be made available to the public through other means. Freedom of information law might be used to obtain public disclosure. / rank
 +
Normal rank
Property / comment
 +
[high-quality collaborative governance in the absence of true public transparency] will depend on the ability of NGOs and other policy advocates to harness the GDPR’s system of individual transparency rights coupled possibly with freedom of information laws to obtain both enforcement and transparency.
Property / comment: [high-quality collaborative governance in the absence of true public transparency] will depend on the ability of NGOs and other policy advocates to harness the GDPR’s system of individual transparency rights coupled possibly with freedom of information laws to obtain both enforcement and transparency. / rank
 +
Normal rank
Property / comment
 +
To accomplish meaningful oversight, both NGOs and the press will need to link individual disclosures into politically effective group narratives, revealing what is going on over an algorithmic decision-making system as a whole. This will be costly and time- consuming and will involve much coordination. Still, it may be possible over time.
Property / comment: To accomplish meaningful oversight, both NGOs and the press will need to link individual disclosures into politically effective group narratives, revealing what is going on over an algorithmic decision-making system as a whole. This will be costly and time- consuming and will involve much coordination. Still, it may be possible over time. / rank
 +
Normal rank
Property / comment
 +
because the GDPR relies on collaborative governance, its attempts at individual transparency must serve a dual role. Absent policy changes, the GDPR’s individual transparency provisions will need to serve both individual dignitary and justificatory ends and as a crucial element of structured accountability in its collaborative governance regime.
Property / comment: because the GDPR relies on collaborative governance, its attempts at individual transparency must serve a dual role. Absent policy changes, the GDPR’s individual transparency provisions will need to serve both individual dignitary and justificatory ends and as a crucial element of structured accountability in its collaborative governance regime. / rank
 +
Normal rank
Property / comment
 +
If aggregated by third parties—the media or civil society—individual stories can trigger soft accountability mechanisms, like market responses or naming-and-shaming. The question is whether the individual transparency produced in the GDPR’s individual rights vindication provisions will be adequate to serve those functions in its collaborative governance regime.
Property / comment: If aggregated by third parties—the media or civil society—individual stories can trigger soft accountability mechanisms, like market responses or naming-and-shaming. The question is whether the individual transparency produced in the GDPR’s individual rights vindication provisions will be adequate to serve those functions in its collaborative governance regime. / rank
 +
Normal rank
Property / comment
 +
Governing algorithmic decision-making is hard. [] in significant part, solving the governance problem is hard because we cannot agree on why to regulate. A growing literature now focuses on regulating algorithmic decision-making in order to solve problems such as error, bias, and discrimination, but ignores or brushes over legitimate dignitary and justificatory reasons for regulating.
Property / comment: Governing algorithmic decision-making is hard. [] in significant part, solving the governance problem is hard because we cannot agree on why to regulate. A growing literature now focuses on regulating algorithmic decision-making in order to solve problems such as error, bias, and discrimination, but ignores or brushes over legitimate dignitary and justificatory reasons for regulating. / rank
 +
Normal rank
Property / DOI
 +
Property / DOI: 10.2139/ssrn.3351404 / rank
 +
Normal rank

Latest revision as of 13:01, 9 December 2019

No description defined
Language Label Description Also known as
English
Binary Governance: Lessons from the GDPR’s Approach to Algorithmic Accountability
No description defined

    Statements

    0 references
    0 references
    Secret profiles and decisions made based on secret profiling can threaten personhood and thus dignity by proscribing active individual involvement in the construction of this objectified version of the self.
    0 references
    the “ ‘data shadows’ . . . threaten to usurp the constitutive authority of the physical self despite their relatively attenuated and often misleading nature”
    0 references
    Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.
    0 references
    Limiting the choices we see—whether by failing to show opportunities or by offering only bad options—limits our freedom to make choices.
    0 references
    Failing to be transparent about the fact that individuals are being targeted or the reasons why they are targeted itself may threaten autonomy. Secret profiling and decision-making can lead to manipulation. Without knowing how we are being targeted or why, we can be manipulated into making choices that are not autonomous at all.
    0 references
    Concerns about autonomy and the potential for manipulation, to a great degree, motivated the indignation around Cambridge Analytica’s targeted manipulation of U.S. voters prior to the 2016 election (and motivated the California legislature to enact the California Consumer Privacy Act in 2018).
    0 references
    the dominant rationale for regulating algorithmic decision-making is an instrumental (or consequentialist) rationale. We should regulate algorithms, this reasoning goes, to prevent the consequences of baked-in bias and discrimination and other kinds of error.
    0 references
    The instrumental rationale for regulating algorithmic decision-making counsels that regulation should try to correct these problems, often by using systemic accountability mechanisms, such as ex ante technical requirements, audits, or oversight boards, to do so.
    0 references
    The other two rationales for regulating algorithmic decision-making, however, suggest that systemic oversight is not enough. Both dignitary and justificatory reasoning point towards including individual rights.
    0 references
    The dignitary argument posits that an individual human being should be respected as a whole, free person. Being subjected to algorithmic decision-making threatens individuals’ personhood by objectifying them. Objectification defeats autonomy: the freedom to make choices, be offered opportunities, or otherwise move freely through the world.
    0 references
    The first [version of the dignitary argument], largely European, criticism of algorithmic decision- making is that allowing a decision about humans to be made by a machine inherently treats humans as objects, showing deep, inherent disrespect for peoples’ humanity.
    0 references
    A second type of dignitary concern appeals, however, across cultural divides. Automatically making decisions based on what categories an individual falls into—that is, what correlations can be shown between an individual and others—can fail to treat that individual as an individual.
    0 references
    [The third type of] dignitary concerns include concerns (more familiar to Americans) about individual autonomy. Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.
    0 references
    The third category of concerns about algorithmic decision-making, justificatory concerns, aims to ensure the legitimacy of a decisional system. Justificatory concerns resonate strongly with calls for rule of law.
    0 references
    Human decision makers have the capacity to expand decisional context when it seems unfair to ignore information a machine might not know is relevant (“You are speeding on the way to the hospital”)
    0 references
    Collaborative governance is, at best, a highly tailored, site-calibrated regulatory system that aims to pull inputs from, obtain buy-in from, and affect the internal institutional structures and decision-making heuristics of the private sector, while maintaining the legitimacy, efficacy and public-interest orientation of public governance.
    0 references
    individual narratives about discrimination or bias may be more palatable to the public than agency-produced reports or statistics, and could feed back into collaborative governance by contributing to ongoing policy conversations about the broader governance regime.
    0 references
    Intriguingly the GDPR’s absence of public-facing and stakeholder-facing accountability suggests that individual transparency rights may have to serve a crucial accountability role in its system of collaborative governance. Thus, even for those focused on instrumental [] goals, individual rights in the GDPR may be necessary for producing effective systemic regulation, too
    0 references
    The GDPR for the most part envisions collaboration as taking place between regulators and regulated private parties, to the exclusion of third parties, such as civil society or external experts. This threatens both the substance and legitimacy of the regime. To some extent, this design flaw may reflect the relative weakness of civil society in the EU.
    0 references
    An impact assessment, in other words, is supposed to be a tripartite conversation between a regulated entity, the regulator, and third parties such as impacted persons or civil society organizations. In the GDPR, it is largely used internally or, at most, in conversation with regulators.
    0 references
    It is possible that despite the GDPR’s lack of public transparency and input by third parties, impact assessments and maybe draft codes of conduct will be made available to the public through other means. Freedom of information law might be used to obtain public disclosure.
    0 references
    [high-quality collaborative governance in the absence of true public transparency] will depend on the ability of NGOs and other policy advocates to harness the GDPR’s system of individual transparency rights coupled possibly with freedom of information laws to obtain both enforcement and transparency.
    0 references
    To accomplish meaningful oversight, both NGOs and the press will need to link individual disclosures into politically effective group narratives, revealing what is going on over an algorithmic decision-making system as a whole. This will be costly and time- consuming and will involve much coordination. Still, it may be possible over time.
    0 references
    because the GDPR relies on collaborative governance, its attempts at individual transparency must serve a dual role. Absent policy changes, the GDPR’s individual transparency provisions will need to serve both individual dignitary and justificatory ends and as a crucial element of structured accountability in its collaborative governance regime.
    0 references
    If aggregated by third parties—the media or civil society—individual stories can trigger soft accountability mechanisms, like market responses or naming-and-shaming. The question is whether the individual transparency produced in the GDPR’s individual rights vindication provisions will be adequate to serve those functions in its collaborative governance regime.
    0 references
    Governing algorithmic decision-making is hard. [] in significant part, solving the governance problem is hard because we cannot agree on why to regulate. A growing literature now focuses on regulating algorithmic decision-making in order to solve problems such as error, bias, and discrimination, but ignores or brushes over legitimate dignitary and justificatory reasons for regulating.
    0 references

    Identifiers

    0 references