{"entities":{"Q2401":{"pageid":25280,"ns":120,"title":"Item:Q2401","lastrevid":69164,"modified":"2019-12-09T13:01:24Z","type":"item","id":"Q2401","labels":{"en":{"language":"en","value":"Binary Governance: Lessons from the GDPR\u2019s Approach to Algorithmic Accountability"}},"descriptions":{},"aliases":{},"claims":{"P3":[{"mainsnak":{"snaktype":"value","property":"P3","datavalue":{"value":{"entity-type":"item","numeric-id":210,"id":"Q210"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q2401$68386354-40ba-b45c-4aad-146d213263bb","rank":"normal"}],"P15":[{"mainsnak":{"snaktype":"value","property":"P15","datavalue":{"value":"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3351404","type":"string"},"datatype":"url"},"type":"statement","id":"Q2401$ca534c80-41f0-59ee-8b13-af24f2920aad","rank":"normal"}],"P121":[{"mainsnak":{"snaktype":"value","property":"P121","datavalue":{"value":{"entity-type":"item","numeric-id":2003,"id":"Q2003"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q2401$e1112c04-40b5-8807-b380-243f729efb68","rank":"normal"}],"P110":[{"mainsnak":{"snaktype":"value","property":"P110","datavalue":{"value":{"entity-type":"item","numeric-id":1949,"id":"Q1949"},"type":"wikibase-entityid"},"datatype":"wikibase-item"},"type":"statement","id":"Q2401$9d8b3de3-481c-dd37-0cee-d41695ad819e","rank":"normal"}],"P126":[{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Secret profiles and decisions made based on secret profiling can threaten personhood and thus dignity by proscribing active individual involvement in the construction of this objectified version of the self.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$d3bcb33e-4aac-4402-52c2-d64f5c50dd1b","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"the \u201c \u2018data shadows\u2019 . . . threaten to usurp the constitutive authority of the physical self despite their relatively attenuated and often misleading nature\u201d","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$e852853b-40e2-77d9-9a91-dc6790f720aa","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$c4c230f7-4271-9218-513b-bb996e9ba509","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Limiting the choices we see\u2014whether by failing to show opportunities or by offering only bad options\u2014limits our freedom to make choices.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$ed4b3050-4388-d4b3-dcf5-7a0edda31e7d","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Failing to be transparent about the fact that individuals are being targeted or the reasons why they are targeted itself may threaten autonomy. Secret profiling and decision-making can lead to manipulation. Without knowing how we are being targeted or why, we can be manipulated into making choices that are not autonomous at all.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$3d6f7f48-4d80-e1ee-b580-082878ee990e","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Concerns about autonomy and the potential for manipulation, to a great degree, motivated the indignation around Cambridge Analytica\u2019s targeted manipulation of U.S. voters prior to the 2016 election (and motivated the California legislature to enact the California Consumer Privacy Act in 2018).","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$8c8281da-44a2-9947-851d-12258c9744d3","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"the dominant rationale for regulating algorithmic decision-making is an instrumental (or consequentialist) rationale. We should regulate algorithms, this reasoning goes, to prevent the consequences of baked-in bias and discrimination and other kinds of error.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$47c394ac-470f-037f-456a-42d1b2477577","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"The instrumental rationale for regulating algorithmic decision-making counsels that regulation should try to correct these problems, often by using systemic accountability mechanisms, such as ex ante technical requirements, audits, or oversight boards, to do so.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$da0953a8-4f84-cf86-4e45-74e680b0a26a","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"The other two rationales for regulating algorithmic decision-making, however, suggest that systemic oversight is not enough. Both dignitary and justificatory reasoning point towards including individual rights.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$88ec80ca-4439-6fc6-fbe3-7fe899fc88d0","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"The dignitary argument posits that an individual human being should be respected as a whole, free person. Being subjected to algorithmic decision-making threatens individuals\u2019 personhood by objectifying them. Objectification defeats autonomy: the freedom to make choices, be offered opportunities, or otherwise move freely through the world.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$ffc7eb61-480e-5fc4-8c57-57af0527d456","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"The first [version of the dignitary argument], largely European, criticism of algorithmic decision- making is that allowing a decision about humans to be made by a machine inherently treats humans as objects, showing deep, inherent disrespect for peoples\u2019 humanity.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$c4c61d7a-49cf-9e83-bacf-f5e09c0ae86a","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"A second type of dignitary concern appeals, however, across cultural divides. Automatically making decisions based on what categories an individual falls into\u2014that is, what correlations can be shown between an individual and others\u2014can fail to treat that individual as an individual.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$40ca261e-4ddc-2e95-c9d9-1880398b79f2","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"[The third type of] dignitary concerns include concerns (more familiar to Americans) about individual autonomy. Algorithmic decision-making founded on individual profiling limits the choices and, thus, the freedom a person will have.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$24a14b76-4a96-a768-21b3-9612d941f778","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"The third category of concerns about algorithmic decision-making, justificatory concerns, aims to ensure the legitimacy of a decisional system. Justificatory concerns resonate strongly with calls for rule of law.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$06624f1b-4ca8-c2e7-45f3-fc62042fa56c","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Human decision makers have the capacity to expand decisional context when it seems unfair to ignore information a machine might not know is relevant (\u201cYou are speeding on the way to the hospital\u201d)","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$34bc78a2-4b50-d67d-44f5-4a8bd6b9ba09","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Collaborative governance is, at best, a highly tailored, site-calibrated regulatory system that aims to pull inputs from, obtain buy-in from, and affect the internal institutional structures and decision-making heuristics of the private sector, while maintaining the legitimacy, efficacy and public-interest orientation of public governance.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$fe2cd471-42f7-cf81-7231-fd2a1fc0479d","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"individual narratives about discrimination or bias may be more palatable to the public than agency-produced reports or statistics, and could feed back into collaborative governance by contributing to ongoing policy conversations about the broader governance regime.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$8d8bd953-4d4e-5c93-6698-0b0d47f77827","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Intriguingly the GDPR\u2019s absence of public-facing and stakeholder-facing accountability suggests that individual transparency rights may have to serve a crucial accountability role in its system of collaborative governance. Thus, even for those focused on instrumental [] goals, individual rights in the GDPR may be necessary for producing effective systemic regulation, too","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$0ce20aee-4711-51f9-3dd1-5defd6e5d823","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"The GDPR for the most part envisions collaboration as taking place between regulators and regulated private parties, to the exclusion of third parties, such as civil society or external experts. This threatens both the substance and legitimacy of the regime. To some extent, this design flaw may reflect the relative weakness of civil society in the EU.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$8d03204a-4282-8934-49df-bc74c3c0929e","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"An impact assessment, in other words, is supposed to be a tripartite conversation between a regulated entity, the regulator, and third parties such as impacted persons or civil society organizations. In the GDPR, it is largely used internally or, at most, in conversation with regulators.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$2b563987-479b-cfbd-d4e3-8fd5c3dde2af","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"It is possible that despite the GDPR\u2019s lack of public transparency and input by third parties, impact assessments and maybe draft codes of conduct will be made available to the public through other means. Freedom of information law might be used to obtain public disclosure.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$ecbcd836-4b6d-dcae-1574-d68ec57014c4","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"[high-quality collaborative governance in the absence of true public transparency] will depend on the ability of NGOs and other policy advocates to harness the GDPR\u2019s system of individual transparency rights coupled possibly with freedom of information laws to obtain both enforcement and transparency.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$b021a374-4523-501d-49ca-7f357cf9e96f","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"To accomplish meaningful oversight, both NGOs and the press will need to link individual disclosures into politically effective group narratives, revealing what is going on over an algorithmic decision-making system as a whole. This will be costly and time- consuming and will involve much coordination. Still, it may be possible over time.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$0bd56018-4b88-fbcd-7d87-69bcc737472f","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"because the GDPR relies on collaborative governance, its attempts at individual transparency must serve a dual role. Absent policy changes, the GDPR\u2019s individual transparency provisions will need to serve both individual dignitary and justificatory ends and as a crucial element of structured accountability in its collaborative governance regime.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$16d10c25-4736-46d7-54ee-88478f82d8d6","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"If aggregated by third parties\u2014the media or civil society\u2014individual stories can trigger soft accountability mechanisms, like market responses or naming-and-shaming. The question is whether the individual transparency produced in the GDPR\u2019s individual rights vindication provisions will be adequate to serve those functions in its collaborative governance regime.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$4775536e-4200-c63e-e291-a6e92bc70216","rank":"normal"},{"mainsnak":{"snaktype":"value","property":"P126","datavalue":{"value":"Governing algorithmic decision-making is hard. [] in significant part, solving the governance problem is hard because we cannot agree on why to regulate. A growing literature now focuses on regulating algorithmic decision-making in order to solve problems such as error, bias, and discrimination, but ignores or brushes over legitimate dignitary and justificatory reasons for regulating.","type":"string"},"datatype":"string"},"type":"statement","id":"Q2401$f4fb5987-48d3-8697-6b40-af1ae15011de","rank":"normal"}],"P62":[{"mainsnak":{"snaktype":"value","property":"P62","datavalue":{"value":"10.2139/ssrn.3351404","type":"string"},"datatype":"external-id"},"type":"statement","id":"Q2401$306dd8c5-47ac-1dff-1b18-7ac245be525f","rank":"normal"}]},"sitelinks":{}}}}