Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Epistemic fragmentation poses a threat to the governance of online targeting

Nature Machine Intelligence volume 3, pages 466–472 (2021)Cite this article

Subjects

Abstract

Online targeting isolates individual consumers, causing what we call epistemic fragmentation. This phenomenon amplifies the harms of advertising and inflicts structural damage to the public forum. The two natural strategies to tackle the problem of regulating online targeted advertising, increasing consumer awareness and extending proactive monitoring, fail because even sophisticated individual consumers are vulnerable in isolation, and the contextual knowledge needed for effective proactive monitoring remains largely inaccessible to platforms and external regulators. The limitations of both consumer awareness and of proactive monitoring strategies can be attributed to their failure to address epistemic fragmentation. We call attention to a third possibility that we call a civic model of governance for online targeted advertising, which overcomes this problem, and describe four possible pathways to implement this model.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Models of governance for online targeted advertising.

Similar content being viewed by others

References

  1. Solove, D. J. Introduction: privacy self-management and the consent dilemma. Harv. Law Rev. 126, 1880–1903 (2013).

    Google Scholar 

  2. Cohen, J. E. What privacy is for. Harv. Law Rev. 126, 1904–1933 (2013).

    Google Scholar 

  3. Waldman, A. E. Privacy law’s false promise. Wash. Univ. Law Rev. 97, 0773 (2020).

    Google Scholar 

  4. Cate, F. H. & Mayer-Schönberger, V. Notice and consent in a world of Big Data. Int. Data Priv. Law 3, 67–73 (2013).

    Article  Google Scholar 

  5. Mantelero, A. Personal data for decisional purposes in the age of analytics: from an individual to a collective dimension of data protection. Comput. Law Secur. Rev. 32, 238–255 (2016).

    Article  Google Scholar 

  6. Zuboff, S. Big other: surveillance capitalism and the prospects of an information civilization. J. Inf. Technol. 30, 75–89 (2015).

    Article  Google Scholar 

  7. Véliz, C. Privacy and digital ethics after the pandemic. Nat. Electron. 4, 10–11 (2021).

    Article  Google Scholar 

  8. Wachter, S. & Mittelstadt, B. D. A right to reasonable inferences: re-thinking data protection law in the age of Big Data and AI. Columbia Bus. Law Rev. 2019, 494–620 (2019).

    Google Scholar 

  9. Zuboff, S. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (PublicAffairs, 2019).

  10. Choi, H., Mela, C. F., Balseiro, S. R. & Leary, A. Online display advertising markets: a literature review and future directions. Inf. Syst. Res. 31, 556–575 (2020).

    Article  Google Scholar 

  11. Advertising and Marketing Communications Code (ICC, 2018); https://cms.iccwbo.org/content/uploads/sites/3/2018/09/icc-advertising-and-marketing-communications-code-int.pdf

  12. Update Report into Adtech and Real Time Didding (ICO, 2019); https://ico.org.uk/media/about-the-ico/documents/2615156/adtech-real-time-bidding-report-201906.pdf

  13. AI Barometer Report (CDEI, 2020); https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/894170/CDEI_AI_Barometer.pdf

  14. Online Harms White Paper (UK Department for Culture, Media and Sport & Home Office, 2019).

  15. Sahni, N. S. & Nair, H. Does Advertising Serve as a Signal? Evidence from Field Experiments in Mobile Search (SSRN, 2018); https://doi.org/10.2139/ssrn.2721468

  16. Sahni, N. S. & Zhang, C. Search Advertising and Information Discovery: Are Consumers Averse to Sponsored Messages? (SSRN, 2020) https://doi.org/10.2139/ssrn.3441786

  17. Kingsley, S., Wang, C., Mikhalenko, A., Sinha, P. & Kulkarni, C. Auditing digital platforms for discrimination in economic opportunity advertising. Preprint at https://arxiv.org/abs/2008.09656 (2020).

  18. Using Technology for Good: Annual Report 2019 (ASA/CAP, 2020); https://www.asa.org.uk/uploads/assets/68dd32b5-ae6a-4993-820a3ff8f1163b8e/ASA-CAP-2019-Annaul-Report-Full-Version-Singles.pdf

  19. Internet Advertising Revenue Report: Full Year 2019 Results & Q1 2020 Revenues (IAB/PWC, 2020); https://www.iab.com/wp-content/uploads/2020/05/FY19-IAB-Internet-Ad-Revenue-Report_Final.pdf

  20. Dave, P. Social media giants warn of AI moderation errors as coronavirus empties offices. Reuters (16 March 2020).

  21. Newton, C. The coronavirus is forcing tech giants to make a risky bet on AI. The Verge https://www.theverge.com/interface/2020/3/18/21183549/coronavirus-content-moderators-facebook-google-twitter (2020).

  22. Heilweil, R. Facebook is flagging some coronavirus news posts as spam. Vox https://www.vox.com/recode/2020/3/17/21183557/coronavirus-youtube-facebook-twitter-social-media (2020).

  23. Ali, M. et al. Discrimination through optimization: how Facebook’s ad delivery can lead to skewed outcomes. Proc. ACM Hum. Comput. Interact. 3, 1–30 (2019).

    Article  Google Scholar 

  24. Datta, A., Datta, A., Makagon, J., Mulligan, D. K. & Tschantz, M. C. Discrimination in online advertising: a multidisciplinary inquiry. Proc. Mach. Learn. Res. 81, 20–34 (2018).

  25. Kim, P. T. & Scott, S. Discrimination in online employment recruiting symposium: law, technology, and the organization of work. St. Louis Univ. Law J 63, 93–118 (2018).

    Google Scholar 

  26. Sweeney, L. Discrimination in online ad delivery. ACM Queue 11, 36 (2013).

    Article  Google Scholar 

  27. Wachter, S. Affinity Profiling and Discrimination by Association in Online Behavioural Advertising (SSRN, 2019); https://papers.ssrn.com/abstract=3388639

  28. EASA Best Practice Recommendation on Online Behavioural Advertising (EASA, 2016).

  29. YourAdChoices (Digital Advertising Alliance); https://youradchoices.com/about

  30. YourOnlineChoices (EDAA); https://www.youronlinechoices.com/uk/about-behavioural-advertising

  31. Kim, T., Barasz, K. & John, L. K. Why am I seeing this ad? The effect of ad transparency on ad effectiveness. J. Consum. Res 45, 906–932 (2019).

    Article  Google Scholar 

  32. Andreou, A. et al. Investigating ad transparency mechanisms in social media: a case study of facebook’s explanations. In Proc. 2018 Network and Distributed System Security Symposium (Internet Society, 2018); https://doi.org/10.14722/ndss.2018.23191

  33. Hern, A. & Ledegaard, F. H. Children ‘interested in’ gambling and alcohol, according to Facebook. The Guardian (9 October 2019).

  34. Harnessing New Technology to Tackle Irresponsible Gambling Ads Targeted at Children (ASA, 2019); https://www.asa.org.uk/news/harnessing-new-technology-gambling-ads-children.html

  35. ASA System Submission to the Women and Equalities Committee Inquiry on Body Image (ASA, 2020).

  36. Medina, D. J. Hermeneutical injustice and polyphonic contextualism: social silences and shared hermeneutical responsibilities. Soc. Epistemol. 26, 201–220 (2012).

    Article  Google Scholar 

  37. Medina, J. The Epistemology of Resistance: Gender and Racial Oppression, Epistemic Injustice, and the Social Imagination (Oxford Univ. Press, 2013).

  38. Fricker, M. in The Epistemic Dimensions of Ignorance (eds Peels, R. & Blaauw, M.) 160–177 (Cambridge Univ. Press, 2016); https://doi.org/10.1017/9780511820076.010

  39. Angwin, J., Larson, J., Mattu, S. & Kirchner, L. Machine bias. ProPublica https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing?token=LrqwtD3z1Jth8ag9cay6c0yzKoghtu9C (2016).

  40. Wachter, S., Mittelstadt, B. & Russell, C. Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI (SSRN, 2020); https://doi.org/10.2139/ssrn.3547922

  41. Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. On the dangers of stochastic parrots: can language models be too big? In Proc. 2021 ACM Conference on Fairness, Accountability, and Transparency 610–623 (ACM, 2021); https://doi.org/10.1145/3442188.3445922

  42. Mittelstadt, B. Auditing for transparency in content personalization systems. Int. J. Commun. 10, 12 (2016).

    Google Scholar 

  43. Chotiner, I. The underworld of online content moderation. The New Yorker (5 July 2019); https://www.newyorker.com/news/q-and-a/the-underworld-of-online-content-moderation

  44. Gillespie, T. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media (Yale Univ. Press, 2018).

  45. Strubell, E., Ganesh, A. & McCallum, A. Energy and policy considerations for deep learning in NLP. In Proc. 57th Annual Meeting of the Association for Computational Linguistics 3645–3650 (ACL, 2019).

  46. Gray, M. L. & Suri, S. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass (Houghton Mifflin Harcourt, 2019).

  47. Keegan, J. Introducing ‘split screen’. The Markup https://themarkup.org/citizen-browser/2021/03/11/introducing-split-screen (2020).

  48. Split screen: how different are Americans’ Facebook feeds? The Markup https://themarkup.org/citizen-browser/2021/03/11/split-screen?feed=biden_trump (2021).

  49. Ad Library (Facebook); https://www.facebook.com/ads/library/?active_status=all&ad_type=political_and_issue_ads&country=GB (accessed May 2021).

  50. Ads Transparency (Twitter, Inc.); https://business.twitter.com/en/help/ads-policies/product-policies/ads-transparency.html (accessed May 2021).

  51. Nguyen, C. T. Echo chambers and epistemic bubbles. Episteme 17, 141–161 (2020).

    Article  Google Scholar 

  52. Pariser, E. The Filter Bubble: How the New Personalized Web is Changing What We Read and How We Think (Penguin, 2012).

  53. Bruns, A. Filter bubble. Internet Policy Rev. 8, 1–14 (2019).

    Article  Google Scholar 

  54. Borgesius, F. J. Z. et al. Should we worry about filter bubbles? Internet Policy Rev. 5, 1–16 (2016).

    Google Scholar 

  55. Online Platforms and Digital Advertising: Market Study Final Report (CMA, 2020); https://assets.publishing.service.gov.uk/media/5efc57ed3a6f4023d242ed56/Final_report_1_July_2020_.pdf

  56. Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on a Single Market For Digital Services (Digital Services Act) and Amending Directive 2000/31/EC COM/2020/825 final (European Commission, 2020); https://eur-lex.europa.eu/legal-content/en/TXT/?qid=1608117147218&uri=COM%3A2020%3A825%3AFIN

  57. Toubiana, V., Narayanan, A., Boneh, D., Nissenbaum, H. & Barocas, S. Adnostic: privacy preserving targeted advertising. In Proc. Network and Distributed System Symposium (NDSS, 2010).

  58. Charting a course towards a more privacy-first web. Google https://blog.google/products/ads-commerce/a-more-privacy-first-web/ (2021).

  59. Taylor, L. in Group Privacy: New Challenges of Data Technologies (eds Taylor, L. et al.) 13–36 (Springer International, 2017); https://doi.org/10.1007/978-3-319-46608-8_2

  60. Mittelstadt, B. From individual to group privacy in big data analytics. Phil. Technol. 30, 475–494 (2017).

    Article  Google Scholar 

  61. Bygrave, L. A. Data Protection Law: Approaching its Rationale, Logic and Limits (Kluwer Law International, 2002).

  62. Mantelero, A. in Group Privacy: New Challenges of Data Technologies (eds Taylor, L. et al.) 139–158 (Springer International, 2017); https://doi.org/10.1007/978-3-319-46608-8_8

  63. Cohen, J. & Fung, A. in Digital Technology and Democratic Theory (eds Landemore, H. et al.) 23–61 (Univ. Chicago Press, 2021).

  64. Landemore, H. Beyond the fact of disagreement? The epistemic turn in deliberative. Democracy Soc. Epistemol. 31, 277–295 (2017).

    Article  Google Scholar 

  65. Estlund, D. & Landemore, H. in The Oxford Handbook of Deliberative Democracy (eds Bächtiger, A. et al.) 112–131 (Oxford Univ. Press, 2018); https://doi.org/10.1093/oxfordhb/9780198747369.013.26

  66. Dryzek, J. S. et al. The crisis of democracy and the science of deliberation. Science 363, 1144–1146 (2019).

    Article  Google Scholar 

  67. Statt, N. Facebook will remove 5,000 ad targeting categories to prevent discrimination. The Verge https://www.theverge.com/2018/8/21/17764480/facebook-ad-targeting-options-removal-housing-racial-discrimination (2018).

  68. Dayen, D. Ban targeted advertising. The New Republic (10 April 2018).

  69. Lewis, R. A. & Rao, J. M. The unfavorable economics of measuring the returns to advertising. Q. J. Econ. 130, 1941–1973 (2015).

  70. Angwin, J. Auditing the algorithms of disinformation. The Markup https://www.getrevue.co/profile/themarkup/issues/auditing-the-algorithms-of-disinformation-284735 (2020).

  71. The Citizen Browser Project—auditing the algorithms of disinformation. The Markup https://themarkup.org/citizen-browser (2020).

  72. How We Built a facebook inspector. The Markup https://themarkup.org/citizen-browser/2021/01/05/how-we-built-a-facebook-inspector (2021).

  73. Bloustein, E. J. Group privacy: the right to huddle. Rutgers Camden Law J. 8, 219–283 (1976).

    Google Scholar 

  74. Bloustein, E. J. & Pallone, N. J. Individual and Group Privacy (Routledge, 2018); https://doi.org/10.4324/9781351319966

  75. van der Sloot, B. in Group Privacy: New Challenges of Data Technologies (eds Taylor, L. et al.) 197–224 (Springer International, 2017); https://doi.org/10.1007/978-3-319-46608-8_11

  76. Horwitz, J. Facebook seeks shutdown of NYU research project into political ad targeting. Wall Street Journal (23 October 2020).

  77. Naughton, J. Facebook has good reasons for blocking research into political ad targeting. The Guardian (31 October 2020).

  78. 2018 European Trends in Advertising Complaints, Copy Advice and Pre-clearance (EASA, 2018); https://www.easa-alliance.org/sites/default/files/2018%20European%20Trends%20in%20Advertising%20Complaints%2C%20Copy%20Advice%20and%20Pre-clearance.pdf

  79. How Facebook ads get approved. Facebook for Business https://www.facebook.com/business/a/ad-review-process (accessed 9 June 2021).

  80. HM Government Communication Service (GCS, 2020); https://gcs.civilservice.gov.uk/about-us/what-we-do/

  81. HM Government Communication Plan 2019/20 (GCS, 2019); https://communication-plan.gcs.civilservice.gov.uk/

  82. Merrill, J. B. Does facebook still sell discriminatory ads? The Markup https://themarkup.org/ask-the-markup/2020/08/25/Does-Facebook-Still-Sell-Discriminatory-Ads (2020).

Download references

Acknowledgements

This work of the Governance of Emerging Technologies research programme at the Oxford Internet Institute has been supported by British Academy Postdoctoral Fellowship grant number PF2\180114 and grant number PF\170151, the Luminate/Omidyar Group and the Miami Foundation.

Author information

Author notes

  1. Christopher Russell

    Present address: Amazon, Tübingen, Germany

Authors and Affiliations

  1. Future of Humanity Institute, University of Oxford, Oxford, UK

    Silvia Milano

  2. Oxford Internet Institute, University of Oxford, Oxford, UK

    Brent Mittelstadt & Sandra Wachter

  3. The Alan Turing Institute, London, UK

    Christopher Russell

Authors
  1. Silvia Milano
    View author publications

    Search author on:PubMed Google Scholar

  2. Brent Mittelstadt
    View author publications

    Search author on:PubMed Google Scholar

  3. Sandra Wachter
    View author publications

    Search author on:PubMed Google Scholar

  4. Christopher Russell
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Silvia Milano.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Machine Intelligence thanks Jathan Sadowski and the other, anonymous, reviewer(s) for their contribution to peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Milano, S., Mittelstadt, B., Wachter, S. et al. Epistemic fragmentation poses a threat to the governance of online targeting. Nat Mach Intell 3, 466–472 (2021). https://doi.org/10.1038/s42256-021-00358-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s42256-021-00358-3

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing