Security without Dystopia: Structured Transparency
Emerging technologies and innovative governance can reshape the landscape of security, privacy, and international cooperation.
The trend towards increased monitoring seems unstoppable. Internet devices are creating an ever-denser web of data collection systems, while fast moving technologies are creating new threats. The specters of rogue AI and bioterrorism are driving calls for national and global monitoring systems.
Dystopian scenarios are easy to imagine, a world where secretive agencies exceed their legitimate authority, privacy evaporates, and centralized power can easily monitor and suppress any challenge. A common view suggests that we have only bad options: accept invasive surveillance for security, or protect privacy and accept vulnerability. This view, however, makes strong but tacit assumptions about the structure of transparency — about how information from monitoring systems is collected, processed, filtered, and applied, and about how these systems are governed.
The concept of structured transparency1 invites us to view diverse controls on information flow, not in isolation, but as building blocks for systems tailored to serve legitimate security needs without granting abusable powers. In the context of civil society, this includes detecting threats while minimizing intrusion on the private sphere; in the international sphere, this includes enabling states to reveal specific kinds of information while protecting secrets.
What if we could design systems that detect genuine threats without invading personal privacy? What if oversight mechanisms could ensure that monitoring systems adhere to agreed patterns of information governance? What if states could negotiate new kinds of transparency-based assurances that help reduce military tension?
To understand possibilities, we need to consider how emerging technologies can be composed in new ways, and how the coming explosion of computational and physical capabilities can open further possibilities. With this understanding, we can reshape the conversation around surveillance and security. We can demand solutions that serve legitimate needs without sacrificing our fundamental values.
Building blocks for transparency structures
Structured transparency can employ a toolkit of complementary mechanisms that, in combination, can change the landscape of possibilities. These building blocks form the foundation of transparency structures, enabling refined control over potentially problematic information flows. Let’s consider key governance tools and their applications.
Redaction and anonymization: These mechanisms strip identifying details from data while maintaining its analytical utility. They’re crucial in public health, allowing data aggregation without compromising individual privacy. Advanced techniques could provide evidence of specific actions without revealing identities, reserving de-anonymization for situations with appropriate permissions.
Rate control: A transparency structure that offers only a limited “query budget” constrains the scale of potential abuse and encourages a focus on mission-critical information gathering. Rate controls can block mass surveillance.
Query filtering: Systems that restrict the scope of queries can enable specific inquiries while blocking privacy-infringing questions. In some cases (for example, in formally initiated criminal investigations) a system might allow more invasive queries under strict oversight.
Time windows: By implementing time windows for access, transparency structures can provide data for addressing active threats while sealing historical records, or can delay access to tactically relevant information while enabling later scrutiny as part of an oversight process.
Pattern discovery: AI algorithms operating on rich information in a secure environment can investigate specific patterns of concern and then report the results through filtered channels without exposing all the clues that led to their discovery. Thus, advanced AI models with access to raw data repositories could apply their full capabilities to detect threats, while reporting only relevant, potentially anonymized evidence for further evaluation.
Revocable permissions: This governance mechanism allows overseers to respond to abuse by restricting transparency, or to unlock capabilities for deeper investigation when threats are detected. Multi-key permissions, implemented through encryption and key-sharing techniques can ensure that approval for sensitive actions requires agreement by multiple parties.
Transparency structures in practice
Flow-control mechanisms are already used to craft transparency structures for a range of purposes:
Financial monitoring: Banks use pattern discovery on anonymized transaction data to detect and report fraud and money laundering.
Health information exchanges: Allow data sharing among providers, using redaction to protect individual privacy while enabling population-level pattern detection and analysis.
Census data: Restrictions on the types and numbers of queries together with anonymization enable the use of data while mathematically guaranteeing individual privacy.
Internet service provider data retention: May restrict queries to metadata, and employ time windows and rate control for access; governance mechanisms control investigation permissions.
GDPR-compliant analytics: Implements flow controls through secure repositories, revocable permissions, redaction and anonymization, query restrictions, rate controls, and time windows to balance business analytics with privacy rights.
Note that the governmental entities that employ these transparency structures are themselves subject to structures of transparency and governance.
Putting pieces together
By integrating multiple mechanisms, we can create flow-control architectures tailored to specific needs and constraints. Consider a potential transparency structure designed to enable detection and investigation of potential domestic security threats (perhaps plans for hijackings, bombs, or bioterrorism) while reliably precluding mass surveillance:
AI systems operating inside an information-security boundary have access to rich information sources.
AI-based pattern discovery can follow any clues, yet can report only specific threat-identifiers.
Flow controls restrict human investigators to permissible, case-focused queries.
Permissible queries are limited in number and scope, ensuring focused investigation rather than mass data collection.
Substantial evidence of a serious threat can unlock access to broader information, a process similar to issuing a subpoena.
Focused information is delivered to decision-makers for potential action.
This sketch illustrates how advanced information flow controls could be combined to build transparency structures that satisfy both security and privacy concerns in civil society.
Well-designed transparency structures can also create new options in international relations. Even hostile states frequently benefit from mutual revelation of selected information, provided that other information remains secret. For example, the US and Soviet Union agreed to:
Allow each other to fly surveillance aircraft over their military facilities, subject to constraints on rates, time windows, and (through restrictions on cameras and sensors) query types.
Allow on-site inspections of missile sites and production facilities, subject to controls on the frequency and nature of access (in other words, restrictions on query types and rates).
Expose the contents of their missile silos to satellite observation, revealing sensitive but limited information that enabled confirmation of compliance to arms-control agreements.
A wider, more flexible range of options for structuring transparency can widen the range of negotiable risk-reducing agreements. In conjunction with other verifiable technologies, structured transparency could contribute to a shift toward a more defensive defense,2 reducing reliance on threats and potentially providing an exit from the classic security dilemma.
In considering the scope of potential agreements, it is important to consider how transparency mechanisms can help bootstrap trust in the transparency systems themselves: The greater the transparency of systems design, implementation, and operation, the greater the potential for assurances among mutually distrustful actors.
Ambitious Goals and Implementation Challenges
The mechanisms of structured transparency open doors to transformative applications, but realizing their potential will require confronting challenges that increase with scale and scope:
Complexity: Extensive systems will require more complex software infrastructure, posing implementation challenges.
Reliability: Systems must operate consistently, and their potential failure modes and attack surfaces increase with scale.
Trust: The design, implementation, and operation of critical systems must satisfy demands for trust.
Understanding these challenges and potential solutions is crucial. It allows us to set realistic expectations and distinguish between near-term achievable goals and long-term aspirations.
Structured transparency is both a concept and a direction for policy and technical research. Information flow controls are already ubiquitous, but the concept of structured transparency invites us to view their mechanisms as building blocks, to seek better mechanisms, and to think more creatively about the structures we can build from them.
And looking forward, the concept of structured transparency invites us to consider how AI-enabled capabilities could enable us to craft better building blocks, build more ambitious architectures, and create new possibilities for overcoming the critical challenges ahead. The future of civil society and the human world may depend on it.
Related posts:
“Beyond Privacy Trade-offs with Structured Transparency” arXiv, 2020 (pdf)
“Defensive Defense: A Better Way to Protect US Allies in Asia,” Washington Quarterly, Winter 2020, pdf.