beyond policy lab 2
The Stories That Read Us Back: A Live Lab for Readers on AI and the Wisdom of What We've Read
The group comprised 6 avid readers, who also happen to be members of the Broke Bibliophiles (Chennai): Divya Sridharan, Geetha Srinivasan, Keerthana Ramesh, Solomon Manoj, Sudharsan Haribhaskar, and Vijay Somanath.
The group began with a reflection on a prompt that invited them to share the kinds/genres of books they each read and one thing they’ve learned about from their journey as readers that the world does not talk about enough.
Sudharsan shared that he reads about public policy and how they are conceived and implemented. Reading this genre has helped him understand that a lot of what we see outwardly is only a small part of the bigger picture that comprises a lot on how things are developed, conceived, and implemented. He explained that most of us see the news carry a few words on a public policy, but not a lot of us truly see what goes on behind the scenes. We don’t get to see what goes into making a decision, what the reasoning process looks like, and what the very process of making a decision that can impact millions of lives looks like. We talk about policies that fail, but we don’t see the reasoning for that policy to have come about in the first place.
Keerthana reflected on her reading journey and anchored her share on books around caste, written by people from oppressed caste groups across India. She notes that we don’t talk about caste enough. She noticed the discomfort that emerges in discussing caste in the public sector and while conducting research - both of these being places where bias and discrimination can have an outsized impact. Books on caste show us that we have a lot to understand about different lives through the voices of the people leading those lives, and that they are willing to talk about it and do in fact unpack it in their books in ways that a lot of people don’t even think about on a daily basis.
Geetha drew from one of her recent reads, Becky Chambers’ Psalm for the Wild Built, a speculative, science-fiction read that falls under the ambit of the hopepunk genre. She talked about how it made her think of a general trend of going along with the inventions of the time rather than taking time to think about alternatives. Progress of this kind is narrow, most often the idea of progress that someone decides for the rest to follow. There is very rarely any diversity in our outlook. When we follow the agenda someone sets, they effectively govern the knowledge we have access to - and that is scary.
Reflecting on her journey as a reader and as a mother reading along with her son, Divya noted that she is sitting with the lack of empathy in the world at the moment. Reading has helped her recognise the value of empathy in dismantling longstanding hierarchies like the caste system. As technology continues to evolve, we are at risk of carrying biases into the future if we don’t care to cultivate empathy.
Solomon reflected on his journey as a fiction reader, and how that has showed him that very few people truly understand politics. Many have the privilege to dissociate from politics and refuse to engage, and that in itself is a cost. Politics plays a very key role in technology and how it is built and shaped. We often find ourselves in echo chambers and don’t necessarily look at other views and perspectives.
Vijay’s journey of reading a wide variety of books - fiction, non-fiction, sci-fi and more helped him see that the American way of doing business is inimical to society and has shaped the entire world, technology and AI included. When he read a book called “100 Companies that Changed the World,” which captured the journey from 1901 to 1990, he found that it was initially the European companies that shaped the world until the World Wars, and then the American companies took over. He found it jarring to note how they think about human life and put profit above everything. The “Titans of Industry” as they are known consistently push capitalism forward as if to say it is an inevitable model. We are seeing this with AI as well, where more and more people are being asked to integrate and use AI in their work, effectively teaching the tool their work so it can replace them. Reflecting out loud, he mused that we are being sold the idea that if we don’t like AI, we will be left behind in the dust - but the irony is that we aren’t particularly soaring right now, given the state of the economy (inflation) and environment (rampant degradation). While he noted that AI is a phenomenal piece of technology, the way it is being developed and talked about is unacceptable.
Naming and framing the problem
The group began with some scene setting that looked at the military origins of AI and how those origins inform the mindsets and worldviews that build AI into existence. With the recognition of profit-generating capacities, capitalism took over and baked AI into everyday life, manifesting in the many ways in which we encounter it in the world around us. The group also recognised that the form of AI we are most familiar with - generative AI and large language models - are only one among myriads of other applications. They made space for the fact that there are promises and perils, and equally, potentials to augment both, with AI. Recognising some of the newest events, the group noted that there is an active push for AI to become ubiquitous and that powerful AI companies are moving toward gatekeeping knowledge. They referenced the example of Google search turning out to make AI the default while relegating the original search results to a page titled “Web.” They also noted Anthropic’s recent move to buy rare books, digitize them, and then burn them.
In looking at the perils of AI, the group reflected on the way AI is built, using data that may not necessarily have been drawn without consent, and causing greater surveillance and privacy violations. They also noted the massive environmental costs associated with AI use, and the escalating challenges to the environment with the expansion of data centres, water consumption, and large-scale displacement. The potential for outer space and the deep sea becoming the sites for new data centres was also recognised. As a tool, the group noted, AI has many aspects that resonate with colonialism, capitalism, and human rights violations. The exploitation of vulnerable labour, particularly women from oppressed caste and indigenous backgrounds in the majority world, has been one of the key components underscoring the emergence of AI.
And yet, the group made space for the vast potential and benefits, as well. Noting that AI has levelled the playing field by making access available where there was none, the group made place for the real-world applications in education, healthcare, and welfare that showed meaningful results with the deployment of AI. They named the benefits that have emerged for particular fields like agriculture, meteorology, education, and protections for vulnerable groups in conflict zones as worthy of attention.
With this in place, the group was invited to bear three overarching, guiding questions in mind in reflecting on the next piece on governance:
Which books already told us this was coming and who dismissed it as fiction?
What does the literature you love tell you about what happens when a few people control the tools that shape everyone’s reality?
What kind of readers does AI governance clearly need?
Creating a repository of wisdom
The group was invited to identify one book, character, concept, value, narrative device, or practice from their reading tradition that relates to governance. During this segment, the group named the following books as catalysts for their thinking:
Never Let Me Go by Kazuo Ishiguro (focusing on the abuse of power and a key character’s failure to act responsibly)
The Kite Runner by Khaled Hosseini (focusing on what we owe to each other in the shape of personal accountability after power is abused)
The Giver by Lois Lowry (focusing on the inability to compromise with unfair decisions and inviting a reflection on who gets to decide what, why, and for whom)
Jiddu Krishnamurthy’s works in general (focusing on the invitation to be an observer and do what is necessary rather than act from extrinsic motivations)
The Truth About Me: A Hijra Life Story by Revati (focusing on how the personal is always political, naming the gap in modelling public behaviours in private)
To Kill a Mockingbird by Harper Lee (focusing on the choice a character made to speak truth to power in an ethos that made no space for it)
Code Dependent by Madhumita Murgia (focusing on the call for accountability, inclusivity, and moral responsibility in building, using, and engaging with AI)
Annihilation of Caste by Dr B R Ambedkar (focusing on caste-based inequities and how deeply they are embedded in society, underpinning all aspects of life)
Dune by Frank Herbert (focusing on the abuse of power and the myopic hyperfocus on short-term/immediate gains while losing sight of enduring harm)
Brave New World by Aldous Huxley AND Fahrenheit 451 by Ray Bradbury (focusing on the role of an uninformed, uncurious citizenry in enabling power to run rampant)
Flash Boys by Michael Lewis (focusing on how far wealthy capitalists will go for the sake of profit)
The Jataka Tales and Panchatantra (focusing on lessons emphasizing the harms of discrimination, and calling for empathy and diligence).
Drawing from these books, the ideas that emerged in this segment reflected on the following themes:
The need to avoid pedestalization, in order to reconcile with the reality that a tool can both harm and empower
The role of voice in a narrative: Who enjoys dominance over whom?
The role of power and how an abuse of power is consistently underlying the way AI is built, developed, designed, marketed, and eventually used.
The importance of different perspectives and voices in the way a story is told
The value of transparency that emerges in how a story is told, with varying layers and nuances being presented
The need for accountability and justice in repairing harm
The importance of world-building by presenting relevant information
The limitation of reflecting on harm as temporary collateral damage
The harm that ensues from flattening a whole narrative, lived reality, or complex experience
Building a container for governance
Drawing from the repository of co-documented wisdom, the group proceeded to frame specifics that could be deployed in governing AI. The group reflected on who gets to speak for AI systems, what they’re obliged to disclose, what people have a right to know about how AI is shaping what they see, read, and offer, whose story is AI most likely to erase, flatten, or never tell and how we can write them back, and finally, what a governance framework look like if it was designed for consequences that unfold over generations rather than right now alone.
The group began with a reflection on the need to do away with pedestalization, which makes it more practically possible to recognize the harm and benefit of a tool. Reflecting on how we have been sold the idea that AI will replace everything or that AI is a know-it-all, one of the participants shared an experiment where tested how AI reviewed a book she had read. She noted that the emerging almost entirely lacked perspective and depth. Reading a book, on the other hand, presents you with a multidimensional understanding, surfaces various ideas and perspectives, and helps you ground yourself in reflection deeper. This dissonance in itself proves that AI should not be given a status anymore than that of a tool, like an encyclopaedia, for example, and the onus is on us to gather varying perspectives and ideas instead of taking what AI says as the absolute truth.
The group also reflected on how many around the world are not entirely aware of the harms that AI is capable of. Political leaders and policymakers have the capacity to reach the masses, and have the power to shape and deliver messages to these masses. Drawing from the wisdom of world-building in fiction, the group reflected on how it is on those in positions of power to be open about the harms and benefits of AI - while someone on social media might have already learned about the level of harm AI use brings to the environment, someone from a region marked by the lack of access to technology might not relate to this technology to start with. Those in power owe transparency in sharing the promises and perils of AI.
Naming the harm that ensues from the flattening of narratives in AI, the group noted that AI is most likely to flatten and possibly entirely erase the narratives, voices, and views of people who seek to hold power accountable. At the moment, there is an assumption that our dynamic with AI is a case of “man versus machine,” when in reality, it is man versus man - with the more powerful emerging dominant in both, controlling the narrative and access to the technology. One of the participants noted reading the line “A machine craves order, it is humans that create chaos.” Reflecting on recent news where factory workers were being trained to use AI, the group noted that anyone whose livelihood is adversely affected by the deployment of AI in a factory would be against its use, but their voices are seldom acknowledged or accounted for.
Writing these voices back into the picture to resist erasure must be a collective activity, and not the freedom of an individual or a small group to shape. The role of voice is critical, because if the hand of the victor holds the pen, we risk losing sight of the voices that must be centred. This always does come with a difficult question around tradeoffs, as the risk of visibility is also accompanied by the burden of surveillance and harm as a result of such visibility. A community is best suited to decide agentically on whether or not it wants to be written back, and how.
The group also reflected on the importance of diverse perspectives across the AI lifecycle. Thinking back to the Internet in the pre-AI days, one participant talked about the loud and imperfect space it was, with all our voices and opinions being presented in their original form. AI tools created by billionaires are re-consolidating our voices and compressing them into a homogenous, uniform stream that has no emotion or empathy, and that is sanitized into sounding perfect. People are pressured into performing this level of perfection by turning to AI tools, which is another form of colonial imposition because it requires us to speak in a language that isn’t our own. Governance in this regard would look like bringing in the voices of those who are affected most by the technology in question.
The group cautioned against the metricization of human engagement. Sharing examples of social media platforms that first emerged as content sharing sites and eventually becoming monetized channels that turn hobbies and interests into metrics to optimize for, the group reflected on how these decisions suck the soul out of everything.
Closely similar to the nature of public libraries, one participant reflected on the potential of looking at AI as a public utility, which would mean that communities - whole societies and people at large - will helm the responsibility of governing it as a utility. It would also mean that people would have the opportunity to opt in or out of usage, and access would be a matter of convenience. This would be entirely unlike the current framework of governance where a small pool of companies are controlling AI for the entire world.