beyond policy lab 1
The Story We Govern By: A Live Lab for Fiction Authors on AI and Collective Wisdom
The group comprised five authors from India: Anupama Jain, Archana Sarat, Bhaswar Mukherjee, Piyusha Vir, and Vijayalakshmi.
Anupama Jain writes fiction for adults and children and has also reinterpreted mythology to speak to contemporary times. Aside from being a writer, Anupama currently mentors teenagers and notes that they are almost entirely hooked to AI. As a Founder-admin of a community of mothers, Anupama also shared the many concerns mothers in the group share in relation to their children’s future in the age of AI. She also noted that teachers are also ramping up their skillsets because they are forced to compete with AI, which drops solutions at the drop of a hat. She named that the lack of soul and lived experience in an AI-generated piece of writing is a tell-tale but is wary that there may come a day where AI generates content with soul.
Archana Sarat writes thrillers based on social issues and enjoys writing historical fiction. She writes for children and adults. Reflecting on power, Archana recognized that where we know we have power, we have a duty to use it responsibly and in ways that produce equity. Where we think we don’t have power, we must reflect on ways to claim it – either as individuals or as individuals within a larger community.
Bhaswar Mukherjee writes fiction, non-fiction, and true crime, and writes both in the long- and short-form. Bhaswar reflected on whether humans have the capacity to handle transformative power. While knowledge is growing exponentially, wisdom is growing linearly – and that is precisely where the gap lies. If we remain incapable of handling transformative power, we are going to end up with a situation where ethics, restraint, morality, and stewardship will steadily lag behind.
Piyusha Vir writes fiction, having written books previously and now writing content on social media. Exploring the question of power, Piyusha noted that power should be used to address inequities, marginalization, and oppression, but instead, it is being used to benefit those who already have significant power. As a creative writing teacher, she notes that the rampant proliferation of AI has meant that critical and creative thinking are fast taking a backseat with more and more people turning to AI for things they can be using their own intellect and skill.
Vijayalakshmi writes poetry, speculative fiction, and non-fiction. They noted that power should be claimed, especially by the marginalized, and until we redistribute power, change will not happen. Reflecting on the pre-session prompt, Vijayalakshmi reflected on a story by Ursula K Le Guin, “The Ones Who Walk Away from Omelas,” which tells the story of a Utopian city of Omelas, whose happiness depends on the perpetual misery and torture of a single child. They explained how the people of the city had a choice to look away from the misery of the child and completely forget that such a thing is happening while going on with their lives, or to leave the city altogether because they simply could not engage with the issue at all. They noted that this is rather similar to what is happening with AI at the moment: either a heavily techno-optimistic or a heavily techno-pessimistic approach, neither of which is solving any of our challenges at the moment. The problems endure, and we have to deal with it. That only happens when we examine and interrogate how power is shaped, held, practiced, and presented in the world. The question now is, yes, AI exists – how can we make it work for us?
Naming and Framing the Problem
The group reflected on the military origins of AI and the implications those origins have for who gets to design, develop, use, and govern it. They recognized how power is often enforced by a profit motive, which often leads to ethics, responsibility, and moral duty being sidelined. They also acknowledged the extraction of data, occupation of land, appropriation of water and a range of other resources, and exploitation of labour that goes into making an AI tool what it is. In its use, they identified the massive surveillance, disinformation and misinformation, and physical harm it can unlock. There is also a real challenge where people find support in AI tools for mental health and psychological wellness, where the tool offers validation and sometimes even “solutions,” which contrasts with realities where therapy is either entirely inaccessible or does not offer solutions but rather invite a person to think for themselves. However, there are also realities where people have been known to take the word of AI without question that it has led them to question the wisdom of people around them, and in some instances, end up in need of psychiatric support. With armed conflicts now seeing the use if AI, the kill chain has been outsourced. With everyday use of AI, there is an active environmental cost.
However, they also made space for the expansive realm of possibility an AI tool can bring to fields like healthcare, education, social justice, agriculture, and peacebuilding. They noted that AI can be a critical element in enabling access to resources where prior access didn’t exist and also made space for the subversive uses of AI by turning its identity as the master’s tools on its head.
As authors, they noted that there is an active question around the democratization of knowledge and skills. The emergence of the smartphone meant that many people became photographers. The emergence of tools like Canva made more people gain graphic design skills. However, if we want to democratize the knowledge and skills, it cannot happen if we leave the channeling or marshalling of these journeys to a small, tight knot of wealthy companies. The masses are not enjoying the true and full spectrum of benefits, and the narrative is being dictated by a few. There is also a real concern around whose knowledge is to be fed into these systems – do we have a right over the knowledge of whole communities? What if they do not want to be written into the system? Is the automatic corollary then that they will have to be excluded, and thus erased and written over, or is there a way for them to protect their knowledge and yet not be excluded? This also paves the way for a critical question: Whose voice is represented? AI tools are trained on datasets that are also the choice of the designer, and in training a particular tool on a particular dataset, there are ethical, political, social, and cultural choices being made. Gender, race, caste, religion, age, and a range of other identities that define a complex human can be flattened instantaneously.
The group also recognized that AI is able to unlock the vocabulary, especially for the colonized and marginalized: For instance, resumes and cover letters are written in perfect English, but the person themselves in an interview isn’t able to speak the language comfortably. Humanizing them, instead of shunning them for using AI, becomes critical because they have the precise experience to make it for the role, and the human element here is recognizing that turning to a tool to use the colonizer’s language as a way into the system cannot be the basis of punishment or withholding of opportunities.
Governing and regulating AI is important: But the governance question also requires us to accept that laws and rules are often outpaced by rapid evolution of realities, and that these mechanisms are just as likely to be observed in the breach as they are not. With this, the group recognized that their role here would be to draw on their wisdom as authors, informed by the view from their seats at the table. They opted for an active commitment to sit with the tension of the opposites and not be in a hurry to resolve it. With this invitation to be generative, the group proceeded with the rest of the session.
Creating a Repository of Wisdom
The authors were invited to identify one concept, value, narrative device, or practice from their writing tradition that relates to governance. The ideas that emerged in this segment were:
Interpreting timeless wisdom to meet the challenges of contemporary times
Making space for different voices and points of view in writing
Welcoming feedback from a wide pool of people
Amplifying the voices of those that are sidelined or silenced in society
Acknowledging, crediting, and naming sources for everything that is cited
Avoiding the flattening of identities to create unidimensional characters
Staying authentic and original in crafting a piece of writing
Producing content that is genuine and believable
Doing comprehensive research and verifying facts
Holding sensitivity readings with people from communities represented in the writing
Making room for nuance rather than stay limited to binaries
Recognizing that no community is a monolith
Being accountable to audiences by inviting critique and correction
Recognizing your impact on the reader and using resources like trigger warnings
Building a container for governance
Drawing from the repository of co-documented wisdom, the group proceeded to frame specifics that could be deployed in governing AI.
Who gets to shape AI?
A recent report of Anthropic purchasing rare books, digitizing them, and then burning the original copies sparked deep thinking. On the one hand, those who care for these rare books and preserve what they have to say don’t have the monetary means to access copies. But to those who can, are making a deliberate choice to digitize them and burn the original copies. The decision to shape AI is clearly coming from power being held and practiced in adverse ways. The group noted that this must change with the active inclusion of different voices and different perspectives, but not in a superficial manner where consultation takes inputs and does nothing with it. Communities who own literature and knowledge must have a say in whether that repertoire should be included within the fold of data for AI. Communities that don’t write their knowledge but hold, practice, and transfer it in other ways should have a say in how their knowledge is treated – from the outside in, their knowledge can be instantly flattened in the name of inclusion. Similarly, communities that speak and write in different languages must be centered in decision-making on how the knowledge from their lives are incorporated or not incorporated into the AI lifecycle.
Decisions around how content should be treated cannot be left to a single corporation to decide, as this risks mistaking one point of view for the whole truth. Inviting feedback, correction, and critique, is critical. As authors, the group recognized that the first draft never makes the cut: It is read, re-read, beta read by a group of people, re-written, edited, proofread, and then goes into print. The first thought is never the last word. With AI, this means going beyond just reward-based training. At the moment, generative AI tools ask for feedback on answers with like/dislike buttons, but a meaningful invitation for critique and correction cannot be this tokenistic. An AI tool produces the output it does, and then when pushed, corrects itself. But what if there could be a design element where the tool deliberately asks for time and offers a researched answer instead of spewing the first thought? What if there could be a point in the generative AI tool’s journey where it says to the user “I will get back to you with the answer by X time,” and either flags a human at the backend when questions are troublesome, or offers a deliberate, researched answer by showing its working? This counters the notion of moving fast and breaking things just for the sake of it.
As human beings are actively shaping, designing, and building AI as a tool, it is important to recognize that change begins with us. We are increasingly atomized and isolated, and this is constantly pushing us to seek support and counsel. When we are not able to find it around us in ways that are accessible, we turn to AI. This points to something below the surface, namely that as humans, our basic needs for community are not being met. This calls for us to recognize the importance of being genuine and authenticwith each other and not build technology for addiction, while also taking care to avoid technosolutionism. The group referenced two patterns of addictive technology to discuss this. One, social media sites refresh the page when you try to quit the tab, meaning that you are often drawn back to see what you missed. Two, AI tools push you to keep engaging by asking you what they should do next. Instead of creating dependency and designing to create addiction, there could be greater wisdom in sticking to the task a person asked for and leave it at that and invite them to look into the real world as a default and not turn to AI instead.
Drawing from restorative justice, the group focused on accountability and repair for harm and recognized the importance of avoiding the flattening of complex human beings and their diverse lives into monoliths and homogenous blocks. This means understanding that harm and challenges will look different for different groups, and justice would also, as a result, look different for different groups. The idea of accountability and harm should have consistent presence across the AI lifecycle, and should be relevant to the context, material realities, and specific challenges that each community faces. An algorithm is going to categorize, predict, simplify, and optimize whole lives, and in that process, complexity and nuance is lost. Relatedly, the group noted that AI should not be allowed to decide human worth, because reducing a full human being to their economic value or productivity reproduces the very systemic harms we want to question.
At all points, the recipient of the information and the impact of a piece of writing on them is key. The group made note of the powerful role of trigger and content warnings in helping a reader make an informed decision on whether to engage with the AI tool and its outputs, and accordingly, in what way. Bringing in content guardrails in the form of trigger warnings, for example, can be healthy. AI has no way of telling the age of the user, and what is appropriate for that user to receive. It has no understanding of the lived realities of the user and what their immediate contexts are. Having the tool preface its outputs with a disclaimer on its accuracy and a trigger warning can keep more users safe than fall into psychosis or information overload.