The Limits of Control

Surveillance as a surrogate for solidarity

Fifty years ago the insightful philosopher of technology Lewis Mumford noted that cybernetic systems of computing machines were becoming almost godlike in their ability to survey and control everyday life, with increasing omniscience and omnipotence:

In the end, no action, no conversation, and possibly in time no dream or thought would escape the wakeful and relentless eye of this deity: every manifestation of life would be processed into the computer and brought under its all-pervading system of control. This would mean, not just the invasion of privacy, but the total destruction of autonomy: indeed the dissolution of the human soul.

This unheeded warning succinctly summarises the stakes of the globalisation and automation of surveillance. Not only structures of power but also what it means to be human are transformed by the extension of systems of surveillance. To unravel these transformations in social practice, we need to grapple with both politico-ethical questions and complex philosophical dilemmas. Through creating dense material webs of dependence—all based on complex infrastructure, huge amounts of energy extraction, and so much waste—cybernetic systems curtail, enable and altogether reconstitute the possibilities of being able to creatively participate in society.

Surveillance has long historical roots; it is thoroughly entangled with power under capitalist modernity and the militaristic and scientific powers that it musters. At its core, surveillance is a social practice that involves watching over something in order to project control over it. It involves extracting data from a limited slice of reality—thereby reducing the infinite complexity of nature to something that can be represented more simplistically—which can grant an organising power that can be projected over unruly subjects and nature. Frequently, this power is used to defend and extend unequal power relations, serving to concentrate and centralise power.

With long historical roots—going back at least as far as the colonial enclosure movements at the beginning of capitalist modernity—surveillance pierced a qualitative threshold with the rise of networked computing machines during the Second World War. Beginning in the research laboratories that underpinned the industrial-military complex, intellectually trained workers in the nascent techno-sciences began automating and outsourcing surveillance, building it into the very functioning of computing machines and the sets of abstract social practices that surround them. It was in this context that the first theories of cybernetics arose, with the concept being understood as the study of control and communication between people and machines. Cybernetics rapidly fused with the dominant and dominating mode of social organisation, creating what I call ‘cybernetic capitalism’. Broadly, cybernetic capitalism can be understood as a mix of profit-maximising corporations, militaristic state power, the techno-sciences and complex networks of computing machines, coming together in an extremely abstract social formation that has far-reaching consequences for questions of power, agency and the human condition.

Like its less abstracted predecessors, cybernetic capitalism seeks to reorganise the world according to the demands of infinite accumulation. This material impossibility at the heart of capitalism—the desperate need for exponential expansion within finite nature, including finite human nature—is central to understanding the strange and increasingly unstable world we inhabit. Under the conditions of cybernetic capitalism, surveillance has frequently been used as a way to sell people things they do not need, encourage them to rack up debt they cannot repay, compel them to work harder for systematically exploitative companies, and induce them to conform to government policy that works primarily in the monopolistic interests of the powerful. It is crucial to understand this dominant and dominating tendency while also recalling that ‘all modern control systems are riddled with contradictions’, as William Burroughs put it.By putting these contradictions at the forefront, it is possible to see how surveillance can protect and persecute, simplify and complicate, personalise and dehumanise, and so on. These contradictions are all entangled with the uneven power structures of capitalist modernity. Through focusing in on these contradictions it is possible to gain insights into the current world order, as well as see possible windows beyond it. In this spirit, it is possible to imagine diverging pathways that these dynamics could take in the future: a grim, totalitarian vision, and the enduring possibility of alternatives. 

In this regard there are helpful parallels to be drawn between Google’s cyber empire and China’s Social Credit System. While plainly very different institutions—one is controlled by the State Council, the highest organ of the Chinese state; the other is a US corporation—they have striking similarities. They are both very powerful in a hierarchical and centralised way, they both operate on a vast scale, and they both use computing machines and surveillance in order to project control. To begin drawing parallels between these two institutions I will explore the rhetoric of ‘trust’. In clunky bureaucratic language, a policy document published by China’s State Council states:

All levels’ Party and units are encouraged to use name list information concerning persons subject to enforcement for trust-breaking, integrate it into their own areas, professional scope and business activities, and implement credit supervision, warning and punishment over persons subject to enforcement for trust-breaking.

Here and elsewhere, the State Council’s vision for the Social Credit/Debt System is filled with the rhetoric of trust, with the system being sold as an apparatus to automatically enhance trust and sincerity across society at large. This can be seen in the slogan coined for the release of AliPay—Alibaba’s key Social Credit/Debt instrument—in 2004: ‘Trust makes it simple’. Half a world away, Google is also fondly framing its dys/utopian vision within the rhetoric of trust. An example of this can be seen in the corporation’s self-styled philosophy that states: ‘our users trust our objectivity and no short-term gain could ever justify breaching that trust’. Another concise expression of this attitude towards trust comes via a top company spokesperson, who repeats variations on the phrase: ‘trust is the most important currency online’. In different ways, both Google and China’s State Council imagine trust through the lens of technology and the market. In a time when cybernetic capitalism reigns supreme, these kinds of ideological framings are unsurprising. 

And yet the social concept of trust goes deeper than this. Trust evokes a combination of confidence, reliance, dependence and hope, making it a thoroughly subjective and cultural phenomenon. Trust is a form of human social relation that is extremely complex, dense with meaning and layers of particular histories—from personal to world historic—that make the concept resistant to quantification. For example, consider all the density of meaning that constitutes deep social relations, be it between lovers, rivals or parents, or the complexity of social practices, from a Balinese cockfight to a Korean shamanic wedding procession. Upon close anthropological inspection, these social practices are infinitely rich and open ended, with innumerable lines tying each element to a larger social symbolic field, where the power structures of hierarchy, the organisation of nature, and the conceptualisation of the human condition—with all of its desires and dilemmas—come into a dense ensemble of social meaning. This meaning making is often composed of layers of ambiguity and ambivalence, complication and contradiction, all woven into contextual and particular histories. For all the material complexity involved in how computing machines draw in and process surveillance data, in some important respects it pales in comparison to the actual complexity of embodied social relationships. A system that tracks credit-card purchases or other such easily extractable data sets is comparatively simple.

Complexity like this is a problem for power. During the Second World War the scientist Norbert Wiener—coiner of the word ‘cybernetics’—created weapons systems that sought to reduce complexity in order to make predictions and project control. In that case, the complexity of artillery calculations was a problem for the Pentagon, which used computing machines to automate the control of weaponry. This is significant, as all cybernetic surveillance since then has followed this broad trajectory: to ‘watch over’ and reduce complexity in order to make predictions and project control. Hence, the vast complexity of the natural world, which includes the social world, and the unpredictability of history are problems for those who profit most from the unequal status quo. With roots stretching back to Francis Bacon in the long sixteenth century, the possibility of using scientific knowledge and technology to make predictions and gain control has a deep history across capitalist modernity. Cybernetic surveillance participates in these developments by attempting to automate the oracle, to have machines reduce unpredictable people to credit scores, profiles for advertisers, and data sets for electoral engineers—all to project control over their future and secure ongoing power.

Processes of surveillance and automated control are necessary at the gigantic scale at which China’s State Council and Google operate. For most of human history, trust has been made, and broken, largely on the personal level, with interactions being mediated by face-to-face communication. Consider Lewis Tappan, the original nineteenth-century American credit rater, and his need to personally meet those he extended credit to and make notes in his dossier recording his personal assessment of the debtor’s character. Trust on a personal level requires interpretation to sense another’s character and intentions, with much of it happening intuitively, through reading subtle signs in the form of embodied expression. This is not to suggest that this is necessarily ideal, for prejudice can easily flourish in face-to-face relations. Rather it is to argue that extending trust requires interpretive labour: effort to understand the other, to imagine things from their perspective, and to relate this to oneself in a particular social context. Calling this interpretive labour is significant in that it emphasises the importance of interpretation for all social practices, that the process requires work, and that it is bound up with uneven power relations. This is to say, the higher in a hierarchy one is, the less interpretive labour one needs to employ vis-à-vis subordinates; one can simply give orders. Those lower in the hierarchies need to pay closer attention to the actions of their supervisors—in various contexts a manager, police officer, prison guard, welfare officer or husband—in an attempt to avoid the punishments of power.

Using interpretive labour to build trust is problematic for institutions such as Google and China’s State Council. Their titanic scales, greatly extended relations and sharply unequal power arrangements make trust in the social sense of the term impossible; the interpretive labour would be too great. In order to maintain their own asymmetrical power, each turns to networked computing machines, and they see surveillance as a solution. They have computing machines programmed in an attempt to automate and quantify trust in order to make it legible for distant government and corporate bureaucrats. To do this, they have surveillance engines extract data from everyday life—using browser history, biometric data, social-media communications, space–time coordinates and so on—from which they attempt to dramatically simplify trust, boiling it down to an extremely reductive number, be it a Citizen Score, a threat rating or the likelihood that one could be manipulated to engage as a gullible consumer. In these cases, and many, many more, interpretive labour is automated by surveillance engines.

This is problematic, for computing machines cannot perform interpretive labour in the way that humans can, for the way that they ‘think’ is radically different. Drawing from theorising on hermeneutics and embodied cognition, interpretation is a fundamentally social practice and it occurs within and between creative, embodied beings. The thinking of computing machines plays out with a totally different materiality whereby they compute, which is to say, they determine by mathematical means. This is significant, for the word ‘computer’ originated in English along with capitalist modernity in the long sixteenth century; it referred to a person employed to perform calculative labour for surveying—thus again showing how surveying preceded surveillance. The term came to refer to a machine during the nineteenth-century ‘information revolution’ before coming completely to denote a mechanised device after the rise of cybernetics. Now, of course, the computer is a machine and the earlier human profession is obsolete. Thus, in the term ‘computer’ itself we can see a history of how human intellectual practices have been automated, outsourced and encoded into machines. So, calculative labour was thoroughly automated, then, as computing machines have become more and more sophisticated, they have started to be able to approximate interpretive labour and other intellectual practices. 

As a type of computing machine, surveillance engines are made to extract data traces from the world via sensors, which they encode into numbers for computation that can be used to simplify nature, make predictions, and project control. This process can be enacted at tremendous scales, with vast databases of centralised and searchable information available to be processed with algorithms and neural networking. This process can potentially reveal surprising patterns that are totally invisible to human interpreters, a process that can give people real insights into the world. Needless to say, these can be extremely powerful tools in the hands of the technocrats who wield them. While lacking the ability to interpret in the social sense of the term, surveillance engines can powerfully automate aspects of interpretive labour and other intellectual practices, operating at scales, depths and breadths that are totally beyond human capacities. Likewise, people act in the world in ways that are fundamentally creative and totally beyond the ability of a computing machine—or even the actors themselves—to fully grasp. This is the open-ended nature of the complexity of the material universe, and this very complexity is a problem for the narrow calculus of power, which attempts to simplify it in order to project control. While the distinctions between the strong subjectivity that humans possess and the information-processing systems of computing machines are profound and enduring, it is plain that they are drawn together in our historical moment into the overarching power system of cybernetic capitalism. This was part of Wiener’s original understanding of cybernetics: control and communication between humans and the machine.

Surveillance has not always been central to how social relations have been organised: the ‘global village’ is very different from traditional villages. First, traditional villages, and other small-scale human social formations, do not necessarily engage in surveillance in the sense of ‘watching over’, as one is not necessarily ‘above’ one’s neighbour in the sense that the state or the capitalist wields structural power over their citizens/employees. This is not to idealise such traditional arrangements; oppression can flourish there as well, albeit on a much smaller and less systematic scale. It is true that secrets are often scarce in village life, with residents tending to know who is sleeping with whom, or other such gossip gleaned from eavesdropping, incidental observation and rumours. Crucially, this knowledge of others’ actions came as a result of the inhabitants’ lives being intimately bound to one another. For most of history, people have been embedded members of place-based communities, where their relationships with the people around them were primarily mediated by face-to-face relations. This is qualitatively different from the abstract technological surveillance engines prying into personal details and scraping data traces from algorithmic processing designed to serve powerful and distant interests. Abstract surveillance deals with categories, not characters.

These qualitative differences are worth emphasising, for centuries of enclosure movements and forcible displacements have seen the effects of uprooting circle the globe, resulting in traditional folkways being smashed and reorganised by the forces of capitalist modernity, with all of its technological and colonial formations. This has been particularly evident as agricultural regions have been devastated by war, debt, industrialisation and the pressures of the global market, whereby enormous numbers of people have been forced to move to cities, driving a historically unparalleled urbanisation process. Physically moving from the land to cities often severs the more intimate social connections of place-based communities, ripping apart the complex social fabric that has made up the bulk of the historical human experience. The cities and slums that the former rural dwellers now inhabit are vibrant places, where the old ways are creatively remixed. They do not simply disappear in a linear fashion but are overlaid, hybridised, and remade on different levels. Nevertheless, the scale and speed of the process of urbanisation and globalisation under capitalism, particularly since its cybernetic reconstitution, has undermined the older social orders. China, which has seen large-scale destruction of the old ways by breakneck modernisation, is a compelling example of this phenomenon. The hugely intensifying levels of inequality released by this process have, among other things, contributed to a crisis in social trust. Deep distrust is common in everyday life in China, with much of it focused on local governments, although paradoxically it appears to accompany increased trust in the distant central government.

Thus, in such an uprooted world, surveillance acts as something like a surrogate for trust, albeit a problematic one that enables the concentration and centralisation of power. As trust is abstracted from its embodied roots in everyday life, where it is created through interpretive labour, it comes back into social organisation as a techno-scientific process that radically enhances forms of social control. The abstract processes overlay more concrete ways of being, with for example a Citizen Score overlaying reciprocal relations. Framing this as ‘overlaying’ is helpful, for it suggests that the less abstract ways of being do not simply disappear into a one-dimensional flattening; rather they persist in a layered and complex, contradictory way. The combination of scientific observation, technical invention, centralised political power and elite economic extraction come together in a system that is enormously powerful but also deeply unstable. Compelled towards impossible dreams of infinite growth within finite nature, the cybernetic capitalist system is decidedly expansionist and colonial as it pushes into more and more parts of the world and life. As it goes, it can replace or remake other ways of being and doing—other ways that are, on careful consideration, perhaps well worth preserving.

Nevertheless, in these cases, surveillance becomes a substitute for solidarity, one that is necessary to the very functioning of global capitalism. This dominant and dominating social formation simply could not function without surveillance: the system of minority ownership and rule over a starkly unequal society—where exploitation is constitutional, social life is alienated, communication is increasingly disembodied and communities are fragmented—could not be reproduced without the overseers and their computing machines and surveillance engines. The organising power of cybernetics is needed to exert control, for without the view from above the abstracted elite could not try to mould the world according to their interests. This is to say that without cybernetic surveillance, capitalism as we know it would collapse.

This piece is an edited extract from Globalization and Surveillance, by Timothy Erik Ström, published by Rowman & Littlefield in March 2020.

About the author

Timothy Erik Ström

Timothy Erik Ström is the editor of Arena Online. He is the author of the forthcoming book Cybernetic Capitalism (Verso), and his collected writings can be found at his website: The Sorcerer’s Apparatus.

More articles by Timothy Erik Ström

Support Arena

Independent publications and critical thought are more important than ever. Arena has never relied on or received government funding. It has sustained its activities largely through the voluntary work and funding provided by editors and supporters. If Arena is to continue and to expand its readership, we need your support to do it.