The Right to One’s Own Reasons: Autonomy and Online Behavioural Influence

1 Introduction

Consider the following situation. One evening, Alice logs into her preferred social medium website, Sharely.^[1] As she scrolls through her timeline, she encounters multiple photos of her friends attending parties, clubs, and other late-night social events. Watching these photos, Alice is filled with an increasing sense of dissatisfaction with her own life, specifically the fact that she has been spending all of her free evenings at home recently. After a number of these posts, her timeline is interrupted by an advertisement for a concert ticket seller. Her desire to attend more late-night events causes Alice to follow the link to the ticket seller’s website, where her attention is grabbed by a concert taking place the following weekend by a band that she knows superficially and vaguely enjoys. She sends a message to her friend Brandon, with whom she has gone to a few concerts before, to ask him if he would like to join. Brandon responds affirmatively, and Alice buys two tickets for the concert in question.

Of course, it was no coincidence that Sharely showed Alice the posts and advertisement that it did. The timeline of Sharely users, after all, is curated by a learning algorithm, for instance a deep neural network (DNN), taking user data (e.g. posts, likes, cursor movements, which posts the user spends most time watching, etc.) as its inputs. This algorithm is rewarded for optimising the time that users spend on Sharely and the frequency with which they click ads. That is because these metrics generate most profit for Sharely’s parent company, Telos. In this case, the DNN has learnt that users with profiles similar to Alice are likely to click on advertisements for social late-night events after being shown posts by friends attending similar events, especially in the evening.

Situations like this fictional story are very common, given that many people spend a significant part of their lives on social media platforms with algorithmically curated timelines. Such algorithmic curation has, in recent years, led to increasing moral unease. Spelling out the precise grounds (if any) for this unease is a topic of ongoing discussion. In this paper, I will defend an autonomy risk view of the moral harm done by algorithmic curation on social media. On this view, at least a large part of the intuited moral harm is that such curation algorithms endanger users’ personal autonomy, that is, roughly, their ability to make their own choices and shape their own lives. Algorithms employed in the service of social media (or advertisers on social media^[2]) may compromise choices such as whether and how much time to spend on social media, but also choices concerning consumption, lifestyle, and political behaviour. The idea is that, when such choices are influenced by curation of timelines carried out by algorithms deployed in social media companies’ interests, they become less responsive to users’ real interests, plans, or identities – in a word, to ‘who they are’. Alice’s decision to buy the concert ticket may no longer really be her own.

The autonomy risk view is contrasted mainly with analyses taking a manipulation view, which says that the main moral harm of (certain) online algorithms is that such algorithms are manipulative. Manipulation, roughly, is the intentional attempt to make someone’s decision-making process deviate from certain rational standards. After introducing the notion of algorithmic social media curation more clearly (Section 2), I will argue (in Section 3) that it is difficult to apply standard accounts of manipulation to social media. This is because such accounts focus on manipulators’ intentions, but social media companies do not straightforwardly possess intentions. Instead, I will suggest that we shift the focus away from manipulators’ intentions and towards the problematic aspects of the influence itself, namely, undermining users’ personal autonomy. In Section 4, I sketch a notion of personal autonomy that poses two conditions for a person to be autonomous: that the person is able to decide in response to normative reasons and that the reasons for her decisions are coherent with her broader projects, commitments, and life plans. The subsequent two sections spell out how online algorithms can undermine both conditions: Section 5 shows how they may undermine reason-responsiveness and Section 6 suggests that they may also undermine personal coherence. Section 7 concludes.

Before continuing, I wish to shortly lay out my goals in analysing wrongful algorithmic influence. I take it that the general goal of philosophical accounts of problematic practices is to assist in pushing back against such practices. One of the main ways in which we could push back against social media platforms’ undermining personal autonomy is by means of regulation. The philosophical account offered here is meant to assist in this effort, not by proposing specific regulation (that would require more legal discussion than I can give here), but rather by helping identify social media practices that pose morally problematic risks to user autonomy, specifying what is wrong with these practices, and demarcating them from morally innocuous forms of online influence. This hopefully helps lay the groundwork for further discussions about regulation, by focusing and legitimising such regulation.

The philosopher’s ideal would be to specify necessary and sufficient criteria that clearly demarcate wrongful from unproblematic online influence. Unfortunately, I am uncertain whether this can be done. The class of problematic algorithmic influence may be too slippery to be neatly parsed under a philosopher’s scalpel. Instead, I will stick to the more modest goal of outlining two plausible effects of some forms of algorithmic influence that would make such influence pro tanto wrongful. I do not, in this paper, provide clear-cut criteria for recognising such cases, although I will suggest some features of social media algorithms that make these effects particularly likely. If some social media platform uses algorithms that have these features, this should ring alarm bells. The alarm may still be false, but the case at least requires further investigation. This does not decide the question whether individual cases of online influence are wrongful, but it helps focus it – which, I think, is as far as a general philosophical account can go.

2 Social Media and Algorithmic Influence

This paper is (mostly) concerned with algorithmic curation of timelines and search results on social media platforms, and with the influence such curation has on users (I will denote this influence as online influence for short).^[3] The term social media is defined in various ways (Aichner et al. 2021). I here stipulatively define a social media platform as an online platform (either a website or an app) that allows users to share content and access a wide range of content shared by other users. The term content here covers different mediums of communication, including text, images, videos, and audio. The term wide range is meant to exclude platforms where the user can only access messages sent specifically to that user. This definition thus excludes messenger apps like WhatsApp.^[4]

In most social media platforms, the main page (i.e. the page shown when a user logs into the platform) is formed by a timeline (also known as news feed or simply feed) (Altman et al. 2013). This is a regularly updated selection of content and ads, with more content available by scrolling or swiping. Users can usually also search for specific types of content. Since both timelines and search results generally contain only a (small) subset of all available content and ads, curation is needed to select and order those contents and ads that are shown to a user (Davis 2017; Khan and Bhatt 2018). Usually, the precise methods of this curation are proprietary, but a broad (although likely incomplete) picture of major social media platforms’ curation methods can be formed based on a combination of leaked documents, unofficial statements by (former) employees, and public statements and publications (some in response to hearings by the US Senate). From this picture, it is clear that deep-learning models such as DNNs, taking user data as inputs, play a significant role in curating major social media platforms’ timelines and search results. For Facebook, for instance, these data include, at least, clicks, likes, shares, comments, cursor movement (Ganjoo 2018), time spent reading posts (Griffin 2015), physical location (Feinder 2019), and, until 2021, biometric data (the latter tracking method was discontinued after privacy lawsuits) (Kracht et al. 2022). Moreover, these data are not only collected from Facebook itself, but also from a number of partner websites with which Facebook has data-sharing agreements.

Using these data, the deep-learning models are trained to maximise metrics correlated with social media platforms’ profits, such as time spent on the platform, ‘engagement’ (user interactions with content, such as liking, sharing, etc.), and click-through rate (an ad’s click-through rate is the number of times users click the ad divided by the number of times the ad is shown). That is, the model receives a positive feedback signal if a user continues to spend more time on the platform, interacts with more content, and/or clicks more ads, and a negative feedback signal (causing it to change its weights) otherwise. The way the deep-learning models’ outputs are translated into timelines or search results is not as simplistic as in the hypothetical example of Sharely, where a DNN directly determines a user’s entire timeline on the basis of all collected data. First, content recommendation, search and ad targeting algorithms (which together I denote as curation algorithms) tend to be separate (Narayanan 2023). Second, deep learning algorithms are likely combined with human-set parameters, preferences communicated by advertisers (e.g. advertisers on Facebook may target ads based on likes or interests), and sometimes human curating (e.g. to remove offensive posts), in order to determine the final timeline (Kim 2017; Narayanan 2023). Notwithstanding these complexities, machine learning-based methods, used to estimate which timeline or search result candidates and which ads are likely to maximise the target metrics for each user, are a significant part of the curating process.

Algorithms – specifically deep-learning models – thus have an important influence on what content and ads social media users see. Given the plausible assumption that what social media content and ads someone sees often influences her behaviour and given the amount of time many people spend on social media, these algorithms also have an important influence on people’s day-to-day lives. It is thus crucial to scrutinise whether, and under what conditions, this influence is problematic.

3 Wrongful Online Influence: from Manipulation to Autonomy

Existing attempts to demarcate between problematic and morally innocuous online influence often appeal to the distinction between manipulation and persuasion – indeed, a key philosophical collection on this matter is titled The Philosophy of Online Manipulation (Jongepier and Klenk 2022). The idea is that content and ads on social media sometimes influence users in a way that constitutes manipulation. Although several accounts of manipulation exist, many current discussions refer back to the now seminal accounts provided by Noggle (1996) and Baron (2003, 2014).^[5]

According to Noggle (1996), manipulation is behaviour that strives to make someone’s beliefs, desires, or emotions diverge from certain normative ideals. The ideals in question are those that allow a person to exercise her rational and moral agency – or, more specifically, that the manipulator believes allow the manipulated person to exercise her rational and moral agency. Different actors may well disagree on which ideals allow such agency – what is problematic is making people diverge from ideals that you think are the right ones. As examples of ideals that are widely shared, Noggle mentions the ideal that beliefs should be true and relevant, the ideal that desires should be reason-responsive, and the ideal that emotions should be appropriate to the situation (i.e. should help the person in question focus on relevant pieces of information).^[6]

Baron’s approach (2003, 2014) focuses, not on the aims behind individual actions, but on a broader character trait of the manipulative person. This character trait Baron calls manipulativeness. According to Baron, manipulativeness is a vice in the Aristotelian sense, which is to say that it is one extreme of a virtue, which also allows for an opposite extreme.

What one gets wrong, in these forms of manipulation, and what the corresponding virtuous person gets right, is how much to steer others – and which others, and how, and when, and toward what ends; and more generally, to what extent – and how and when and to whom and for what sorts of ends – to seek to influence others’ conduct. (Baron 2003, 48)

Manipulativeness, then, is not one clearly circumscribed character trait, but can manifest itself in a variety of ways – in steering others too often, or too much, or to inappropriate ends, etc. A person with the opposite vice (which Baron calls isolationism) tries to influence people too little, or too few people, etc. The reason why manipulativeness is morally bad, according to Baron, is that it shows a wrong attitude towards others: the manipulative person does not view others as rational beings capable of making their own decisions. Because this attitude is crucial, Baron, like Noggle, stresses that manipulation requires intent on the part of the manipulator (although this intent may be subconscious^[7]).

Noggle’s and Baron’s accounts, then, are different but largely complimentary, the former focusing more on what is wrong with individual acts of manipulation, and the latter on what is wrong with the manipulative person. What both accounts have in common is the focus on some type of mental state or attitude that is supposed to underlie manipulative behaviour (whether it be an intention or a more general motivational attitude). This mentalistic aspect has been taken over by most authors aiming to analyse problematic online influence in terms of manipulation. Susser et al. (2019), in an influential account, for instance, define manipulation as ‘imposing a hidden or covert influence on another person’s decision-making’ (26), where it is specified that this hiddenness must be intentional.

There are, of course, good grounds to impose this mentalistic requirement on manipulation. In most instances where I influence someone in ways that undermine her rational or moral agency, but I have no intention of doing so, it would be counterintuitive to charge me with manipulation (though for possible counterexamples, see Manne 2014). For instance, imagine a variation of the above example where Alice has again been scrolling through posts of her promiscuously outgoing friends, building up a sense of insecurity. In this variation, instead of being presented with an advertisement, Alice is interrupted by a chat message from her friend Brandon, who asks her if she wants to join for the concert. We can imagine that, due to her anxious mental state, Alice’s decision does not meet ideals for rational agency. But we would not say that Brandon is manipulating Alice, for the reason that he did not know that she was in this state and therefore could hardly intend to make use of it (for similar examples, see Pepp et al. 2022). Examples like this one provide a plausible refutation of views on which manipulation is just any type of influence that undermines standards for rational decision-making.

However, the imposition of the mentalistic requirement comes at a price once we want to use standard accounts of manipulation to analyse online influence. This price is that, when the influence is brought about by a social media company, it is unclear to whom or what we can ascribe the relevant intention. There seem to be three options: either to ascribe the intention to the company, or to ascribe intentions to individuals within the company, or to say that the algorithm itself has intentions.

The first option runs into immediate controversy over the question whether group agents, such as companies, can have intentions. Although several philosophers have offered analyses of group intentions (or collective intentions) (Bratman 1993; Searle 1990; Tuomela 1991; Tuomela and Miller 1988), most of these state that such intentions are to be ascribed to, or reducible to intentions ascribed to, individuals. Perhaps it may be thought that we can ascribe intentions to companies in an interpretivist way: if a company’s policy decisions and public statements are easier to explain or predict by postulating an intention, then we should do so. But this strategy would run into epistemic problems: company decisions and especially statements are generally highly controlled, and therefore dubious grounds to establish whether or not manipulation is present. This echoes remarks by Noggle (1996, 51–52), who notes that it is difficult to identify cases of manipulation since it is difficult to identify intentions. Due to companies’ institutional mechanisms for controlling official statements, these epistemic problems are significantly heightened. Insofar as we have sufficient grounds to ascribe intentions to them, these are likely insufficiently specific to ground manipulation. For instance, Nys and Engelen (2022) argue that Google has a manipulative mens rea since it has stated: ‘to maximize your revenue, consider multiple monetization models for your app’ (cited in Nys and Engelen 2022, 142). But surely this in itself does not show that Google is intentionally manipulating anyone, since the quote says nothing about the types of influence by which Google aims to maximise its customers’ revenue.^[8]

The second option is to say that some individual agent or agents within a social media company must have a manipulative intention for manipulation to be present. But the problem here is that companies may make decisions that are not explicitly intended by any individual within the company. As Van de Poel and Royakkers state, ‘we might sometimes judge that none of the individuals could reasonably foresee a certain harm, whereas at the collective level that same harm is foreseeable’ (2015, 6). If an outcome was not foreseeable for any individual, it could also not be intended by any individual. In such cases, every individual within a social media company can plausibly deny having intended the outcome, even though its products have foreseeable harmful effects on users’ autonomy or rational decision-making.

The third option is to say that social media algorithms themselves may have intentions. But ascription of mental states such as intentions to algorithms is highly controversial. Someone who wants to push back against wrongful online influence should not stake their analysis on claims that are so widely disputed.^[9]

Can we do better? That is, can we analyse what is wrong with certain forms of online influence in a way that does not rest on (standard accounts of) manipulation, without including too many forms of influence (such as Brandon’s asking Alice out to the concert) as morally problematic? One suggestion is provided by Pepp et al., who aim to develop an influence-centric account of manipulation that focuses ‘on the manipulative influence rather than on the manipulator’s state of mind’ (Pepp et al. 2022, 93) (for a similar move, see Keeling and Burr 2022). At the same time, they aim to avoid counterintuitive implications similar to the idea that, in the above example, Brandon manipulated Alice. Their proposal is as follows:

For any entity (person, animal, institution, machine, etc.) X, a behavior or feature of X having a certain influence on another entity Y is an instance of manipulation only if the occurrence of the behavior or feature in X is partly explained by its tendency to have that influence on Y or on other entities relevantly like Y. (Pepp et al. 2022, 97)

Indeed, this suggestion correctly captures the above example: the fact that the timing of Brandon’s message has a certain negative influence on Alice’s decision-making does not explain his behaviour, since if the message had not had this effect, he would still have sent it in the same way, at the same moment. In contrast, the fact that Alice’s Sharely timeline has the same influence on Alice’s decision-making explains why the timeline is as it is, because Sharely’s algorithm has learnt that these types of timelines are particularly effective to stimulate advertisement clicks from users like Alice (during the evening).

Pepp et al.’s suggestion thus moves in the right direction as a starting point for analysing wrongful online influence. However, the question of what explains particular instances of online influence is often hard to answer. Especially when dealing with complex self-learning algorithms, such as DNNs, explanations of the algorithms’ outputs are often impossible to give. Still, intuitively, we should not let Sharely (or its parent company, Telos) off the hook because we cannot explain how its algorithm operates. What matters is that Telos, by letting its algorithm loose on potentially vulnerable users, predictably puts its users’ autonomy at risk. In other words, it either knew or should have known of this risk, and yet did not take steps to prevent it.

For these reasons, I propose an autonomy risk view: online influence is pro tanto wrongful if 1) it predictably risks some person’s or persons’ autonomy to be undermined and 2) the influencer could refrain from exercising it.^[10] I remain neutral on the question whether the type of wrongful online influence I am henceforth concerned with should be described as a form of manipulation or not. My aim is to identify what is wrong with this influence, not to regiment its conceptual status (see Barnhill 2022).

On the autonomy risk view, wrongful online influence can be seen as a form of negligence, the preventable risking of harm – in this case, harm to autonomy.^[11] The explanation why some forms of online influence are wrong, then, is analogous to the explanation why it is wrong to expose people to significant risks of other types of harms (here I use the word harm in a broad sense that is meant to be consistent with both consequentialist and nonconsequentialist accounts of the value of autonomy and thus of the wrongfulness of undermining autonomy). If, through negligence of a housing company, a building collapses onto its tenants’ heads, the housing company is clearly in the wrong, regardless of its precise intentions, because it is wrong to expose people to such risks.

Before continuing, let me consider one further objection to influence-centric accounts of wrongful online influence. Nys and Engelen (2022) claim that such accounts presuppose that users would, absent the influence, have acted autonomously (or rationally). But, they object, this is often not the case, since our behaviour is often led to diverge from rational standards (or to be nonautonomous) through accidental environmental factors anyway. I have two responses to this point. First, as we will see, autonomy is not an all-or-nothing matter. Therefore, it is not true that, on the autonomy risk view, ‘non-rational [or autonomy-undermining] factors should be absent for decisions and decisionmakers to be respected’ (Nys and Engelen 2022, 140). Such factors should only be less egregious than in the case of wrongful online influence.

Second, my account only requires that social media create a risk to autonomy for their influence to be wrongful, not that we can prove autonomy to be undermined in some specific instance. Compare: we do not have to know whether someone who was exposed to asbestos really develops cancer (or, if she does, whether the cancer was really caused by the asbestos) in order to know that she was put at an undue risk (and therefore that whoever is responsible for the presence of the asbestos is morally culpable). Of course, this analogy would not work unless people are generally speaking in a better position to exercise autonomy when not using social media than they are when influenced by algorithmic curation. I hope to establish in Sections 5 and 6 that this is indeed plausible. Social media algorithms target individual weaknesses in a way not present in day-to-day interactions and thus create risks to autonomy over and above those unavoidably present in the human condition. Before developing this argument, however, we need to clarify what the conditions of personal autonomy are, exactly.

4 Two Conditions for Personal Autonomy

Let us take stock. Standard manipulation-based accounts of wrongful online influence are flawed because they make it very difficult to identify problematic instances of online influence, which significantly hampers their pragmatic value. A more helpful account should therefore focus on features of the influence itself. So far, I have summarised these features under the broad header of ‘putting users’ autonomy at risk’. But here, the real work for an influence-centric account begins. What types of online influence can be said to risk undermining users’ autonomy?

Jettisoning the mentalistic requirement makes this question more pressing. Recall that Noggle, for instance, can remain relatively noncommittal on which standards for belief, etc. are the relevant ones to consider when determining if someone’s intention is manipulative. This is because he can, as it were, outsource these questions to the potential manipulator: it is the standards according to the influencer that count. But once we abstract away from the mental states of the influencer, we need a more substantive account of what forms of influence are and are not appropriate.

When I speak of autonomy, I mean personal autonomy (as opposed to for instance moral or political autonomy). I understand personal autonomy as a desideratum related to people’s personal lives – although, as Raz (1986, 372) notes, a ‘secondary sense’ of the notion of personal autonomy involves the capacity required for achieving autonomy as a desideratum.^[12] A desideratum for what, exactly? One option is that autonomy is a desideratum for individual actions (e.g. Faden and Beauchamp 1986, 235–237). However, this misses what appears to be an important value underlying much talk about autonomy, namely, the value of being in control of one’s life more generally. Raz summarily expresses this desideratum as follows: ‘The ruling idea behind the ideal of personal autonomy is that people should make their own lives. The autonomous person is a (part) author of his own life’ (Raz 1986, 369). Following Raz, I will consider autonomy as a desideratum that applies to an agent’s life. However, we can still speak of autonomous actions, decisions, desires, etc., meaning that these contribute to an autonomous life. I assume (without arguing for it) that there is such a desideratum – leaving undecided the question whether autonomy is inherently valuable or has merely instrumental value (e.g. in being a means to satisfy one’s personal interests).

I do not here have the space to survey all of the rather expansive philosophical literature on the topic of personal autonomy.^[13] Instead, I will, on the basis of some of the more prominent accounts, characterise two plausible conditions for a person to be autonomous. I do not claim that these conditions exhaust all that is meant with the term personal autonomy. Indeed, the conditions mostly concern what Mackenzie calls self-governance, which ‘involves having the skills and capacities necessary to make choices and enact decisions that express, or cohere with, one’s reflectively constituted diachronic practical identity’ (Mackenzie 2014, 17). Mackenzie views self-governance as only one out of three dimensions of personal autonomy (next to self-determination and self-authorisation). In focusing on self-governance, I do not claim this to be the most important dimension of personal autonomy, but it is the dimension most clearly affected by online influence.

5 How Algorithms May Undermine Reason-Responsiveness

Social media risk undermining both the competences and the conditions required for reason-responsiveness. Starting with the latter: people cannot be expected to respond to reasons when presented with considerations that are not normative reasons but are easily mistakable for normative reasons. It is not hard to find examples of such influence in the literature on social media. Many of the examples canvased by Sahebi and Formosa (2022) fall under this kind. For instance, fake news presents false information as if it were legitimate (Sahebi and Formosa 2022, 69). A user acting under the influence of fake news does not act for a normative reason, since what is false cannot count in favour of anything. Targeted advertising, based on psychological profiles of users, can target users’ psychological weaknesses, such as irrational fears, to make them act in ways that are irrational, and therefore not reason-responsive (Calo 2013; Spencer 2020). This may happen either because users act for motivating reasons that are not in fact normative reasons – that is, they are mistaken about what reasons they have – or because they fail to act for their acknowledged reasons due to (temporary) weakness of will (see also Noggle 2025, 69–74, for a distinction between manipulation through inducing faulty mental states and manipulation through inducing akrasia, which, as Noggle stresses, is often porous).

Social media influence can also undermine users’ competence to appreciate normative reasons, even if they are not actively misled or psychologically targeted. General cognitive skills are often needed to understand the effects of one’s actions and therefore to know whether there are good reasons for these actions. For instance, to assess whether there is a good reason to vote for a certain political candidate, a voter would need to gather potentially complex information about the candidate’s policies and the likely effects those policies would have and compare this information to information about rival candidates’ policies. Research suggests that (excessive) social media use can have several adverse cognitive effects, such as attention deficits and worse memory retention (for a review, see Shanmugasundaram and Tamilarasu 2023).

Instead of further going into examples (see the above references), I will consider the question why algorithm-enabled timelines tend to jeopardise autonomy in this way. The main problem lies in the way that the deep-learning algorithms that are likely used for timeline curation, such as DNNs, are trained. As mentioned in Section 2, these algorithms are trained to change users’ behaviour to maximise metrics such as time spent on a platform or click-through ratio. This training process is blind as to whether the stimuli used to maximise these metrics are reason-tracking or respect users’ own capacity to track reasons. A DNN does not mind whether its outputs are reason-tracking or not – indeed, it has no conception of normative reasons to begin with.

At the same time, the algorithms used in timeline curation may be very effective in selecting stimuli that influence the behaviour of a particular user, either at a particular point in time or repeatedly. This is because they have access to large amounts of fine-grained data that plausibly correlate with the user’s mental properties (Burr and Cristianini 2019) – both general traits and real-time mental states. On the plausible assumption that certain types of mental states make people more vulnerable to forms of influence that are not reason-tracking, this makes undermining users’ reason-responsiveness particularly likely (Calo 2013; Spencer 2020).

This combination of deep-learning models’ strong ability to track users’ mental properties and their inability to grasp normative concepts creates particular risks to the first condition of autonomy. Moreover, given the black-box nature of deep-learning algorithms, it is not possible for humans to actively engage with the way they generate their recommendations. Therefore, the only way that concerns for reason-responsiveness can be incorporated is to employ human moderators to filter the recommendations of the algorithms in question a posteriori – for instance by removing posts that contain false information. But such moderation is likely insufficient to remove the risk of undermining reason-responsiveness. First, the sheer amount of content on large social media platforms makes human moderation very difficult. Second, given their access to individualised information, social media algorithms may learn to target users’ individual biases (e.g. Alice, in the evening, is likely to respond to certain types of influence that do not provide her with normative reasons). To avoid such targeting, human moderators would need to have knowledge of individual users’ weaknesses, which is both infeasible and would be problematic in terms of privacy. Thus moderators can, at most, remove general falsehoods, which are only a subset of those types of online influence that undermine reason-responsiveness.

The discussion so far suggests a possible solution: to be more explicit about the ‘reasons’ timeline selections are based on, so that users are in a better position to critically assess these.^[17] Indeed, some social media platforms have made steps in this direction. Facebook has a ‘Why am I seeing this ad?’ feature that explains to users why they are shown certain advertisements. In an example posted by Meta’s (Facebook’s parent company) global director of ads and monetisation privacy (Pavón 2023), an ad for bath bombs is explained by stating (among other things) that the user has interacted with ads about personal care and with pages and posts about haircare on Facebook and visited websites about personal care that shared data with Facebook. Of course, these explanations are very incomplete; they do not, for instance, explain the timing of advertisements. But we could imagine that a user, upon being faced with explanations like this, is better positioned to consider whether the considerations behind the advertisement correspond to normative reasons. Consider again the example of Alice’s being induced to go to a concert. This time, imagine that Alice is shown an explanation along the lines of ‘Your Sharely activity indicates an interest in spending your evenings more actively’, and decides, upon reflection, that she would indeed like to spend her evenings more actively. Here, her behaviour would more clearly be reason-responsive. Would that mean that Sharely’s influence would be acceptable? To answer this question, we must consider the second condition for personal autonomy.

6 How Algorithms May Undermine Personal Coherence

It may seem that influencing someone by making them respond to a reason is always unproblematic. However, Mitchell and Douglas (2024) have recently argued that this is not the case and that problematic reason-giving influence is especially likely in online contexts (although, unlike here, they focus only on persuasion, that is, on intentional reason-giving influence). Mitchell and Douglas claim that some of these cases of reason-giving influence are wrongful because they harm autonomy. The second criterion for personal autonomy, identified in Section 4.2, suggests another way (different from, though somewhat related to, those analysed by Mitchell and Douglas) why reason-giving influence can harm autonomy: the reasons for which someone is made to act do not cohere with that person’s broader commitments, projects, etc.

Again, we may differentiate between cases where an agent is placed in conditions that make it hard for her to act for reasons reflecting her broader commitments and cases where an agent’s general capacity to form and follow up on such commitments is eroded. The first type of case may occur when an agent is tempted by a reason that is particularly salient given a transient motivational state she is in. Alvarez (2010, 60–63) observes that there is a close connection between motivating reasons and motives. She characterises a motive as either a desire (e.g. the desire for revenge may be a motive), or an indication of a certain (type of) desire (e.g. ambition may be a motive which is not itself a desire but indicates that an agent desires success). When an agent has a certain motive, then, he has a certain desire, and ‘given that desire, certain facts (or apparent facts) will seem to the agent to be reasons for him to act’ (Alvarez 2010, 62). I wish to slightly broaden Alvarez’ notion of motives to what we may call motivational sets, which include all of an agent’s pro-attitudes (not only her desires) that make that agent more likely to act for certain types of considerations, by making these considerations appear to the agent as (strong) reasons. Since someone’s receptiveness to certain reasons is often dependent on temporary mental states (e.g. moods or emotional states), many motivational sets are transient. This means that certain motivational sets can cause a person to act for reasons she would not otherwise have acted for – potentially reasons that do not cohere with her broader commitments. An influencer who wants a person P to act for a certain reason R can make use of this, in two ways:

1. The influencer can cause P to have a motivational set M that makes P more responsive to reasons of the same type as R. In this case, we can say that the influencer has induced M.

2. The influencer can present R at exactly the moment when P has motivational set M, which makes P more responsive to reasons of the same type as R. In this case, we can say that the influencer has exploited M.

To be certain, inducing or exploiting a certain motivational set in someone is not always pro tanto wrongful. Noggle offers the following example (in arguing that nonrational persuasion is not necessarily manipulative): ‘Suppose you remind me of starving children in Rwanda and describe their plight in vivid detail in order to get me to feel sad enough to assign (what you take to be) the morally proper relevance to their suffering’ (Noggle 1996, 49). The vivid details, we may assume, are not themselves reasons for Noggle to do anything (e.g. donate to a famine relief charity). Assuming that there is a normative reason for Noggle to donate, it is the fact that the children are starving, but this fact remains the same no matter how it is presented. The influencer thus relies on inducing a motivational set, rather than directly providing a reason. Still, Noggle is surely right that such influence is sometimes innocuous or at least excusable.

However, we can also add further context to Noggle’s example such that the influencer is doing something pro tanto wrongful. Say that Noggle has a limited amount of money to spend on charity and has for a long time considered which cause to spend it on before deciding that he cares most about protecting the natural environment and should therefore donate to the World Wildlife Fund. If his acquaintance is aware of this but still proceeds to remind him of the plight of the Rwandan children, the behaviour appears more problematic (for a discussion of advice that is offered too late, see Mitchell and Douglas 2024). The coherence condition of autonomy explains this: Noggle has decided to prioritise the value of supporting the natural environment, but his acquaintance makes it more likely that he will act for a reason that is not in line with this commitment, thereby hindering his personal coherence.

The case where Alice is made to act on insecurity is similar to this latter case. Since the insecurity that Sharely has induced in Alice makes her particularly responsive to certain types of reasons (namely, those to do with popular activities), this makes it more difficult for her to act in line with her broader commitments (e.g. her commitment to learn French during her evenings). It may even be that the reason Alice has to attend the concert would not exist absent her insecure state of mind. In this case, although Alice has a normative reason, the normative weight of this reason only depends on changes that Sharely has made in her mental state. When she then acts for that reason, she acts for a reason that is not really her own. Once the bout of insecurity has faded away, Alice may feel alienated from her decision.

The problem still persists in case Sharely is not itself responsible for Alice’s insecurity. Say that Alice feels insecure for reasons that have nothing to do with her Sharely timeline. Still, Sharely’s algorithm may detect this insecurity through Alice’s clicks, cursor movements, etc. and as a result target the concert advertisement to her. (This possibility is not purely hypothetical, as we will see in a moment.) It may then still be the case that she acts for a reason that is less coherent with her overall commitments than some competing reason.

Again, Alice’s autonomy does not hinge on her decision to spend any particular evening this way or another. The problem arises when Alice’s evening plans are repeatedly interfered with by motivational sets pushing her away from her projects or commitments. This is likely to happen if she spends a lot of time on Sharely – like most people who are active on social media. After all, if Sharely’s algorithms have learnt that Alice is responsive to reasons playing into her insecurity during the evening, they are likely to provide such reasons whenever Alice logs in during the evening. Activities chosen for these reasons may therefore occlude Alice’s original goal to learn French.

Now, it can be questioned whether Sharely (or Telos, its parent company) can be blamed for Alice’s loss of personal coherence, no matter how unfortunate. In the second version of Noggle’s example, the pro tanto wrongness of the behaviour of Noggle’s acquaintance rested on the latter knowing that Noggle had a commitment to donating to natural preservation projects. But surely, social media algorithms – or the companies behind them – cannot be expected to know about the personal commitments of their users.

Perhaps not. But social media companies may still be in a position to predict that the way their algorithms work makes it very likely that users’ personal coherence is undermined, without knowing any details of specific users’ commitments. Indeed, two features of deep-learning social media algorithms, such as DNNs, make such effects plausible: 1) their lack of selectiveness in inducing and exploiting motivational sets and 2) their precision in detecting motivational sets.

With regards to 1, note that Noggle’s acquaintance, if he is not manipulative, ought to be selective in inducing and/or exploiting Noggle’s motivational sets. Specifically, he should only do so in order to help Noggle to act for reasons that he (the acquaintance) believes are particularly important – such as the reasons to donate to famine relief in Rwanda. However, social media algorithms have no such restraint. Their reward functions cause them to continuously affect users’ motivational sets, and to exploit detected motivational sets whenever this is likely to maximise one of the training metrics.

With regards to 2, algorithms have access to many real-time behavioural signs that would elude most human influencers, such as cursor movements, scrolling behaviours, etc., and these signs plausibly correlate with motivational states (Burr and Cristianini 2019). As an example, consider an internal Facebook document that was leaked to The Australian in 2017. The document states that by monitoring the online activity of teenagers (some as young as 14) in real-time, Facebook ‘can work out when young people feel “stressed”, “defeated”, “overwhelmed”, “anxious”, “nervous”, “stupid”, “silly”, “useless”, and a “failure”‘, so that they need ‘a confidence boost’ (Davidson 2017). Although Facebook denied that this information was used to target advertisements (Levin 2017), the apparent availability of such methods, combined with Facebook’s profit incentive, gives cause for concerns. What is more, given that users’ emotions at least in part depend on their timelines, social media algorithms may, at least in principle, learn to induce these emotions, then detect whether and when exactly they are successful in doing so, and subsequently provide advertisements that play into these emotions.

In short, the nature of social media algorithms creates a risk that users are, whenever they are on social media (which, for many people, is a significant part of their lives), pushed to act for a selective subset of the normative reasons they could reasonably act for (provided they are made to act for normative reasons at all). It would be very fortuitous if these reasons coincide with their personal projects and commitments. In particular, social media appear to make reasons related to popularity or social status particularly salient for many users. It has been found that Facebook users tend to compare themselves primarily to other users who they perceive as having more positive characteristics than themselves, for example, as being more popular or successful, which reduces their self-esteem (Vogel et al. 2014). Plausibly, this induces in some users a desire to be more popular, causing them to undertake action to align their lifestyle more with what is perceived as popular. Now, the fact that an option is popular may be a good reason to pursue it – for instance, if a tv series is very popular, it may be reasonable to assume that its quality is high and therefore to watch it. But the problem is that if a social media platform makes popularity-based decisions too frequently, users may lose a sense of their lifestyle choices as responsive to their personal commitments (unless doing what is popular is an enduring personal commitment). In this way, a series of choices that are individually reason-responsive may lead to a lifestyle pattern that a user ends up feeling alienated from.

This discussion also suggests ways in which social media users’ competence to maintain personal coherence is undermined (i.e. the top right cell of Table 1). Meyers (1989, 83–84) notes that one of the competences required to formulate and act in line with coherent life plans is the capacity to stand by one’s plans in the face of opposition from others (unless of course one decides to take this opposition to heart upon reflection). This requires a sense of self-worth and self-esteem as an independent decision-maker, which, Meyers argues, is often undermined by socialisation which pressures individuals, especially women, to defer to socially approved projects (Meyers 1989, 167–171; see also Hill 1991, chap. 2; Mackenzie 2014, 35–39). Research suggests that social media tend to reduce self-esteem not only as a transient emotional state (state self-esteem), but also as an enduring characteristic (trait self-esteem) (Vogel et al. 2014). Findings about the connection between social media use and self-esteem are somewhat inconclusive, however, possibly because the relation is strongly mediated by individual differences (Cingel et al. 2022; Saiphoo et al. 2020; Valkenburg et al. 2021). Still, it is likely that at least some social media users’ self-esteem is negatively impacted. Such users would plausibly be less able to fulfil their personal projects in the face of external pressure.

More radically, continued exposure to social media may imperil the formation of coherent life plans and self-conceptions altogether. Our preferences and commitments do not only influence our actions but are in turn influenced by them. If Alice continuously finds herself unable to follow up on her commitment to learn French, she may start to question this commitment itself, and eventually her self-conception as a person curious about language and cultures. In this context, Smith (2020) argues that social media users’ authentic digital identities are under threat from algorithmically created ‘corporate identities’ over which they have no control, and which thus undermine their autonomy. Similarly, Bonicalzi et al (2023) argue that recommender systems, such as those used by social media, ‘participate in the construction of users’ identity in ways that may or may not be responsive to their view of themselves or that may be detrimental to their healthy and satisfactory development’ (828), and as such threaten personal autonomy. I here cannot go more deeply into the question whether algorithmic online influence may undermine users’ life plans and identities as such, since this would require a discussion of identity theory that is beyond the scope of this paper. I merely note that a coherentist view of autonomy underscores the importance of formation and maintenance of a coherent identity and refer to Smith’s and Bonicalzi et al.’s work for further discussion.

7 Conclusions

Departing from the intuition that some forms of online influence exercised by social media platforms leave users in a position where they are insufficiently able to make their own choices, I have considered the commonly made claim that these platforms manipulate their users. The conclusion was that such claims are difficult to substantiate due to their general reference to intentions or other mental concepts and that it is better to shift to the autonomy risk view, which focuses directly on the predictable harms to autonomy that social media pose. I then sketched the outlines of an account of which cases of algorithmic influence (via social media) wrongly undermine users’ personal autonomy. In summary, it was concluded that a person’s autonomy can be wrongly undermined by a curation algorithm if (or insofar as) one of four situations obtains (and can be predicted to obtain with high probability). First, the algorithm may provide the user with considerations that are not normative reasons but are easily mistaken for reasons. Second, the algorithm may undermine the user’s competences required for recognising and acting for normative reasons. Third, the algorithm may make the user act for reasons that do not cohere with her broader commitments or life plans. Fourth, the algorithm may undermine the user’s competences required for forming and maintaining broader commitments and life plans. Due to the nature of deep-learning curation algorithms, it is likely that these algorithms regularly cause all four situations to obtain. If this is true, the companies deploying such algorithms put their users at an undue risk with regards to their autonomy and are therefore morally culpable.

Being the author of a coherent life is difficult – it requires the concentration and mental energy to dedicate oneself to a subset of all reasons one could possibly act for in the face of other reasons and other stimuli pulling one in different directions. The introduction of social media into many areas of our lives risks undermining these mental resources by continuously exposing us to nonreasons or to reasons that may appear particularly salient during certain motivational episodes (often induced by those same social media) but do not in the end reflect who we really are or want to be. The protection of ourselves as autonomous creatures requires that we get this risk more clearly in view so that we may act against it.