Assistive technologies and habit development: a Semiotic Model of Technological Mediation

4.1 Assistive technology has a semiotic potential

First premise: Any kind of assistive technology has semiotic potential (see Sørensen 2019; Sørensen et al. 2020). But what is meant by “semiotic potential”? According to Peirce, semiotic potential refers to the fact that everything, by virtue of being something, has the potential to be interpreted. Or, as Peirce scholar James Liszka puts it, everything has an inherent “presentative dimension,” meaning “a power to be interpreted” (Liszka 1996: 19). Peirceʼs typology of signs further elucidates this idea, identifying three interrelated types of signs: qualisigns, sinsigns, and legisigns. These three types of signs are characterized as follows:

A Qualisign is a quality which is a Sign. It cannot actually act as a sign until it is embodied; but the embodiment has nothing to do with its character as a sign. A Sinsign is an actual existent thing or event which is a sign. It can only be so through its qualities; so that it involves a qualisign, or rather, several qualisigns…legisigns, [are] signs that [are] general types, laws, or habits. (EPII: 291; 1903^[2])

The semiotic potential of assistive technology lies in its presentative dimension. Essentially, assistive technology has the potential to elicit interpretative responses from its users. However, it remains unclear what assistive technology can do precisely and how it achieves its semiotic potential. Semiotically speaking, assistive technology has the potential to bring regularity/order into events/actions – in relation to the purpose/end of the user. The Medimi^®Smart system provides an example of this potential. The system is a digital medicine handling tool that reminds users when itʼs time to take their medication. Specifically, the dispenser installed in the userʼs home flashes green (Figure 1), and after the user presses the button, the dosage is released into a cup (Figure 2) (the pills are stored in cartridges in the dispenser).

Figure 1:

The button on the MS flashing green to alert the user; https://www.medimiab.com/ Medimi AB (Source: This and the following photos are used with the permission of Medimi AB).

Figure 2:

The dosage has been released into the cup and the user is now taking out the pills; https://www.medimiab.com/ Medimi AB.

If the user fails to respond to the initial alert and does not press the button, a rhythmic sound will gradually increase in volume to further alert the user. Should the user still not respond after a certain period, a voice prompt will be activated and say “It is time to take your medication – please press the green button.” Upon pressing the button and dispensing the medication into the cup, the user will receive a prompt to confirm that the dosage is correct by removing the cup and verifying the number and type of pills. Simultaneously, a message will appear on the Medimi^®Smart display instructing the user to: “Take out the cup and confirm it contains x number of pills” (Figure 3).

Figure 3:

The display of the Medimi^®Smart with the text instructing the user to control the content of the cup; https://www.medimiab.com/ Medimi AB.

The Medimi^®Smart thus provides a range of alerts and instructions for the user, including light, sound, speech, and display text. The Medimi^®Smart reflects the userʼs perspective, which focuses on the realization of its presentative dimension or its capacity for interpretation. This potential, as we remember, is based on qualisigns, sinsigns and legisigns and their inter-relations. So, by looking at the series of alerts and instructions we can describe the presentative dimension of the Medimi^®Smart as follows: The button on the front panel of the Medimi^®Smart has various qualities that allow the user to identify and press it, including its form, texture, and dimensions. These qualities can only be intelligible as qualisigns. Similarly, the sound alert and display text have potential qualities that become actualized as sinsigns in the here-and-now when an event occurs. Sinsigns are not randomly distributed but follow the law of the legisign, which renders them significant. This law-like character of language is evident in the instructive speech and display text of the Medimi^®Smart, where words are identifiable replicas of general types and follow syntax rules to form larger phrases.

The light and sound alerts of the Medimi^®Smart follow a precise rhythm, starting at a specific time and following a specific order. These alerts are governed by the legisign, which is the physicianʼs prescription determining the dosage of pills to be dispensed. The interplay between qualisigns, sinsigns, and legisigns described above allows for the Medimi^®Smart to be interpreted as assistive technology in relation to the userʼs attentive perception and goals. Although we will not delve into the semiotics of the Medimi^®Smart, we can use Peirceʼs framework of qualisign, sinsign, and legisign to understand how assistive technology can mediate and be meaningful for the user. Assistive technology must provide positive qualitative possibilities and processes that regulate events and actions in the future. The Medimi^®Smart, through its legisigns, has the potential to bring regularity and order to the userʼs relationship between dosage and dispensation time, which is meaningful for the purpose of taking a correct dosage on time. Additionally, it has the potential to bring order and meaning into the userʼs sociocultural world. The potentiality of assistive technology to bring regularity and order is vital, and Peirceʼs concept of habit will help clarify this idea in the following discussion.

4.2 Habits and the user of assistive technology

Second premise: Using assistive technology involves habit taking and (possible) habit development. So, what is a habit? For Peirce habits are fundamental. A human being, for example, is: “a bundle of habits” (CP 6.288; 1898), where habit is “some general principle working in [the] nature” of human beings “to determine how [they] will act” (CP 2.170; 1902). Therefore, a habit, according to Peirce, involves “really modifying something” (MS 671: 5; 1911^[3]). Importantly, a habit concerns a potentiality and, therefore, habit refers primarily to the future, as Peirce writes: “every habit has, or is, a general law. […] It is a potentiality; and its mode of being is esse in futuro” (CP 2.148; 1902). Peirce often uses the term “habit” in relation to action, but he also recognizes and emphasizes that there are habits of thinking and feeling. This means that under certain circumstances, a person has a readiness not only to act in a certain way, but also to think and feel. Peirce explains how he uses the “word ‘Habit’ to denote any state of mind by virtue of which a person would, under definite circumstances, – mostly, if not invariably, consisting in his experiencing conscious experience of some kind, – either think, or act, or feel.” (MS 852: 8–9; 1911). In addition to habits of bodily actions or physical ways of doing something, Peirce mentions judgments and leading principles of reasoning as mental habits, and perceptual habits as habits of feeling, such as hearing. However, there are also habits of feeling that have been deliberately formed through self-reflection and criticism, as Peirce says, some habits of feelings have: “grown up under the influence of a course of self-criticisms and of a hetero-criticism” or there is taking place “a deliberate formation of habits of…feelings.” (CP 1.574; 1906; see also Nöth 2016: 34–45; Sørensen and Thellefsen 2004). In our everyday lives, habits are typically comprised of a blend of feelings, thoughts, and actions. It is difficult to envision any action occurring without some accompanying emotional state, such as motivation. Additionally, for Peirce, thinking itself can be regarded as a form of action, in which one can exercise self-control, even with regards to physical actions. A central concept, for Peirce, in relation to habit, is belief. And a sign of habit is, indeed, the feeling of belief as Peirce states: “the feeling of believing is a more or less sure indication of there being established in our nature some habit which will determine our actions” (CP 5.371; 1877). Sometimes Peirce speaks of belief as the establishment of habit which will “determine what we do in fancy as well as what we do in action.” (CP 3.160; 1880). Belief is a habit which human beings are aware of (or at least are possible of being aware of) and the results of which will be accompanied with a feeling of satisfaction. Peirce writes: “A state of belief is when [someone] has a habit, knows he has it, and is satisfied with having it.” (MS 873: 4; 1901). Doubt is the opposite of belief; being in a state of doubt is an “uneasy and dissatisfied state” (CP 5.372; 1877); and, therefore, is doubt a state of mind that human beings struggle to free themselves from – the struggle taking place with experience and the use of signs. Based on the above, we can conclude that the initial use of assistive technology must mean the potential beginning of habit taking; and using assistive technology repeatedly means strengthening and, perhaps, developing, a habit. Furthermore, the successful use of assistive technology leads to the establishment of belief and the cessation of doubt. Therefore, the most important conclusion is that when assistive technology brings regularity and order into events/actions this process must be understood in relation to the user’s habits, purposes, and ends. We believe that there are three overall ways, in which assistive technology works in relation to the habits of the user, namely:

1. Assistive technology can make possible the establishment of a new habit in the user.

2. Assistive technology can strengthen an already established (but faltered) habit in the user.

3. Assistive technology can re-establish a habit in the user which, for some reason, has been brought to an end.

These three possibilities, of course, concern both habits of actions, feelings, and thoughts and combinations thereof – which probably will be the most common situation. Let us imagine an elderly female user of the Medimi^®Smart who has suffered an ischemic stroke.^[4] Having the stroke has caused certain complications for the woman, namely, cognitive complications in relation to memory and attention as well as physical complications including a loss of functional (right) hand movement. Hence, before having the Medimi^®Smart installed at home (as part of her rehabilitation process), the woman had difficulties remembering when to take her medication and how many times the medication had already been taken. Furthermore, it was difficult for the woman to open pill bottles and/or to take out pills from blister packs (due to the loss of functional hand movement). The result was that the woman had a visiting healthcare professional to administrate her daily medication. However, with the Medimi^®Smart the woman has learned to take her medication without help from a healthcare professional. Firstly, because the woman understands that when the alerts of the Medimi^®Smart catch her attention, she must click the button on the front panel of the Medimi^®Smart in order to release the dosage; secondly, she is (still) able to take out the cup with the dosage and empty the content, and, thirdly, she can compare the content of the cup, concerning the correctness of dosage, with the description of the dosage on the display. As Peirce accentuates about habit: “What the habit is depends on when and how it causes us to act. As for the when, every stimulus to action is derived from perception; as for the how, every purpose of action is to produce some sensible result.” (CP 5.400; 1878). Thus, the Medimi^®Smart is bringing regularity and order concerning the dosages of the female user in relation to dispensation time. And, relative to the purpose of the user (to take the correct dosage on time), the Medimi^®Smart is establishing new habits in her. Firstly, a cognitive habit is established concerning the alerts, the immediate attention and the understanding of the woman – that it is now time to take the medication and that she must compare the content of the cup with the description of the dosage on the display of the Medimi^®Smart to control the correctness of the dosage. Secondly, and intermediary to the new cognitive habit, a physiological habit is established, concerning, of course, what the woman must do to take her medication on time: Clicking the button, taking out the cup, and comparing the content with the information on the display. Finally, using the Medimi^®Smart also taps into already established cognitive and physiological habits in the woman; habits which now, however, are blocked – such as the cognitive habit of knowing that repeated intake of medication must take place within certain intervals (to avoid over medication) and the physiological habit of taking out pills from a bottle or blister pack. Prior to experiencing her ischemic stroke, the woman exhibited no cognitive impairments in relation to managing her own medication regimen. Specifically, if required to consume various medications over a certain duration, she was capable of accurately recalling the timing, dosage, and duration of each treatment course. Hence, by using the Medimi^®Smart the woman re-establishes an adequate cognitive and physiological habit; that is, a habit involving attentiveness towards the timing of medication intake, frequency, and the correct method of administration. When the woman is pressing the button on the front panel of the Medimi^®Smart this is an individual act; however, this act is related to her experience as well as her expectations for the future. And, as a user of the Medimi^®Smart she believes in her habit of using the Medimi^®Smart. She believes that the habit is true or that when she is pressing the button the correct dosage will be released on time because as Peirce writes: “a belief is a controlled habit. Being formed, or continued, under control, there is an expectation and hope that it will last” (MS 1893: 146). With the establishment of her belief also follows a state which is satisfactory and calm, which she, again with the words of Peirce, does: “not wish to avoid, or to change to a belief in anything else.” (CP 5.370; 1877). Hence, when assistive technology brings regularity and order to events/actions, it does so in relation to the habit formation and/or habit development of the user. The user is, as Peirce puts it, a “bundle of habits,” and sometimes he/she need the establishment of a new habit or perhaps the re-establishment of an old habit to meet certain purposes or ends. These habits can involve both physiological ways of doing things and mental habits, but both involve how assistive technology is and can become meaningful for the user. From the userʼs perspective, the meaningfulness of assistive technology is accompanied by a feeling of belief, and this belief will guide the actions and thoughts of the user – such as clicking the button on the Medimi^®Smart because the user expects the release of a correct dosage on time. Peirce believes that meaning is closely related to habit, stating that “what a thing means is simply what habits it involves” (CP 5.400; 1878). In relation to assistive technology, we can add that it places the user in a position to react – it simply causes/enables actions, thoughts, and feelings to be directed towards purposes and ends. One way to address the use of assistive technology, in relation to meaningfulness, is to look at use as identical to the interpretative responses of the user.

4.3 The use of assistive technology is identical with interpretative responses

Third premise: The use of assistive technology is identical with interpretative responses. When the presentative dimension of assistive technology is actualized, this is related to use and experience; and the user actualizes (parts of) the semiotic potential of the assistive technology by interpretative responses. And, of course, these responses, also, concern habit formation and possible habit development. For Peirce interpretation must be seen through his concept of interpretant because as he writes: “I pass now to the…essential ingredient of the interpreter, or as I prefer to call it, the interpretant. I might call it the Meaning, since it includes all that the sign really does convey to the interpreter, its entire essential influence, in its capacity as sign.” (MS 318:37; 1907). Already perceiving assistive technology is, from a Peircean perspective, an act of interpretation and what we find particularly relevant in Peirce, is that he understands interpretation in relation not only to conceptual import but furthermore to feeling and physiological effort (which we should also expect from our description of habit above). This becomes clear when Peirce introduces this classification of the interpretants:

It is now necessary to point out that there are three kinds of interpretants. Our categories suggest them, and the suggestion is confirmed by careful examination. I term them the Emotional, the Energetic, and Logical Interpretants. They consist respectively in feelings, in efforts, and in habit-changes…The majority of signs, in their significative capacity, provoke efforts in the outer or inner world, or whether they be efforts of inhibition, or self-restraint, which make so large a part of the effort to pay attention. (MS 318: 35–37; 1907)

And similarly, assistive technology, in its capacity to provide meaning, is no different. The meaningfulness of assistive technology for the user comes in the form of the three interpretants as described by Peirce. Taking the example of the Medimi^®Smart again, its alerts – light, sound, and speech – produce the initial feeling of familiarity and recognition in the user, serving as emotional interpretants. The effect of action or exertion on the user is the energetic interpretant, which can be either physiological or mental (see also Liszka 1996: 26). For instance, when the sound alert of the Medimi^®Smart prompts the user to click the button on the front panel to release the dosage or when the user thinks to herself or says to her relative “now it is time to take my medication,” the energetic interpretant is involved (see also West 2020: 154). However, when the user begins to reason or to exercise self-controlled thinking about the functioning of the Medimi^®Smart, it brings about the logical interpretant or an intellectual concept, in line with Peirceʼs theory. It is essential to understand how the three interpretants form a hierarchy (see also Liszka 1996: 26–27), with the emotional interpretant being the first proper significative response of the user to the Medimi^®Smart, involved in the energetic interpretant. Thus, there must be a feeling of recognition and motivation to click the button on the front panel of the Medimi^®Smart before the user actually shows the effort to do so. Furthermore, when the user begins to compare the content in the cup with the information on the display, this mental action involves both the emotional and energetic interpretants. It is partly triggered by the conditions and structure of experience before the user can begin to read and understand the text on the display. However, when the user of the Medimi^®Smart reads the text on the display, it is the law-like characteristic of the logical interpretant that enables him/her to understand the conventional meaning of language and numbers. These three different but inter-related interpretants together make up a possible framework within which we can understand the use of assistive technology as meaningful – seen from the perspective of the user. But we must accentuate that we understand Peirceʼs interpretants, not as ʻcarriersʼ of subjective psychological content; rather, they are universal cognitive structures for the production of significative effects. As a man of his time and influenced by (some of the) leading ideas of evolution theory and well versed within the methodology and history of science, Peirce expresses himself as follows with relevance for the interpretants: “Manʼs mind has a natural adaptation…It is somehow more than a mere figure of speech to say that nature fecundates the mind of man with ideas which, when these ideas grow up, will resemble their father, Nature. (CP 5.591; 1903). This also means that even though the interpretants (probably) always will involve content from a sociocultural context (which will influence the significative effect), they cannot be reduced to this content and that they, ontologically speaking, as a form for meaning, will be prior to the (specific) sociocultural content. Finally, let us remember how Peirce defines meaningfulness in relation to the conceivable practical consequences of an object. He writes famously as follows (namely describing his pragmatic maxim): “Consider what effects, that might conceivably have practical bearings, we conceive the object of our conception to have. Then, our conception of these effects is the whole of our conception of the object.” (CP 5.2; 1902). Therefore, like any object, assistive technology has practical consequences that give it meaning to the user. The Medimi^®Smartʼs most significant practical consequence for the user is that, when the button on the front panel is clicked, the correct dosage is released on time. However, habit formation and development are essential aspects of the Medimi^®Smartʼs design, as the users form habits of interpretation in response to the device, which involve emotional, energetic, and logical interpretants. As the users repeat their interpretative responses to the Medimi^®Smart, a general usage pattern emerges, giving the device meaning for the user. Although the habit of use cannot be reduced to specific interpretative responses, it is meaningful to the user through memory and positive expectations. The law-like characteristic of the logical interpretant systematizes the emotional and energetic interpretants and makes the Medimi^®Smart meaningful to the user (see also Savan 1988: 11–12, 45; Danaher 1999: 564–65). The Medimi^®Smart brings regularity and order to the userʼs dosages in relation to designated time slots and purposes, whether or not the user clicks the button on the front panel. Therefore, technological mediation involves a process of meaning-making, which is the fourth premise of the SMTM.

4.4 Assistive technology mediation is a process of meaning making

Fourth premise: Assistive technology mediation is a process of meaning making. From what we have been writing in the previous pages the fourth premise cannot come as a surprise of course. We understand assistive technology as capable of bringing regularity, order into events/actions; as mentioned more times before, the Medimi^®Smart, for example, brings regularity, order into the userʼs dosages in relation to time and the correctness of the dosage (types of pills and amount). However, to do that, in relation to the user, the Medimi^®Smart must have a presentative dimension, that is, a potential to be interpreted (concerning qualisigns, sinsigns, and legisigns, and combinations thereof). A dimension which, firstly, materializes itself in the different alerts of the Medimi^®Smart and, secondly, becomes meaningful for the user in his/her interpretative responses to the alerts (involving emotional, energetic, logical interpretants, and combinations thereof). Moreover, the interpretative responses form and are formed by (already existing) habits in relation to the purposes and ends of the user. And, finally, the interpretative responses are identical with the use of the Medimi^®Smart because these involve both mental responses (e.g., self-controlled thinking) and concrete ways of doing things. This process is, obviously, a process of meaning making – however, where, more precisely, is the mediation taking place or, perhaps, someone might say, that there is something missing from our description so far. Well, let us first remember how Verbeek explains technological mediation and what it means to study it. Hence, he writes, that such a study addresses: “technology in terms of the relations between human beings and technological artifacts, focusing on the various ways in which technologies help to shape relations between human beings and the world… as mediators of human experiences and practices.” (Verbeek 2016: 189). And, furthermore, he accentuates: “In order to develop a full understanding of processes of mediation, we should not only study ‘what things do’…but also how humans give meaning to these mediations – both empirically and conceptually.” (Verbeek 2016: 190). As mentioned, before, we do agree when Verbeek describes mediation as a relationship between a user and the world, and how technology shapes experience and practice. However, semiotically speaking, mediation is the process of meaning-making, rather than how humans give meaning to mediation. The relation involves a meaningful connection between users and their environment, and technologies shape this relation. The Medimi^®Smart exemplifies this by shaping the userʼs experience of time and the practice of taking medication regularly. Peirceʼs understanding of mediation adds to this description, particularly in terms of how assistive technology can shape the relationship between the user and the environment. Firstly, Peirce says, that mediation concerns a process whereby: “a first and a second are brought in relation.” (CP 6.32; 1891); not wanting to go much into Peirceʼs three categories, here it suffices to notice that they involve, as he writes, the following characteristics:

My view is that there are three modes of being. I hold that we can directly observe them in the elements of whatever is at any time before the time in any way. They are the positive qualitative possibility, the being of actual fact, and the being of law that will govern facts in the future. (CP 1.21–22; 1903)

We have, already, addressed the categories in the above, but without naming them categories; that is concerning the three types of signs and interpretants – both trichotomies of qualisign, sinsign, legisign and emotional, energetic, logical interpretants, correspond, respectively, to firstness, secondness, and thirdness in their mentioned order. Furthermore, is the law-like regularity of habit related to thirdness. However, now, when we are looking at Peirceʼs definition of mediation concerning the relation between the assistive technology, the user and the environment, we will say, that it is the technology which is bringing a potential user, as first, into a relation with an environment, as a second of actual facts. And, that the assistive technology, furthermore, brings the potential user and the environment into a general, intelligible relation because it represents how facts will be governed in the future. The last mentioned also fits nicely with the way in which we have already described assistive technology as “bringing regularity, order into events/action”. Yet, for Peirce, and not surprisingly, mediation is also closely related to his definition of the sign, as he accentuates, mediation concerns: “the character of an object…which reaches its fullness in Representation.” (EPII: 183; 1903). Representation is just a synonym for the sign relation and Peirce describes this relation in the following way: “a sign is a thing related to an object and determining in the interpreter an interpreting sign of the same object. It involves the relation between sign, interpreting sign, and object.” (MS 462: 74; 1903). Hence, with this definition of the sign we get an understanding of technological mediation as a triadic relationship always involving technology, user, and environment simultaneously – in order to be meaningful. An important point to stress, however, is that the object dimension, with Peirce, is actually two-fold (Sørensen et al. 2016). As he writes:

…we have to distinguish the Immediate Object, which is the Object as the Sign itself represents it, and whose Being is thus dependent upon the Representation of it in the Sign, from the Dynamical Object, which is the Reality which by some means contrives to determine the Sign to its Representation. (CP 4.536; 1906)

Simply put, this means that there is a reality which determines and/or constrains technological mediation; a reality, parts of which can become represented with the function of the technology and through the interpretative responses of the user. This, of course, also means that the technology, itself, can have (probably always will have) a determining and/or constraining function, for example, in relation to experience, practice – including what is considered good and bad, right and wrong, true and false etc. Ontologically speaking, everything real can have a determining and/constraining function. And, from a Peircean perspective, everything which is real must be included in one of the three categories mentioned in the above (CP 1.21–22; 1903). Hence, the positive qualitative possibility, such as the mere ideas of the technology user, has real being; actual facts, not surprisingly, also, have real being, including the spatial-temporal context, where the user is interacting with the technology. And, finally, involved in the real is, furthermore, law-like phenomena governing facts in the future – such as the different habits of the user (universal physiological, perceptual and mental habits etc.), but also the sociocultural context, where, as Boshuijzen-van Burken writes (2016), “certain rules, norms, procedures etc. are in place.” (Boshuijzen-van Burken 2016: 186). In short, that which has real being, also, has the possibility to determine and/or constrain technological mediation – such as what is physical and/or technically possible, user context, cognitive structures (e.g., the interpretants), as well as sociocultural conditions (we know that these determinants/constrains often are intertwined) (see also Vermaas et al. 2011: 83–99). To analyze technological mediation, it is important to consider three semiotic dimensions: the technology (as a sign), the user (as an interpretant), and the environment (as an object). The Medimi^®Smart serves as a sign by representing a part of the environment (a number of pills in relation to time) and directing the userʼs attention, memory, and understanding towards a certain action (clicking the button on the front panel). The userʼs interpretative responses form habits that shape their experience and relation to the environment. Technological mediation, however, can also, and probably often will, shape the userʼs self-experience and their relation to the sociocultural environment (also a major point in the research of Verbeek as well as of Ihde). Regarding the Medimi^®Smart we can imagine that the technology, qua function and its use, is shaping the userʼs self-experience and the way in which the user is feeling and thinking about him/herself in relation to a sociocultural environment; logical interpretants such as “independence”, “freedom”, and “dignity” easily come to mind – as interpretants which could be meaningful for the user and guide his/her actions and feelings. And, of course, the sociocultural environment serves here as context because the meanings of the interpretants involve a relation to other people, perhaps relatives, friends and/or healthcare professionals, which the user of the Medimi^®Smart now (at least concerning taking medication) is able to live without not needing their help or influence. However, a point is, furthermore, that technological mediation can tap into, influence or even change the normativity of the sociocultural environment; in the case of the Medimi^®Smart, it represents, for example, what it means for a senior citizen to live an independent life (connoting dignity inter alia) – yet, the senior citizen is still relying on a robot (not a human sic!?) for taking the correct dosage on time. And, let us therefore finally remember, that there is significant sociocultural difference whether we call the Medimi^®Smart a “medication reminder”, an “automated medication dispenser”, or a (medication) “robot”. Now we will turn our attention to put the elements together from the four described premises and make a conclusion by proposing the sketch to a semiotic model for technological mediation.