Research

Language is a particular kind of semiotic system, but there are also others like gesture, depiction and music. When they combine in communication, as they commonly do, we have polysemiotic communication (polysemiosis). Examples are when we speak and gesture, as well as use facial expressions. Or when we combine these semiotic systems with depiction, either online while drawing (on the sand, or on a whiteboard), or offline in various media genres like advertisements. This theme studies how these semiotic systems combine and interact with one another, across different cultures, and media.

Metaphor has been studied in cognitive linguistics in terms of unconscious mental processes such as  “cross-domain mappings” and “simulations” (Gibbs, 2006; Lakoff & Johnson, 1999). We accept some ideas from this work, but maintain that it confuses motivations for using metaphor, from metaphors themselves, as a particular kind of iconic (i.e. resemblance-based) sign. We have developed a theoretical model that distinguishes between the Embodied, Sedimented and Situated levels of meaning making, called the Motivation and Sedimentation Model, and apply it to metaphors in different semiotic systems.

The term “intersemiotic translation” introduced by Jakobson (1959) has been influential, at the same time as it has outgrown Jakobson’s initial narrow definition as “an interpretation of verbal signs by means of signs of nonverbal sign systems” (ibid: 233).  We  generalize that intersemiotic translation involves a source message expressed through one or more semiotic systems into a target message that is also composed by one or more semiotic systems. By “semiotic system”, we mean signs of a particular kind, with typical for them interrelations and combinations (Louhema et al. 2019).  We study different kinds of intersemiotic translation, such as audio description, where the visual contents of films are narrated in speech for audiences with visual impairments.   

Louhema, K., Zlatev, J., Graziano, M., & Weijer, J. v. d. (2019). Translating from monosemiotic to polysemiotic narratives: A study of Finnish speech and gestures. Sign Systems Studies, 47(3/4), 480–525. 

Debates in the field of language evolution often focus on the nature of human protolanguage, understanding this notion along the well-known conception proposed by Bickerton (1990: 128) as “a more primitive variety of language” that serves as a stepping stone in language evolution. In particular, theorists differ on whether such protolanguage was “musical”, “gestural”, “lexical” or otherwise (Fitch 2010). This starting point, however, assumes that the first step toward human communicative specificity was in fact a language. Complementary to such approaches, we focus on what kind of communicative and semiotic systems paved the ground for human evolution. Bodily mimesis, as theorized in mimesis theory (Zlatev 2019) played a key role in the process.

 Zlatev, J. (2019). Mimesis theory, learning and polysemiotic communication In M. Peters (Ed.), Encylcopedia of Educational Philosophy and Theory. Dordrecht: Springer.

Children develop from birth, and possibly even earlier, not just cognitively, i.e. what they know about their surrounding physical and social environment, but in terms of meaning, i.e. their value-based relationship to the world as subjects of experience.  With time, this relationship changes, acquires new dimensions and undergoes transitions. In other words: children undergo semiotic development. We study the cognitive processes and semiotic systems in which this takes place, which focus on language, gestures and depiction (e.g. Zlatev & McCune 2014).

Zlatev, J., & McCune, L. (2014). Towards an integrated model of semiotic development. In R. Chen (Ed.), Cognitive Development: Theories, Stages, Processes and Challenges (pp. 59-76). New York: Nova Publishers.

Where do language and other semiotic systems reside: in the mind, in the brain, in behaviour, in social norms, or perhaps even in some "Platonic" realm? This ancient philosophical question is still relevant, and has important ramifications. We address it not just in principle, but in relation to specific empirical studies and their methodology. One of our main "tools" is phenomenology, the philosophy of human experience and the objects that appear in it (e.g. Zlatev & Blomberg 2019). 

Zlatev, J., & Blomberg, J. (2019). Norms of language: What kinds and where from? Insights from phenomenology. In A. Mäkilähde, V. Leppänen, & E. Itkonen (Eds.), Normativity in language and linguistics (pp. 69-101). Amsterdam: Benjamins.