Punctum. International Journal of Semiotics

By: Jussi Pekka Holopainen, Xuqin Sun and Max Hattler

ABSTRACT

This paper presents a novel framework for understanding and analyzing semiotic processes in abstract animations. The framework builds on predictive processing, a paradigm in cognitive neuroscience proposing that perception, and consequently action and interaction, involves the brain making multi-level and parallel predictions of potential sensory data and then modifying the predictions based on the actual data received. Building on this approach, we use embodied simulation theory to explain how humans comprehend animations (or any other stimuli) in general and, in our case, abstract animations in particular. According to this theory, humans generate several potential simulations of how we would bodily and actively respond to the given stimuli. For example, when we see a cup (or a picture of a cup), the generated simulations involve our motor circuitry in a way similar to if we would actually reach out and grasp the cup (cf. Gibson’s affordances). Next, we use conceptual metaphor theory and the notion of image- and motor-schemas to explain how the motor simulations activate more abstract metaphoric, but still embodied, interpretations of the animation sequences. We explain how this semiotic chain supports
the Waltonian mimesis as a make-believe theory of fiction. Finally, the paper uses the proposed framework to analyze three prize-winning abstract animations by Max Hattler. The analyses demonstrate the framework’s crucial features and provide examples of how to use the framework for concrete semiotic analysis of animations.

 
Download full text: PDF