Total Theatre and the Transformative Potential of Augmented Total Theatre In Arts, Culture, and Education

posted in: augmented reality, essays | 0

Total Theatre and the Transformative Potential of Augmented Total Theatre In Arts, Culture, and Education

Sergio Cicconi Dept. Information Engineering and Computer Science - University of Trento (ITALY)
16th annual International Conference of Education, Research and Innovation (ICERI) 2023 Seville: 13-15 November, 2023, ICERI2023 Proceedings, pp. 3991-3998, doi: 10.21125/iceri.2023.1002
Abstract
This work examines the concept of Total Theatre, explores the potential contribution of Augmented Reality in revitalizing this artistic form, and sheds light on the transformative impact of Augmented Total Theatre—the most mature manifestation of total theatre—in the realms of arts, culture, and education.Total theatre represents a theatrical form that seeks to immerse the audience through the integration of diverse artistic and sensory elements. Rooted in a rich historical tradition, total theatre has engaged numerous artists, theorists, and art movements throughout its evolution. While total theatre has achieved a certain level of success in creating immersive and interactive theatrical experiences, it remains constrained by the limitations imposed by the physicality of the real world. The objects on stage, the stage itself, and even the actors are bound by the laws of physics and the materiality of everything represented in the theatre. Augmented reality emerges as a potential solution to these constraints by seamless blending real and virtual elements. Augmented reality expands and enriches the physical stage, enabling dynamic interactions between real actors, physical objects, locations, and virtual holograms. This new form of theatrical representation – we called it Augmented Total Theatre -- opens up new horizons for innovative and captivating experiences, where almost anything becomes possible. The integration of augmented reality and total theatre marks an exciting frontier, not solely within the realm of theatre, but also for the broader landscape of arts, culture and education. This breakthrough enables artists, cultural operators, and educators to transcend physical limitations, empowering them to design captivating and immersive experiences that push the boundaries of audience engagement. Augmented theatre emerges as a powerful tool for transforming the way we experience and participate in creative and cultural expressions, fostering a deeper connection between the audience and the theatrical representation.
1. INTRODUCTION
Theatre, in its many forms, has always been a powerful tool for reflecting reality and engaging the audience. It is in this context that the concept of “total theatre”, a theatrical form popularized in the 19th century by Richard Wagner and continued by various artists and movements of the 20th century, is inserted. Total theatre aims at a harmonious integration of different artistic elements - music, lights, scenography, dance, acting, and more - to immerse the audience in an increasingly deep and engaging experience.
In this paper, starting from the exploration of the ideas underlying total theatre, we propose the concept of augmented total theatre achieved through augmented reality technologies. We argue that augmented reality, allowing the smooth integration of 3D virtual content into real environments, can push the boundaries of total theatre even further, overcoming some limitations of current theatre. Through the analysis of the most important features of traditional theatre, and of the key features of augmented reality, we will highlight the new creative possibilities offered by the convergence between total theatre and augmented reality.
Particularly, we will examine the potential of augmented reality in transforming the role of the viewer, as well as the very space of theatrical representation. We will discuss how augmented reality can pave the way for more portable, interactive and customizable forms of theatre. Finally, we will consider the potential of this new form of augmented total theatre not only for theatre but also for arts, culture and education in a broader sense.
2 TOTAL THEATRE
The concept of Total Theatre (TT) refers to the idea that theatre represents the convergence point of all the arts. TT represents a theatrical form that seeks to involve the audience as much as possible through the integration of different artistic and sensory elements. It was the composer Richard Wagner who, in the Nineteenth Century, with some of his writings [1] popularized the term Gesamtkunstwerk (total work of art), that is, an artwork, concept, or creative technique in which several art disciplines are merged to make a single coherent whole. Among his many works, we recall Der Ring des Nibelungen (1876), which established him as the most famous and influential composer of his time. The opera combined drama, literature, songs and dramatic settings to create a unified and coherent experience of great impact for the audience.
This idea of total artwork, when applied to the theatre, implies an approach that today we define as holistic: TT encompasses a collection of various components such as music, lighting, movement of bodies and objects, voice, painting, costumes, makeup, design, set design, architecture, literary expressions, and more, and composes them together in a homogeneous unit that is worth more than the sum of its parts. Therefore - and here lies the essence of TT - it is not simply a matter of accumulating these components, but rather of creating an effective interaction and integration between them. Totality, in this context, can vary in its scope, including more or fewer components, but its essence lies in the intensive relationship between these components. It is the integration of the components that distinguishes TT, rather than the simple accumulation of artistic means.
From Wagner onwards,  TT has involved numerous artists, theorists and artistic movements. However, we will not deal with the details that recount the evolution of the idea of TT. Here, it is enough to mention those authors who have contributed to defining the concept and enriching it. After Wagner, we recall artists and theorists such as Maurice Maeterlinck, Stéphane Mallarmé, Adolphe Appia, Paul Claudel, Antonin Artaud, Martha Graham, Jerzy Grotowski, John Cage, Eugenio Barba, Peter Brook, the Living Theater, Robert Wilson, La Fura del Baus, just to name a few. These authors and many others have contributed  to expanding and occasionally dismantling the confines of traditional theatre, aiming to captivate more and more the audience through innovative and multisensory experiences [2], [3], [4], [5].
Here, what we are really interested in is the idea that unites so many theories and authors who have dedicated themselves to TT: traditional theatre provides a limited representation of reality; TT attempts to extend this traditional form of representation by including more aspects of reality within the theatrical experience, for the purpose of involving the audience in a deeper and more complete way.
Also, there is another idea that we think is useful to consider. It is about an alternative perspective on the concept of Gesamtkunstwerk. This perspective is derived from a collection of Wagnerian writings titled “The Artwork of the Future” [1], where Gesamtkunstwerk is also meant as the artwork of the future. According to this perspective, TT, as a total work of art, can be thought of as the future of theatre. As such, TT represents a never completely fulfilled potential, so that its ultimate realization is perpetually sought after but never fully attained [2].
Let's see how to integrate the two ideas to move forward with this work. The Twentieth Century saw many realizations, always new and never definitive, of total theatre. And this also thanks to technological evolution. Indeed, technological innovations have offered TT new creative possibilities, and have allowed the integration of ever newer multisensory elements into theatrical representations. We can think, for example, of the recent use of sophisticated computerized lighting systems to create visual spaces, or the use of elaborate sound spaces, interactive sets, or video projections superimposed on people and things on stage. These, and many other technological stage components have increasingly enriched the sensory experience of spectators and expanded the boundaries of traditional theatrical representations.
Yet, there seems to be a limit that even the most recent representations of TT have to reckon with: the physicality of the real world. In a theatre that is total, but still essentially physical, analog and mechanical, even if increasingly contaminated by digital interventions, many of the objects on stage, the stage itself and even the actors cannot escape the laws of physics and the materiality of the real world.
Such limit seems difficult to overcome, even with the more sophisticate technologies we find nowadays in TT representations. Unless TT opens up to a new technology: Augmented Reality (AR). AR seems, at least for the moment, the only technology with the potential to overcome that barrier that even the most recent (but yet analog and mechanical) forms of TT collide with.
This idea is the core of our inquiry into AR in relation to TT. Indeed, we believe that the use of Augmented Reality with TT can truly lead to a more mature and richer form of TT. We think that, through AR, the total theatre of the future  becomes the theatre of the present. Therefore, in the remainder of this work, we will lay the foundations for an analysis of this idea, leaving further investigations to explore it more thoroughly.
Before closing this section, we want to make a final remark. We want it to be clear that we do not intend here to propose a new theory on theatre. We are not trying to explain how to do theatre through AR. Nor do we mean to argue that theatrical representations built using AR are in any way better than those proposed by the actual TT. Here, we only intend to show the potential that a tool like AR has in allowing the creation of new  forms of communication that are impossible without the use of AR. And, of course, we leave it to theatre professionals to understand how to best use this tool to obtain those specific forms of communication that make their creative visions possible and concrete.
3 AUGMENTED REALITY
Before clarifying how AR can make itself available to an ideal form of total theatre, it seems necessary to introduce some ideas about what AR is. In other works we already examined in depth AR properties and we also investigated the products of AR, meant as new forms of communication [6], [7], [8]. Therefore, here we limit ourselves to discuss only those aspects of AR that will be useful for our discourse.
According to a by now classic definition “AR allows digital content to be seamlessly overlapped and combined into our perceptions of the real world in real-time” [9]. This means that, through the use of some technological device (such as, for example, a cell phone with a video camera and a display) the user’s view of the real world can be enriched in real-time with digital content (virtual objects, backgrounds, people, buildings, etc.) to integrate seamlessly with real content and interact with it.
This definition, although correct, is too general for this work. It needs to be refined by specifying that the AR we are dealing with is immersive, which is a more complex form of AR than that used on mobile devices (which remains non-immersive). It is a form of AR that uses wearable devices (also called Head-Mounted Displays (HMDs), such as Microsoft HoloLens 2, or the more recent Meta Quest 2, Meta Quest Pro, and Apple Vision Pro) to create an augmented space within which the user is immersed[n1].
Let’s review and partly deepen the most important features of AR already defined in the 1990s [9], [11].  Augmented environments built using AR technology are:
  • Augmented: This is a distinctive feature of AR. A layer of virtual world is added and seamlessly overlaid in real-time on the real world. In the user's perception, the resulting environment is a single, richer, more complex world, laden with more information than the real world.
  • Three-Dimensional: all objects defined in the virtual world (called holograms) are 3D and remain attached to real world elements (which, of course, are 3D as well) in a realistic way. They are therefore able to constantly adapt to the real environment which, in the user's perception, constantly changes in relation to his movements. In this way, the user perceives the holograms as real objects among other real objects. This feature adds depth and realism to the user's experience.
  • Interactive: this is not a distinctive feature of augmented environments. For some time now many interactive installations have been created in physical environments without using AR. In general, in such environments the user is free to interact, with gestures or body movements, with real physical objects or with virtual objects usually projected on the walls (therefore two-dimensional), or projected onto real 3D objects present in the environments (thus simulating a semblance of three-dimensionality).
    In the case of AR, however, this interactivity is enhanced. All virtual objects are 3D, can be anywhere within the augmented space, and can interact with each other, with real world objects, and with the user, reacting in real-time to visual and sound stimuli, to voice commands or user gestures, or to commands given through input devices such as pads and controllers, or from scripts attached to the objects themselves. In this regard, we want to make explicit here an aspect that is rarely emphasized in discussions about AR, and which is nonetheless at the basis of interactivity itself: the programmability [12]  of all the virtual elements that together make the “augmented” component of augmented environments. It is only thanks to programs, or scripts, that it is possible to define the rules for interaction, to modify the shape, color, size and position and, in general, the behavior of virtual objects in response to various inputs or conditions that arise during the users' experience in augmented environments. Programmability adds a level of dynamism and customization to the experience of users in augmented environments that is unprecedented. In fact, the entire augmented world, which seamlessly overlays the real world, is interactive precisely thanks to the programmability of virtual objects.
  • Immersive - immersivity too is not a distinctive feature of augmented environments. Nor is it new to the theatre[n2]. Rather, it is a property that emerges from the other features examined so far. Given an augmented environment built by integrating the real and the virtual, which is 3D and interactive and is also visitable, navigable and in a sense “inhabitable” by the user, it is reasonable to state that such an environment immerses the user within it. In fact, the user is enveloped in and involved in the immersive experience, and feels like an integral part of that augmented and altered world surrounding him/her, which is a little bigger and richer than the real world.
  • Portable: portability expresses AR's ability to adapt in real-time to a variety of real environments, its flexibility in quickly interacting with the surrounding physical world, allowing users – no matter where they are -- to immediately access virtual content. Certainly, we can somehow consider as portable the many installations and theatrical works that are set up in one place and then disassembled and reassembled in another place, as it happens during a tour. But their adaptability to different physical environments is the result of often lengthy and tiring manual work, and cannot even be compared to the almost immediate adaptability of augmented environments.
We will make use of the features listed above in the remaining of this work.
Before closing this section, we want  to clarify our approach to AR. Here, we will not deal with AR technologies so much as with the forms of communication produced through AR technologies and usable through HMDs. These forms of communication are what we have previously called augmented environments, that is, those environments that are partly real and partly digital and virtual, and are created by software running on HMDs. However, we want to make it clear that augmented environments - and particularly the augmented environments here under examination, which are spaces for theatrical representations - are "cultural objects" that can (and must) be produced, studied, and used within a variety of cultural contexts and domains of knowledge such as, for example, new media, digital media and digital arts, communication studies, semiotics applied to different media and to different artistic forms, human-computer interaction, and so on.) [6].
In short, we think of AR as a medium, or as a particular system of signification and communication [6], that is: (i) a set of features (i.e., possible genres and formats of the objects that can be produced with the medium, possible types of communication allowed by the medium, etc.); (ii) a set of rules for the formation, organization and manipulation of the features; (iii) a set of conventions (culturally, socially and historically determined, and that people follow when they communicate using technologies) that can be used in certain communication contexts to produce particular and tangible forms of communication (the augmented environments).
3 AUGMENTED TOTAL THEATRE
We said that augmented reality seems to have the potential to overcome the barrier that even the most recent forms of TT collide with. Now that we have a clearer idea of the properties of augmented environments generated through the use of HMDs, we can examine this statement in more detail.
Let's start by defining Augmented Total Theater (ATT). In short, the ATT is a form of theatre based on the principles of Total Theater, that uses the expressive potential of AR to create theatrical representations in augmented environments.
Table 1-4 compare the key elements characterizing most traditional theatrical representations (including TT representations) with the essential elements of theatrical representations that utilize Augmented Reality, i.e., ATT representations.

Table 1: Comparison of Traditional and Augmented Theatre:
PERCEPTIVE SPACE OF REPRESENTATION
Figure 1. The Space of Paratext: Navigation and Help Interfaces


Table 2: Comparison of Traditional and Augmented Theatre:
PSYCHO-PHYSICAL SPACE OF THE SPECTATOR/USER
Figure 1. The Space of Paratext: Navigation and Help Interfaces

(*) Of course, there are many exceptions to these rules, which are primarily defined by traditional forms of theatre. In fact, as previously mentioned, many TT representations of the 20th Century were created specifically to transcend the constraints imposed by these rules and incorporate as many aspects of reality as possible into the theatrical experience. In Table 1-4, our aim is not to rigidly delineate the differences between traditional theatre, TT, and ATT, or to attribute the origin of a certain idea or tool to specific authors or movements. Instead, our interest lies in highlighting the differences in expressive possibilities between what theatre (be it traditional or total) currently is and what it could become if Augmented Reality is utilized.

(**) Typically, the augmented environment that hosts an augmented theatrical representation is designed for a single user wearing a Head-Mounted Display. However, there are instances where multiple users, each equipped with an HMD and a system of synchronized video cameras, can share the same augmented stage.


Table 3: Comparison of Traditional and Augmented Theatre:
ADDITIONAL FEATURE
Figure 1. The Space of Paratext: Navigation and Help Interfaces


Table 4: Comparison of Traditional and Augmented Theatre:
AUGMENTATION OF REALITY - - LOCATION AND TIME OF REPRESENTATION
Figure 1. The Space of Paratext: Navigation and Help Interfaces


Tables 1-4 display the key elements or characteristics that have contributed to defining and shaping traditional theatrical representations (Raw 1), as well as the less conventional, but still analog and mechanical TT representations of the 20th century (Raw 2). Raw 3 (AUGMENTED TOTAL THEATRE), on the other hand, presents some essential characteristics of ATT. By examining and comparing these two columns, we can gain a deeper understanding of the various contributions that Augmented Reality can offer to contemporary theatrical representations.
This is not the place for an in-depth analysis of the tables' contents. As previously stated, in this work, our aim is to lay the groundwork for an analysis of the concept of augmented total theatre, leaving further investigations to future works. However, we encourage readers to delve deeper into the differences between the three forms of theatre outlined in the Tables, and possibly to add new elements or details not considered here. Similarly, we invite theatre professionals to explore how they can leverage all the possibilities offered by the new technological tool known as augmented reality to transform their ideas and visions into tangible and innovative forms of communication.
5 CONCLUSIONS: AUGMENTED TOTAL THEATRE IN ARTS, CULTURE, AND EDUCATION
We have observed that through the integration of virtual elements in a theatrical context, AR opens up new creative possibilities. This allows those who work in theatre to overcome certain limitations and offer even more engaging and immersive theatrical experiences. In this sense, augmented reality can lead to a more mature and enriched form of Total Theater, transforming the theatre of the future into the theatre of the present.
On the other hand, the analysis of Table 1 also prompts us to rethink the very concept of TT. We have seen that some unique characteristics of AR introduce new elements, aspects, or features into the representations of an ideal augmented total theatre that are impossible in more traditional theatre. We understand that AR, thanks to its portability and adaptability, offers us the possibility to extend and transform the traditional space of theatrical representation into a space of representation (almost) as vast as reality itself, ‘where (almost) anything becomes possible’[6].
We realize that this space of representation, which is almost as vast as reality itself, not only involves theater, but also extends to the broader territories of the arts and culture in general, even encompassing the world of education [14], [15]. An art installation, a conference, a visit to a museum or art gallery, a physics or history lesson, and in general, any event that occurs publicly or privately and has a social, commercial, celebratory, cultural, educational, or artistic character can be represented through some form of Augmented Total Theater.
A simple web search is sufficient to find numerous projects, platforms, and applications that implement what we have defined here as representations of ATT across a wide range of fields in the arts, humanities, and sciences [n3]. It doesn’t matter that most of these projects, platforms, or apps still propose limited, imperfect, sometimes somewhat primitive versions of augmented environments that are mostly accessible via mobile phone, and are, therefore, interactive but not immersive. What is important to highlight here is a fact: a door has been opened. The worlds of arts, culture, and education have discovered the existence of augmented reality behind that door. Slowly and with difficulty, through more or less significant products, these worlds are embracing the idea that the reality of the future will be increasingly enriched by a layer of virtuality. They are opening up to the idea that what we have termed Augmented Total Teatre is becoming the most mature way to build a Total Theater with a stage as vast as the world.
Notes
[n1] Recently, the Danish media artist Carl Emil has developed a technology, called 4D Box, that allows to “produce hologram-like illusions for a large audience” [10]. This technology, an alternative to both mobile and HMDs, is very interesting because it can create augmented environments for large audiences rather than for single users at a relatively low cost (without resorting to HMDs). On the other hand, it must be considered that the augmented environments created with the 4D Box lack, at least for the moment, immersiveness and interactivity, so that the spectator of these augmented representations remains outside the space of the representation and maintains an essentially passive role.

[n2] In immersive theatrical representations, the audience is within the representation. But AR is not necessary to achieve immersiveness. Just think of the theatrical experience Sleep No More [13] (2011), by the London theatre company Puchdrunk, which pushes the concept of immersiveness to the extreme, asking the audience to wander freely inside a six-story building with more than 100 rooms, each of which hosts a moment/fragment of two different theatrical works (Shakespeare’s Macbeth and Hitchcock’s Rebecca) appropriately changed and mixed.

[n3] Here, we provide just a few examples of what we have discussed so far. We start with Cosmogony (2022), an almost-augmented reality show with three live dancers dancing at the Gilles Jobin Company studio in Geneva while simultaneously living in real-time in the space of a theatre somewhere else in the World. The dancers are "captured" in Geneva and instantly sent across the network to reappear on the large screen of a theatre hundreds of kilometers away as dancing avatars with changing shapes, and no longer subject to physical laws [16]. Cosmogony is certainly still an imperfect vision of the ATT we talked about, because it proposes a weak non-immersive form of AR, since the viewer is still passive, and remains outside the space of representation. And let's also consider some augmented reality experiences more oriented towards the world of education. We have the Merge Cube [17], a physical cube used together with a mobile app to teach subjects like math, science and history interactively. Or let's consider Labster, a virtual learning platform that incorporates augmented reality to offer online scientific lab experiences, so students can perform virtual experiments and interact with equipment and chemicals safely [18]. Also, we want to mention Anatomy 4D, an augmented reality application developed back in 2014 that offers an interactive way to study human anatomy in detail [19]. Or the more sophisticated application Civilisations Augmented Reality [20], developed by the BBC, which uses augmented reality via mobile phone to bring important historical objects to life and promote interactive learning of art and cultural history. Finally, let's close this short list by mentioning what at the moment, thanks to the involvement of technology giants such as Meta, Microsoft, and Apple, promises to be an entire augmented, interactive and immersive world with great potential in education: the Metaverse for Education, an immersive augmented and virtual environments where students and teachers can interact [21].


REFERENCES
[1] R. Wagner, A. H. Goldman, and E. Sprinchorn, Wagner on music and drama: a compendium of Richard Wagner’s prose works. in A Da Capo paperback. New York, N.Y: Da Capo Press, 1988.
[2] E. T. Kirby, Total Theatre. A Critical Anthology. New York, NY: E. P. Dutton & Co., Inc., 1969.
[3] "What is Total Theatre? | Total Theatre Magazine Print Archive." http://totaltheatre.org.uk/archive/features/what-total-theatre
[4] C. B. Balme, The Cambridge introduction to theatre studies. in Cambridge introductions to literature. Cambridge, UK; New York: Cambridge University Press, 2008.
[5] C. Innes and M. Shevtsova, The Cambridge Introduction to Theatre Directing, 1st ed. Cambridge University Press, 2013. doi: 10.1017/CBO9781139016391.
[6] S. Cicconi and M. Marchese, "Analysis of an E-Learning Augmented Environment: A Semiotic Approach to Augmented Reality Applications," in ICERI2019 Proceedings, Sevilla, Nov. 2019, pp. 4921–4931. doi: 10.21125/iceri.2019.1204.
[7] S. Cicconi and M. Marchese, "‘Augmented Classrooms: A Generator of Augmented Environments for Learning,’" in INTED2023 Proceedings, Valencia, Jun. 2023, pp. 3405–3413. doi: 10.21125/inted.2023.0929.
[8] S. Cicconi and M. Marchese, "Augmented Learning: an E-Learning Environment in Augmented Reality for Older Adults," in INTED 2019 Proceedings, Valencia: IATED, 2019, pp. 3652–3662. doi: 10.21125/inted.2019.0937.
[9] R. Azuma, "A survey of augmented reality," Presence Teleoperators Virtual Environ., vol. 21, no. 6, pp. 34–47, 1997.
[10] "Crossing into the 4th dimension – how Carl Emil Carlsen creates generative art with Unity | Unity Blog." https://blog.unity.com/games/crossing-into-the-4th-dimension-how-carl-emil-carlsen-creates-generative-art-with-unity
[11] B. MacIntyre, J. D. Bolter, E. Moreno, and B. Hannigan, "Augmented reality as a new media experience," in Proceedings - IEEE and ACM International Symposium on Augmented Reality, ISAR 2001, IEEE, 2001, pp. 197–206.
[12] Lev. Manovich, The Language of New Media, vol. 27. MIT Press, 2001.
[13] "The McKittrick Hotel | Home of Sleep No More NYC & Gallow Green." https://mckittrickhotel.com/
[14] M. Akcayir and G. Akcayir, "Advantages and challenges associated with augmented reality for education: A systematic review of the literature," Educ. Res. Rev., vol. 20, pp. 1–11, 2017.
[15] B. Ralhan, "AR: Transform your surroundings to learn anytime, anywhere!," 2017.
[16] "Cosmogony di Gilles Jobin," MEET, Oct. 27, 2022. https://www.meetcenter.it/it/event/cosmogony-di-gilles-jobin/
[17] "MergeEDU: Learn Science, Master STEM, Be Future Ready. | AR/VR Learning & Creation," Merge. https://mergeedu.com/
[18] "Labster | Virtual Labs for Universities and High Schools." https://www.labster.com/
[19] New Update to DAQRI’s Anatomy 4D App- The Human Body, (Jun. 02, 2014). [Online Video]. Available: https://www.youtube.com/watch?v=gKkgCc85vBc
[20] "Civilisations AR," BBC Taster. https://www.bbc.co.uk/taster/pilots/civilisations-ar
[21] S. Hussain, "Metaverse for education – Virtual or real?," Front. Educ., vol. 8, 2023, [Online]. Available: https://www.frontiersin.org/articles/10.3389/feduc.2023.1177429