An Open Space event Hosted by Royal Holloway, University of London and Improbable
Outline
What are the possibilities for Augmented and Virtual Reality within the context of theatre?
An Open Space event Hosted by Royal Holloway, University of London and Improbable
Held online 16th July 2020
Contents
Learning how to use virtual reality technology. 3 Feedback on presentation and brainstorm on how NEOS VR (social VR) can facilitate creative collaboration.. 3 How can we create a sense of shared social space in AR and VR?. 3 How do we ensure diversity is embedded from the beginning of this burgeoning artform?. 5 Notation Tools for Dramaturgy. 6 What are the scenographic possibilities of AR/VR (spaces + materialities) 7 Me & You: Intimacy in VR/AR.. 8 Care of participants & ethical considerations of VR.. 9 Digital Participatory Theatre – using the tools to participate with equality and access.. 10 Who should a theatre director be collaborating with to incoproate VR/AR into a new show?. 10 How can we use AR/VR/MR as a means to reforming the theatrical experience of disabled audiences during the Pandemic and long after?. 12 VR/AR/MR Theatre outside the field of theatre.. 12 Shared Languages of hybrid forms.. 14 Creating theatrical experiences in VR that would be financially impossible in embodied space.. 14 What platforms/pieces of software are needed to develop the art-forms? 14 What are the specific possibilities of Augmented Reality?. 14 The Art Room.. 15–
Learning how to use virtual reality technology
Session Convener: Hortense Participant names: No notes.–
Feedback on presentation and brainstorm on how NEOS VR (social VR) can facilitate creative collaboration
Session Convener: Joris Weijdom Participant names: Hortense Gerardo, Nadja Lipsyc, Andy Lavender, Sarah Sigal, Chiel Kattenbelt, Rebecca McCuthcheon, Annie & Paul – Any One Thing, Sophie, Any One Thing AL – is it a live space or a virtual space? JW – the goal was to show the design of the piece, rather than a reconstruction of the virtual experience. AL – was there a difference between the presentation vs what a visitor would have experienced in Prague? JW – enter the building, one of 4 stations. Take off shoes and wait your turn, part of the experience.–
How can we create a sense of shared social space in AR and VR?
Session Convener: Aneta Participant names: Tom Parkinson, Maya Chowdhry, Lucy Thornett, Sarah Spil, Imanual, Aneta Mancewicz, LIn_C, Jonathan Hamilton Lucy – University of the arts, Phd – Audience + AR. Favourite animal is a bear. Maya – freelance digital theatre-maker, interested in using MR/XR to create works that are participatory. Favourite animal = albatross Tom – composer, works in theatre and dance, lecturer in composition in Royal Holloway. Favourite animal = owl Imanuel – Hamburg, dramaturg. Lecturer performance studies. Participatory work. Bring VR/AR to more open participatory to urban planning. Who does it mean to be a producer in shared social space. Favourite animal = dolphin Sarah – member of an amateur group that performs in small venues. Animal = kangaroo Aneta – lecturer in drama and theatre arts at. Animal = Koala A screen can be disruptive – it detaches us from what we could see around and makes us feel alone Sound offers an opportunity to connect that maybe a virtual image does not A shared private space, where people could invite someone and the experience could be shared One-to-one experience is easier More people connecting is difficult; the experience of VRHAM- Festival (for VR pieces) Having a collective purpose makes people more connected an intention for the audience is important having an imprint on the virtual space is effective and makes us feel part of the experience, We live in the ocean of Air and Marshmallow Laser Feast (Saatchi Gallery) – it was abstract and played on breath, pulse and energy in making others visible rather than trying to replicate a person through a digital avatar Watching others experience VR can also provide a sense of participation Group VR experience at Abandon Normal Devices Festival https://www.andfestival.org.uk/we-dwell-below/ interactivity eye contact is not possible.. paralell conversations are not possible Virbela VR platform – everyone is an avatar in space, can have paralel virtual spaces. But network issues/capacity/bandwidth. But always avatar not real person. Using Zoom so consistency of platform. But body language/meaning of the person Eye contact in public space – oxytocin is released when people have an eye-contact Sharing obvious emotions through the expericence How sound can be the binding thing, sound creating a sense of shared space In Zoom we can’t talk over each other, mask environmental sound and privaledge the voice. Technologically facilitating natual conversations. We learn ways of speaking and attempt to speak over each other less. Vocal technolgoies privaledge certain voices. Normalitivity – frequency bands, certain voices make their presence felt and are picked up by the Zoom codex. Being playful and having common tasks creates a sense of shared space playing helpsHow do we ensure diversity is embedded from the beginning of this burgeoning artform?
Session Convener: Sophie (Any One Thing) STAY IN TOUCH: sophie@anyonething.co.uk / @sophielarsmon @any_one_thing Attended by Paul, Annie, Josh & Andy Within this discussion, the term ‘diversity’ focussed on Racial diversity and accessibility for Disabled Artists.- As “traditional” theatre seems to be taking this enforced pause in proceedings to have a moment of reflection which includes dialogue and action regarding diversity within its art-form (e.g. Culture Reset; What’s Next?, Inc Arts*), we asked what action can we take to ensure diversity is included in the conversations around the creation of VR & AR content from the very start.
- So where do we start?? Perhaps looking at the entry points to the industry…
- Training Institutions (including Universities like Royal Holloway; Conservatoires like Guildhall where Andy is a member of the Faculty; other Drama Schools?) Within Higher Education settings, steps are being taken to take action in this area, e.g. Andy presenting a plan to the Principles of Guildhall. This was deemed to be very important to the group as these are often where talent is first spotted and developed. However….
- Finding & Attracting the new talent We need to look outside of our echo chambers to encourage and nurture the new or soon to be new artists in this field. HOW DO WE REACH INTERESTED PEOPLE?!? These people might not know of the training opportunities available. What courses are out there other than StoryFutures Academy? And how do we advertise courses/opportunities/commissions to a wide pool of people? No specific answers were found within the group, but there are resources at the end of these notes that could provide assistance in creating action
- Funders setting requirements: Just as Arts Council insists on diversity quotas and targets in order to qualify for funding within the publicly-funded sector (although arguably, not enforced enough), who is setting the ‘requirements’ for the Immersive Technologies World? Sad that Diversity often relies on having to tick boxes, but until true diversity is intrinsically ingrained, these nudges are still required. Positivie discrimation – WHO WILL BE RESPONSIBLE FOR SETTING THESE QUOTAS WITHIN VR/AR?
- How do we capitalise on the multi-sensory nature of immersive technologies to encourage the creation & consumption of content by disabled artists / audiences?
- Josh from Graeae asked the question “how does someone entirely new to this art form even begin to get an understanding for what’s possible?” An education is needed in order to be able to convince his Artistic Director (and many more), that VR & AR is worth exploring. WHERE DO WE START?? Is there anyone out there leading on this work? Please signpost information to us if you are.
Notation Tools for Dramaturgy
Session Convener: Imanuel Participant names: miro as a tool to collaborate how to annotate, concept and work as a dramaturg with – multiplayer – multi linear – MIT – BeAnotherLab – avatar – embodiment. technology is ahead of our possibilities to write it downWhat are the scenographic possibilities of AR/VR (spaces + materialities)
Session Convener: Lucy Thornett Participant names: Maya Chowdhry, Lucy Thornett, Lin C, Joris Weijdom, Theo, Imanuel, Joris Weijom, Andy Lavender, Sarah Sigal What dramaturgies and narrative strategies required? The materiality and audience space relations. Lucy wrote article on this topic. Stenographic strategies? Digital scenographer Intermedial textures – how they function when they’re digital? From a phenomenological perspective. Filters in video, digital extraction. Play with what it represents, what it can be on itself. Link that involves the body Sensory misalignment? Gap between worlds, what seeps in? Digital sand and real sand under your feet in Joris MR piece v’s trying not to pretend that it is there. CREW – video 360 – 60 audience members and 30 on headset, wearing video screen – tactility, involving your body. Brain triggered by new and unclear, wakes up. Align or disabling the illusion. Multiple overlapping worlds in scenography. Blur of borders between different worlds. Digital Copy of what imagined? The technology doesn’t disappear, so draw attention to it. When you enter the space is part of the design, not when you put the helmet on Site-specific scenography, what is the audience looking at? Eyes shut. Scenography of the memory evoke. We like to augment reality with what’s happening in our brains. Sound as an immersive AR technology. brain filling in the gaps. AR/VR suggestive liminal space. Embrace glitchyness and synthetic? What can virtual space do that the real world can’t? 3D scanning of enviroments and use mesh. Glitchy, blocky weird. Exciting aesthetic v’s too precise and too real. 4D scanning of actors in the uncanny valley. Strangeness of real, not real. Minecraft and low-fi possibilities of what you can do, construct, gravity in there. SecondLife – fly, construct. Older and newer AR – marker and markerless – spatially aware. People physically relating to what you’re digitally overlaying VR helmet knows where it is in the space AR kit, put doorway in your phone and walk through in real world parallel Mapping to real space – placing heat fan etc. Portal AR, show landscapes in Scotland Designing a show, many storylines, many frames, many interactions, spaces, objects. Portals, possibilities, audiences. How do we notate this? 2D? 3D? Document? Discuss? Rehearse? A multi-player Collaboration team, communication, talking imagination needed. LARP systems? Immersive theatre, e.g. punchdrunk? But keyframes and timelines clear Minecraft = more open world Game designers? Pervasive Games: Theory and Design Book by Annika Waern, Jaakko Stenros, and Markus MontolaMe & You: Intimacy in VR/AR
Session Convener: Joanna Bucknall (University of Birmingham) Participant names: Joanna Bucknall-practitioner scholar-interested in one-on-one performance and audience-centric experience that reframes female social spaces and rituals I have been using audio tech to augment intimate experiences between my ‘guests’ and myself during one-on-one performance but have been wondering what other options for AR might be available. It is also a sense of access too-where can the possibilities be uncovered and what are the opportunities to ‘play’ with the tools that are already at hand or currently being developed? In my work I task my ‘guests’ to work with me in that moment to create or generate something and I find myself wondering how this can happen in the virtual-what does me and you mean in a virtual space? What tools are there that we might use to build something together, live in -real time together? I have felt a pressure to create something digital during COVID but found that the tools I have access to simply don’t offer me the opportunity to find ways of making intimate connections and I have resisted making work for the digital platforms that have become so dominant during the lockdown. As I sit here on my own I wonder what constitutes an invitation, how does a digital hand get extended into a ‘join me’? How do we come together in this liminal site to generate something between us that is meaningful? I feel my way through the experiences that I create with my ‘guests’ it is a shared energy and endeavour that is rooted in presence and a sense of shared space but I don’t know how to feel my way across the noughts and ones of digital space with the same intuitive care that informs my work in the material world. I am going to step out of this space now but if you come in, please feel free to respond and record your thoughts. Perhaps this suggests that intimacy an sharing can be undertaken separately (or rather involve asynchronous encounters) you can have your body in the virtual space through headsets-build a domestic space that can be met in how do we meet with our breathe, our eyes-the chemicals this produces? Haptics-sensation what is the cost? bringing together the tech etc when it is quite ‘cheap’ to have real bodies in space Competence Access-what can access-where do we play-how do we know where to play? In lockdown we miss touch but perhaps have been more connected-how do we touch? Dreams-hugging as transgressive adaptation-from the material to the digital-migration?Care of participants & ethical considerations of VR
Session Convener: Hortense Participant names: No notes.Digital Participatory Theatre – using the tools to participate with equality and access
Session Convener: Maya Chowdhry Participant names: No notesWho should a theatre director be collaborating with to incoproate VR/AR into a new show?
Session Convener: Sophie STAY IN TOUCH: sophie@anyonething.co.uk / @sophielarsmon / @any_one_thing Participant names: Sophie & Theo An intimate conversation between a sound designer who is experienced in Immersive Technologies and a Theatre Director who is an Immersive Technologies novice: S: Where do we start?? T: So much of the development process for this artform has to be play; making work like this is very hard work [but a lot of fun], and more time is needed to develop new practices. So often the projects that go ahead have vast sums of money behind them and the producers want bang for their buck: this means time for genuine exploration is sparce. What excites me is the prospect of making content for the ‘extra moments’ which are not the same for everyone. More interesting than the “A track” – big bang moments S: It seems these financial barriers are preventing us from creative innovation. Are their software out there that could reduce the financial barriers? T: That’s what we need. New software need to be developed in order to speed up the iteration processes. The iteration cycles in Immersive Technologies are incredibly long. In theatre, we have the ‘tech’ period, and we can stop & start to re-programme cues or re-stage an actor in the space. In the digital sphere, this sort of change could take up to 2 weeks! Audio within the Immersive sphere has managed to streamline their workflow to a large extent; the big roadblocks are modelling animation & level design. It needs to be quicker, cheaper, easier. Less clunky mo-cap or use of Hololens could be useful in development. Perhaps we should also be focussing on smaller-scale R&D / tech demos which would allow for more time to play? Perhaps we also need to lean into the Low-fi (like Minecraft). Working like this rather than the CGI/movie-like expectations might allow us to create more interesting content / emergent moments / audience building their own narrative… S: Yes – theatre audiences are used to using their imaginations to fill in the gaps – it’s an abstract artform unlike film which can capture naturalism better. Perhaps we can encourage these audiences to use similar tools when consuming VR/AR content – allow for a more shared experience (and would also be cheaper!) – and help avoid the uncanny valley issue. S: What software do I need to learn? T: In terms of Game Engines, Unity is a good prototype tool; you do need to know how to code (it’s possible to self-teach – you don’t need to be a Computer Programmer to do this). In Unreal Engine, there is the option to use graphical scripting language (less coding required). S: Where do the current content producers come from? T: A lot of content currently is often created by digital / multimedia agencies. Digital producers (crucial role) often come from this world too. Digital Producers are key to the development of this artform. They need to be a translator to be able to be communicate between worlds/industries. There are a few out there, but more are needed. More will develop as the number of projects increase and therefore people’s experience. S: What other roles are important within the creation of a live piece of work that uses VR? T: The ppl who install the CCTV! Watching how audiences react during previews is crucial to the creative development of the piece – spying on the audience and seeing what works and what doesn’t. S: How about the role of the writers/dramaturgs? T: Look to the role of the writers within game design; it’s now seen as crucial. Perhaps it’s time to look at that world and see how those writers are creating strong narrative drives WITH audience agency (not sacrificing one over the other). Technical Directors crucial who has experience of things that go wrong Ps. as a theatre company who can’t make live theatre right now, we have been delving into the world of VR & 360 Cinema and getting inspired by those who are more experienced than us. Here’s a dedicated YouTube page which is home to our discoveries: bit.ly/2ZudjP9How can we use AR/VR/MR as a means to reforming the theatrical experience of disabled audiences during the Pandemic and long after?
Session convener: Josh No notes.VR/AR/MR Theatre outside the field of theatre
Session Convener: Imanuel Schipper Participant names: Sarah Sigal, Maya Chowdhry, Sarah Spil, Aneta Mancewicz, Jonathan Hamilton City as character in AR works VR in civic planning, getting an audience to visual space As citizens are there planning or political processes in VR VR/AR re-invision worlds post-covid-19? Use to reconfigure spaces for social distancing Difficult to comprehend and visualise unless you’re in the space. New construction of academic buildings – citizen participation, how should we work together. Can we make decisions in VR Maya experience of IOT and smart city consultation experience: https://cityverve.org.uk/what-is-cityverve/ Clean air inside buildings – do citizens know what they want Development of whole city parts – what does it mean? How would it feel like to be there? What about sound? A street without cars – stand in the middle of the street, helpful for decision making Change in urban spaces, empty office spaces. Pedestrian streets Public space corporatized Buildings as investment – empty, financial purpose Participation in urban planning? Citizen’s not really involved in choosing the topics, outreach, but mainly tick boxes. Feels like they don’t really want the opinions, or not implement anything. Future Everything festival – has created physical pop up citizen venues for debating urban planning. Participatory work with dogwalkers identifying spaces in the city for refugee camps Having the knowledge to read plans and understand what it means AR to place a new building, would I be in the shadow? Housing doing a virtual tour of a house and send to a prospective person’s phone Jewellery Quarter in Birmingham – stories about people buried in cemeteries In Germany – planning participation – are the citizen’s empowered to do this? It took 3 years longer with a public library project, improved project. Users know what they want to use. Design Storytelling – user journey experience emotion Participatory theatre work and AR – how do you draw citizens in the project when working in a remote model. How long can you sustain using this methods. Time – privilege of time to participate in participatory artworks? In past but also during Covid-19 Digitality – Pokemon Go game, people moving around the city, walking. Go to new places. Gamification – hunting, collecting, meeting, finding. Geolens with apple Nexus animation studio doing work not theatre work, e.g. sports: https://nexusstudios.com/work/arvr/ AR in football Technology and devices restricting the story AR in museums and heritage sectors Being with an AR curator on tour Bringing objects from a museum into your own room: https://nexusstudios.com/work/arvr/ https://play.google.com/store/apps/details?id=uk.co.bbc.civilisations&hl=en_US IntersectionsShared Languages of hybrid forms
Session Convener: Joanna Bucknall Participant names: Joanna Bucknall-scholar practitioner at the University of Birmingham. I make one-on-one work that interrogates domestic spaces and social rituals through intimacy and well being approaches (Joanna) I am really interested in the terminology that is used to describe encounters, participants -the ontological and epistemological concerns that these raise for the various forms of ‘experience’. I was really struck by the term puppeteer yesterday and the usage of immersion what gets lost in translation between disciplinary approaches and what is shared? Do different roles or definitions for the ‘audience’ come with differing ethical considerations/obligations/duty of care? I am very happy to be contacted at j.j.bucknall@bham.ac.ukCreating theatrical experiences in VR that would be financially impossible in embodied space
Session Convener: Adam Participant names: No notes.What platforms/pieces of software are needed to develop the art-forms?
Session Convener: Sophie Participant names: No notes.What are the specific possibilities of Augmented Reality?
Session Convener: Lucy Thornett l.thornett@lcc.arts.ac.uk Participant names: Lucy Thornett, Joanna Bucknall, Adam Lenson, Chiel Kattenbelt, Sarah Spil, Aneta Mancewicz, Imanuel, Maya, Theo VR is more often discussed than AR We need tech playgrounds to try out technologies and work out their possibilities what do we mean by AR? How do we facilitate play/onboarding and competence? DIY-hands on in order to feel how these technologies work Tension between difference between creating new sensory experiences and making connections – what can that do for experiences of space? Embodied/gestural relation to device and to space AR not tied to a particular technology or set of technologies, makes it harder to define How might we subvert habitual use of technology to navigate space? Could AR facilitate the blurring of boundaries between art and the everyday in interesting ways? How does the layer over the physical world occur? What are the limitations of the current technology? How can AR transform the physical environment rather than just overlay onto it? How does sound augment space in more interesting ways? AR could be quite theatrical – lineages of theatre that understand how to work with layered realities Magic dust-conjuring layers into being in the material worldThe Art Room
