System Design for the live processing of video material

System Design for the live processing of video material

In this modern day transcription of the play, we are called to create a dynamic environment surrounding the audience and changing during the course of the play. This will be made possible through the installation of multiple sources of picture and sound and light around the audience, which will create the sense of multi-dimensional action. The use of interactive systems allows for the immersion of the spectator in this environment; the spectator’s reactions will co-defines the overall aesthetics of the play.

One of the basic needs for the design of a system that will process live visual material is adaptation in different spaces of presentation of work, such as theatres, foyer and open-air spaces. This need led to the design and implementation of a flexible system, constituted from interconnected units of image processing. This way, depending on the particular venue a custom structure is built with the respective interconnection of these units, covering thus both the technical needs of the show and the team’s collective creative expression. Such needs are for instance a change in the number of image sources (projections) so that each particular place is covered, the effortless re-arrangement of the visual effects layout (so that the desired aesthetic result is achieved) and the connection of an external control device for the live mixing and editing of visual data.

An increase in the number of projections in the venue, has an analogous effect on the processing power needed, with a single computer not being able to handle the workload. Hence the network of personal computers used, one for controlling the others, and the others for image rendering.



Two software packages have been used for the generation, processing and control of the video material:
Max/MSP/Jitter for the creation of a series of management and processing units, which are used according the presentation condition: video playback, visual effects, picture separation, to name a few.
SuperCollider controls the aforementioned units and the inter-computer communication. Both software packages are in direct communication through the use of the OSC [open sound control] protocol.




Interactive Condition

In modern performance, technological means are the sine qua non of the creation and the expression of the play-act-drama. Interactivity in its bare-bones version entails the communication among the performer, the musicians and the public. Breakthroughs in technology led to the establishment of a new relation – between man and personal computer. This opened new grounds for change in performance art.

Medea Electronique suggests a convolution of different art forms, such as theatre, dance, sonic art, video and music under an interactive condition, providing the communicative means that will give the artist/ creator with the possibility to express all man’s experience. The spectator for his part exposed in this rich breadth of stimuli experiences a particular and “extended” experience.

In brief, the aim behind the designing of the interactive systems for the present application is the recognition of human body movement and its conversion to digital data, using a personal computer. These data are used for the controlling of the sound and visual result as the play develops.

Taking under consideration the need for the presentation of the play in different types of venues, two different interactive systems have been developed, focusing on the performer’s actions and audience’s movement respectively.

1. Interaction between performer and computer: In plays taking place in theatres and other venues where people are seated, the interactive system analyses the performers motion, providing him with the ability to add to his expression arsenal new means, on top of movement and speech.
2. Interaction between audience and computer: In cases where the venue does not have seats – foyers, lounges, open-air festivals – the sensors capture the action/reaction of the public, thus influencing the visual environment and lighting conditions of the show. The interaction between the audience and the computer increases the former ’s participation and thus its holistic experience; this way the outcome of the act is - to an extent - modified.



Design of the Lights Control System

In our aim to create a dynamic environment, the fixed and mobile sources of lighting function as the unifying element for the sense of sight. Briefly put, they express the sentimental alternations of the performers in each scene and unify the multiple sources of visual material (projections) in the space. Further, with the combined use of interactive systems they express at times the action performer and when the spectators, depending on the system and the space of presentation of work.

The controlling parameters of the lights system come from three data inputs, which are blended together depending on the circumstances.

1) manual settings stored in memory presets
2) data from the performer’s movement recognition
3) data from the audience’s movement recognition



Technology

The system used for controlling audio, video and lights is based on a multimedia platform designed and developed by Panagiotis Tsagkarakis and Iannis Zannos (Head of the “Audio – Visual” department, Ionian University). It was firstly presented in the “Programming Interactive Installations” workshop (“Info-System” exhibition, Hellexpo, Thessaloniki 2006).