type of project: AI Assisted Theatrical Design

published: 2021

by: Elena Tilli & Samuel Chan

website(s): https://www.elenatilli.com https://www.samuelcvlx.com

maintainer(s)/contact: nena.tilli@gmail.com info@samuelcvlx.com

repository:

AI Assisted Theatrical Design (Elena Tilli & Samuel Chan)

From Text to music, from picture to video, different tools of Artificial Intelligence have been used in art making. Could AI also be applied to theatre making, particularly theatrical design, which is a more complex “problem” for the machine?

Coming from a theatrical design background, the research team would like to explore if these AI tools could do us any good in “assisting” theatrical design.

This X doesn't exist, Theatre version

reference website: https://thisxdoesnotexist.com/

1.0 “This picture doesn't exist in the production of XXX”

Using GAN, and time lapse pictures of certain production as a dataset as Discriminator to create a picture that never appears in a production.

1.5 “This production doesn't exist in the theatre xxx history”

Same idea, but with different dataset. The dataset could be the pictures archive of a theatre company.

1.6 “This dance pose doesn't exist”?

Collect dance pictures for the same purpose?

  1. High Resolution Style GAN? Pipeline?
  2. StyleGAN with less sample?

Dataset:

TN (15000 Steps, trained with pretrain “landscape” model)

Log 20210921:

Image generated from the model lack diversity even with 1.0 Truncation and 1.0 Sampling Distance. This might due to the lack of diversity of the original model, and lots of similar pictures in dataset.

Some sample processed with gigapixel (with watermark)

Action: Removed preshow water projection images and reuploaded new dataset, also might try to use another pretrain model to see if it make any different.

Log 20210923:

Tried inverted dataset in photoshop and trained in runway ML again, didn't made much different after inverting again in photoshop

YSD2 (10000 Steps, trained with pretrain “illustration” model)

Seagull Opening (10000 Steps, trained with pretrain “illustration” model)

Limitation of Runway:

1. Size of pictures input is limited to 320 X 320, can't process more details that are needed for a bigger “scene”

2. Algorithm limited to the original stylegan2 published by Nvidia

Next step: try to create model that run locally and with a higher pixel (1024 X 1024)

Question:

1. How long will it takes for an algorithm to process such a high resolution image with mid range GPU like RTX2080 TI?

2. Could we use 4:3 or 16:9 image? (seems impossible)

For presentation:

Create an interactive website/app that show a grid of real and fake images

STATE OF THE ART

AttnGAN run on the App Runway ML seems to be the most accessible Algorithm to transform the text into images.

Starting from research material collected about the dramaturgical topic of our research, it is interesting to see how we can export a series of images that keep changing while adding letters and word in a word that becomes a sentence, then a paragraph and so on.

Here the complete text been digited:

Lilith is the story of a woman who threatens to shake the very foundations of male-centered faith and discover a new self. Her name means 'night monster' and is derived from the name of the Babylonian female demon Lilit whose male counterpart is Lilu. Lilith did not like to live as the subservient wife of Adam and hence went to the deserts, desiring solitude and freedom. Three angels tried to force her to return, but in vain. She is also described as a vampire child-killer like her classical counterpart Lamia. The evil threatened by her against children was said to be counteracted by wearing an amulet, which bore the name of the three angels.

  • projects/aiassistedtheatricaldesign/start.txt
  • Last modified: 18.10.2021 18:36
  • by Samuel Chan