VERTIGO is funded under the H2020 European STARTS initiative, innovation at the nexus of Science, Technology, and the ARTS. STARTS promotes the arts as catalysts for efficient conversion of science and technology knowledge into products, services, and processes.

August 16, 2019

August 1, 2019

August 1, 2019

Please reload

Latest

AIBO Inception Weeks

August 16, 2019

On Tuesday, July 9th project manager Patricia Delgado and project sponsor Aleksandar Valjamae from Goprosocial at Tallinn University and I signed the Vertigo STARTS Laureate residency contract to develop AIBO, an emotionally intelligent, artificial intelligent brainwave opera.

 

                              Patricia with the copies of the contract

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                                      Aleksandar writing up a summation of the workflow of AIBO in his office at                                            the Department of Human Computer interaction at Tallinn University.

 

The next day I went over to EKA, the Estonian Academy of Art to start working with my technical director, Hans Gunter Lock.

 

                            Flags outside the new Estonian Academy of Art

 

                             Front door into the academy and its reflective surface

 

                            Hans wearing an Emotiv headset to test the Max/MSP patches.

 

                            Writing up our routing and workflow.

 

 

                         My going over the specifics with Hans

 

We discovered quite early on that the middleware that connects the Emotiv EEG                                      brainwave headset called Mind Your OSCs and the measuring of the four emotions of                              interest, excitement, meditation and frustration did not work. The new version of the                                  software development kit (SDK) that Hans used was missing emotional detection. 

 

 I posted a series of urgent messages on the Emotiv development Github. Github is the                             website where programmers all over the world post their code. This is what their very                               responsible developer Hoang Pham wrote back to me -

 

So what that meant is that they took all functionality out of the software that we critically need, and they will probably write it back in on a new product called the Cortex, but that won’t be available for a few months at least. Fortunately I have an older version of the hardware and software, but did not bring it with me to Estonia, as I did not think I would need it. Hans and I will develop for it using a workaround, hopefully it will be successful.

 

I was pleased that Kristi Kuusk who is a really great smart textile designer at the Estonian Academy of Art and won a STARTS Residencies grant last year for her piece “Magic Shoes” came to visit before her summer holidays. Kristi and one of her students will help design the body suit of light that will be part of the performance. 

 

                                                 Kristi drops in to say hello

 

 

Outside the Estonian Art Academy

 

 

 

Week 2 at EKA - Estonian Art Academy                          

 

Inner courtyard of Estonian Art Academy

 

 

Here I am working away at the media lab editing 140 videos in a marathon editing session

 

 

 

The lovely key card pass that allows access to the lab

 

 

Hans Guther Lock at EKA, as it is called in Estonia, got me a visitor’s card that lets me come and go in the media editing studio, which is exactly what I needed.

 

But that was not all. Hans worked on a way to display the videos and the text in Max/MSP.

                      Hans working away on coding Max/Msp for the project

 

It was now time for Hans to meet some of the other collaborators. The first was my fantastic composer Peter Michael von der Nahmer, or as I call him “Mike”. We did this by Skype, since Mike was in calling from the midwest of America.

 

 

                        Mike on a Skype with Hans and me in Tallinn at EKA

 

Mike talked about composing databases of sound based both on emotion, and how they interact with the brain. Considering this is a brainwave opera with AI, that makes a lot of sense.

 

Then Hans and I had a call on Zoom with two of the team’s AI programmers, based in Singapore, Wilson Tjhi and Jonathan Heng, programmers at ThoughtWorks.

 

 

Here is Hans talking, I am in the upper left, then Jonathan and then Wilson.

We discussed how to get the AI’s answers out of the computing cloud to a format where Hans can use them in Max/Msp - this is quite important.

 

So I put in at least 40 hours of video editing this week, as well as these Skype and Zoom calls, plus research and coordination between team members - and we are just getting started. I have less than a week left in Tallinn, still so much to accomplish.

 

 

Week 3 - Finishing Up in Tallinn

 

 

Hans and I began by doing a planning session in his office. Things got pretty intense rather quickly.

On of the most important things we did was take the draft libretto from the Eva character, and translate it into text that could project onto a screen. This was in anticipation of it transforming into the cloud where the test AIBO database that Wilson and Jonathan were working on could be tested to see if it could actually return an answer. It did. The YELLOW is the Eva text, and the BLUE text is the successful reply from the ABIO database.

 

 

This was an incredible breakthrough. 

The next day Alek and a cameraman came to EKA to interview me for a film they will be working on about the project.

Setting up the camera

 

Here I am looking very intense for the interview

 

After more days of editing videos it was finally time to leave Tallinn.

 

My favorite graffiti art from Tallinn - from Mad Magazine's Spy vs. Spy  

 

 

The next day I went to Riga, Latvia, where I had a meeting with my performer, Sniedze.

 

Sniedze Strauta, who will play the man character EVA , in Riga.

 

 

Once I returned to NY, my costume designer Dila Demir, who is also at EKA, sent me some sketches and prototypes for a body suit of light. We will make many adjustments, but it is a start.

 

Sketch 1

 

 Sketch 2

 

Prototype 1 

 

Things are really underway, and this short time in Tallinn showed a lot of progress

 

 

 

  

 

 

 

 

 

 

Share on Facebook
Share on Twitter
Please reload

Search tags
Please reload

Archives