AIBO Inception Weeks

Aug 16, 2019

On Tuesday, July 9th project manager Patricia Delgado and project sponsor Aleksandar Valjamae from Goprosocial at Tallinn University and I signed the Vertigo STARTS Laureate residency contract to develop AIBO, an emotionally intelligent, artificial intelligent brainwave opera.

Patricia with the copies of the contract

Aleksandar writing up a summation of the workflow of AIBO in his office at the Department of Human Computer interaction at Tallinn University.

The next day I went over to EKA, the Estonian Academy of Art to start working with my technical director, Hans Gunter Lock.

Flags outside the new Estonian Academy of Art

Front door into the academy and its reflective surface

Hans wearing an Emotiv headset to test the Max/MSP patches.

Writing up our routing and workflow.

My going over the specifics with Hans

We discovered quite early on that the middleware that connects the Emotiv EEG brainwave headset called Mind Your OSCs and the measuring of the four emotions of interest, excitement, meditation and frustration did not work. The new version of the software development kit (SDK) that Hans used was missing emotional detection.

I posted a series of urgent messages on the Emotiv development Github. Github is the website where programmers all over the world post their code. This is what their very responsible developer Hoang Pham wrote back to me -

So what that meant is that they took all functionality out of the software that we critically need, and they will probably write it back in on a new product called the Cortex, but that won’t be available for a few months at least. Fortunately I have an older version of the hardware and software, but did not bring it with me to Estonia, as I did not think I would need it. Hans and I will develop for it using a workaround, hopefully it will be successful.

I was pleased that Kristi Kuusk who is a really great smart textile designer at the Estonian Academy of Art and won a STARTS Residencies grant last year for her piece “Magic Shoes” came to visit before her summer holidays. Kristi and one of her students will help design the body suit of light that will be part of the performance.

Kristi drops in to say hello

Outside the Estonian Art Academy

Week 2 at EKA - Estonian Art Academy

Inner courtyard of Estonian Art Academy

Here I am working away at the media lab editing 140 videos in a marathon editing session

The lovely key card pass that allows access to the lab

Hans Guther Lock at EKA, as it is called in Estonia, got me a visitor’s card that lets me come and go in the media editing studio, which is exactly what I needed.

But that was not all. Hans worked on a way to display the videos and the text in Max/MSP.

Hans working away on coding Max/Msp for the project

It was now time for Hans to meet some of the other collaborators. The first was my fantastic composer Peter Michael von der Nahmer, or as I call him “Mike”. We did this by Skype, since Mike was in calling from the midwest of America.

Mike on a Skype with Hans and me in Tallinn at EKA

Mike talked about composing databases of sound based both on emotion, and how they interact with the brain. Considering this is a brainwave opera with AI, that makes a lot of sense.

Then Hans and I had a call on Zoom with two of the team’s AI programmers, based in Singapore, Wilson Tjhi and Jonathan Heng, programmers at ThoughtWorks.

Here is Hans talking, I am in the upper left, then Jonathan and then Wilson.

We discussed how to get the AI’s answers out of the computing cloud to a format where Hans can use them in Max/Msp - this is quite important.

So I put in at least 40 hours of video editing this week, as well as these Skype and Zoom calls, plus research and coordination between team members - and we are just getting started. I have less than a week left in Tallinn, still so much to accomplish.

Week 3 - Finishing Up in Tallinn

Hans and I began by doing a planning session in his office. Things got pretty intense rather quickly.

On of the most important things we did was take the draft libretto from the Eva character, and translate it into text that could project onto a screen. This was in anticipation of it transforming into the cloud where the test AIBO database that Wilson and Jonathan were working on could be tested to see if it could actually return an answer. It did. The YELLOW is the Eva text, and the BLUE text is the successful reply from the ABIO database.

This was an incredible breakthrough.

The next day Alek and a cameraman came to EKA to interview me for a film they will be working on about the project.

Setting up the camera

Here I am looking very intense for the interview

After more days of editing videos it was finally time to leave Tallinn.

My favorite graffiti art from Tallinn - from Mad Magazine's Spy vs. Spy

The next day I went to Riga, Latvia, where I had a meeting with my performer, Sniedze.

Sniedze Strauta, who will play the man character EVA , in Riga.

Once I returned to NY, my costume designer Dila Demir, who is also at EKA, sent me some sketches and prototypes for a body suit of light. We will make many adjustments, but it is a start.

Sketch 1

Sketch 2

Prototype 1

Things are really underway, and this short time in Tallinn showed a lot of progress

Search tags

VERTIGO is funded under the H2020 European STARTS initiative, innovation at the nexus of Science, Technology, and the ARTS. STARTS promotes the arts as catalysts for efficient conversion of science and technology knowledge into products, services, and processes.