Interstitial Stitches

 
 

 

I build work that bridges gaps between the study of human experience in the neurosciences and the creation of human experience in the arts. This is where subjectivity and objectivity meet, in the struggle to experience new understandings of experience. Art that engenders and organizes self-reflection is experimental phenomenology--thus brain science. In turn, science that lays bare questions which have heretofore been hidden by reductionist answers, is art. My work mixes both to create spaces where boundaries blur between science and art, reverence and irreverence, the many vocabularies within and without the human. Art/science pieces that question the futures we are building as scientists and engineers, that perform inquiry that may not yield to quantification and objective investigation, and that repurpose technology for probing outside the bounds and demands of utility. You’ll see below pieces that let us control the unconscious in dreams, step inside neuroscience in VR, program aesthetic experience into the brain via the body, and question objectivity by experiencing emotions as a wholly quantified self.

Apart from building projects to bridge ideas, I need to bridge peoples and silos to make real progress at the art/science intersection. I am proud that my projects are not done alone, that they involve translation and cooperation. My time as a Masters Student at MIT has thus involved building community, serving as Creative Lead of MIT Hacking Arts, lead Curator for the Algorithmic Justice League at the Boston MFA, Creative in Residence at MIT's OpenMind:OpenArt Gallery, co-founder of the HotMilks Foundation artist's collective linking 6 labs across campus, and founding member of the Generism collective for art and neuroscience . My artwork has been shown at Cannes Film Festival, Ars Electronica, the Boston Museum of Fine Arts, Transmediale, SXSW, SF MOMA's Open Space, World Economic Forum, the National Academy of Sciences, the MIT Compton Arts Gallery, Harvard's metaLAB, The Museum of the Modern Renaissance, Arts@29 Garden, Rainbow Unicorn Gallery, Berlin's ACUD and on 60 Minutes.

 

Dormio

This project, Dormio, exists across sleep neuroscience, performance art and speculative design: we track sleep stages and look for key moments on the precipice of unconsciousness where we can control dream content. Using this system, we can put people in touch with their most abstracted, fluid versions of themselves, the state of mind where prefrontal cortex function diminishes and fluid thinking takes over, where sense of self and time fade into semi-lucidity. We can influence dreams reliably, and since dreams turn concrete daily input into abstract divergent thought, we can turn memories into metaphor, turn logic into poetry, and put people in touch with their most creative selves.

We create performances in which dreams are audibly shared between sleeping and waking persons in real time, as with the performance below. We write papers showing inception of early stage dreams, and augmentation of human creativity by extracting dream content. We create videos, shown below, which draw out a science fiction future of where dream control could take us. This project allows a linking of worlds, a bridge across interior and exterior, art and science, observer and observed, conscious and unconscious. Importantly, by allowing for control of dream content, it potentiates controlled experiments on dreams—bringing the strange world of our nighttime fantasies back into the scope of rigorous scientific investigation.

giphy.gif
 
Screenshot+2019-03-22+16.02.13.jpg

Sleep Science Performed

This project, as it links the deeply personal and subjective world of dreams with the objective study of memory and emotion which dreams compose, has been shown in many venues which genuinely link artistic and scientific vocabularies—including the National Academy of Sciences, Ars Electronica, the MIT Museum Studio and more. But even more interesting, it has existed simultaneously as an art work and an object of scientific study, with the same technological interface used to create performance art, video work and novel sleep studies (the device will be used by 5 sleep labs in the coming year, as well as another performance piece).

Engineering Emotion

Where an artwork is most meaningful, a song most emotional, or a speech most resonant we all share a nearly universal physiological reaction: frisson, shivers down the spine, goosebumps. At once transcendent and physiological, magic meets mechanism as the sublime cascades across skin. I have built a system to induce frisson, or aesthetic chills, automatically using a wearable device inspired by work in embodied cognition. Now we're testing whether inducing chills during viewing can increase deep attention and openness to art experiences—psychophysiology driving thought from the spine upwards to the mind. This work is a collaboration with Abhinandan Jain and Félix Schoeller, and currently we’re beginning projects testing it in real world museum, movie and theater settings to create new art experiences and empathy from the bottom up.

IMG_3301-2.jpeg
 

The Blank Canvas

A bridge between the worlds of neuroscience and VR animation to tell an immersive, accurate scientific story about content ranging from virotherapy to optogenetics in the brain. The Blank Canvas is a platform for VR science storytelling, for users to step inside science and feel it. Biology is inspiring, but often happens on scales so small they are incomprehensible on paper--which is where VR comes in. Our mission was to make science media as cool and illuminating as the science behind it... learn more here!

Shown at the World Economic Forum, Cannes Film Festival 2017, UniteSXSW, premiered at VR Sci Fest. Powered by Unity, Houdini and two other storyteller engineers Boo Aguilar and Peehalho Magalhaes.

Animate Objects

A piece in the spirit of the hugely influential neuroscience video piece Warring Triangles and the Herbert-Simmel Illusion it illustrated, wherein viewers automatically assign agency to objects even without faces or dialogue, with the chaotic light aesthetic of the Moholy-Nagy Light Space Modulator. I worked with Adam Burke, Owen Trueblood and Ben Miller to make a contemporary shadow projection piece. Watch it and feel your internal urge to create narrative where there is none. The use of robotic mechanism to create a subtle story was inspired by a trip to and short residency at Arthur Ganson’s studio, an opportunity we had through the Generism Collective.

Working with Ganson at Halo Hill

Working with Ganson at Halo Hill

Experiments in projection shadow pieces

Experiments in projection shadow pieces

 

Quantifying Care

A thread throughout my work is an uneasiness about the quantified self. Measurement illuminates and also colors our worldview. Inspired by an urge to make clear to viewers that the way in which new tools allow us to see the unseen is not neutral, I did a series of gestures of care (patting a belly, stroking a cheek) shown in FPS 10000. Each comes to light in super slow motion as simultaneously soft and surprisingly violent, subtle and shocking. A tool lets us see the invisible, and this is never neutral.

Filling What Was Empty

Cognitive science has learned so much about our normal functioning by investigating aberrant brains from the 3rd person perspective. Art has taught us so much by allowing us 1st person glimpses into worlds and sights distinct from our own. Filling What Was Empty, done with the unique brain of cyborg Neil Harbisson, bridges both kinds of work. Neil's antenna, which gives him color vision by subtly vibrating his skull, is an illustration of the constant sense-making and new pattern incorporation that the brain excels at—ordered shaking becomes sight. And this same pattern making urge exists in all brains, such that in the absence of ordered stimuli, we assign and experience order. The brain, in the absence of stimuli, attempts sense-making of nothingness, so we hallucinate patterns and can see our mechanisms of sense-making in action--allowing people to perceive their own mechanisms of perception. I wanted to give an experiential, artistic sense of this experience.

We create a uniform empty light field, called the Ganzfeld, and people see patterns emerge in the nothingness.  A darkness darker than dark. An emptiness filled by the brain. Neil experience the Ganzfeld effect as well, to the greatest extent in red light just like subjects with color vision, showing a novel cross-sensory color-sensitive experience of the Ganzfeld. I worked with Oscar Rosello to turn the Ganzfeld into a performance piece at Media Lab, taking 30 people through the experience, and a video piece for Berlin's Transmediale Festivale, Shown at Harvard metaLAB, Berlin's ACUD and Rainbow Unicorn Gallery. See show here.

 

Digital Divining Rod

This project is inspired by the practice of 'Divining', or 'Dowsing', in which practitioners watch their own subtle hand movements as they walk to locate deep subterranean water sources. Psychological investigation of dowsers suggests these 'water witches' already know the locations, but unconsciously in their bodies, not consciously in their brains. The practice of revealing micro-gestures in the hand to investigate water amounts to a form of embodied self-examination.  I took inspiration from this practice to create a kinetic sculpture responding to Seymour Papert's famous dictum, "you can't think about thinking without thinking about thinking of something". In it, participants reveal their subtle hand movements as they look up at a tree, creating unique alterations in portraiture even as they claim to hold their hands still. Are these hand movements their embodied understanding of the concept tree, revealed in subtle hand movements? Or simply an artistic fingerprint?

In Digital Divining Rod, science and art intersect allowing new, material inquiries into our subconscious cognition. How can we think about things we know but do not know we know? I create an interactive sculpture for thinking about thinking without thinking about thinking. Here people can encounter their subconscious through physical, digital, and environmental sleight of hand. They can discover their unconscious, embodied knowledge. An artistic agitation of a scientific challenge.

Purposeful Play

Bridging multiple programs and 7 different MIT labs, I founded an artist's collective building installation art and performance art around Cambridge and Boston: The HotMilks Foundation. Since our inception 1 year ago, we have been commissioned 6 separate times by the Boston Museum of Fine Arts, building pieces linking science and the arts, as well as building uncommissioned pop-up performances around MIT. Projects Affective Spa, Partition and Cat Café detailed below:

The Affective Spa

A performance piece, The Affective Spa, in which neuroscience and art converge to transform the art experience. We skip the unpredictability of the aesthetics and instead induce the art experience directly with emotional induction techniques from brain science. We are, after all, quantified selves, measured and measuring. Affect and artistic aura are not immune to mechanical reproduction. And yet the many techniques used to create certain emotions in neuroscience laboratories, to study fear, joy, nostalgia and more, have never left the laboratory to engender emotion in an art context. Here we create and perform a future in which vibratory, auditory, light-based, tactile and embodiment-based neuroscientific induction techniques make us feel the way art feels without any need for the art object itself. Read more here. A Menu and Manifesto for The Affective Spa is shown below. Piece commissioned by the Boston MFA, and co-created with the Hotmilks Foundation. 

Matthew and Milks.JPG
 

Partition

The body knows so much. Inaudible infrasound vibratory communication <20 Hz is a method of communication throughout the animal kingdom, and infrasound vibration is known to have pronounced emotional effects on humans. What can vibrations shake loose within us? This piece folds this nascent science question into a performance, asking a question of relation and empathy, to audience and one another. How can we make each other feel, go beyond language to pure embodied emotion, how can we communicate entirely honestly? 

Sit on the bench and be shaken by a friend's heartbeat. By laughter, and sobbingSit across from a lover, and whisper secrets into the microphone. Each will be translated into vibrations in real time. Your partner will feel whatever you cannot bring yourself to say to them, what they cannot hear but nevertheless can understand. Make them feel. Many thanks the Hotmilks Foundation and Council for the Arts at MIT for making it happen, and the Boston MFA for commissioning and hosting us. More here and here.

Partition: Vinyl, benches @ 90", Buttkicker transducer sonic shaker, MaxMSP, microphone, participants, secrets

Poster Hotmilks Screenshot.png
 

Uncanny Cat Café

An exploration of the uncanny notion that memories in the brain are entirely malleable, that history is more workable material than static foundation. A cat café populated by robotic cats and waitresses from the future, where childhood toys (veridical memories) are cut into parts by attendees and reassembled into chimeric, animatronic futures (remembered memories). A menu full of options like switches, pacer motors and programmable voice boxes to animate these forgotten toys. Make your own memory monster, just as we daily unconsciously make our own memories. See here.

Attendees made their own futures, rewriting old toys' histories together. As the MFA openly tries to transition from a place of preservation, of memory, of static frames, into a space for community and dialogue and futuring, so must the objects within transform from memories into material again. Many thanks to the Hotmilks Foundation, the MFA and CAMIT for making this project possible.

This piece was re-commissioned, with alterations, to coincide with the the Boston MFA's opening of Takashi Murakami's work, shown below. Hotmilks Foundation again are co-creators here. 

Cat Café
Hotmilks Murakami.png
 

MIT Hacking Arts Signature Hack 1

Fictional Animals Operated By Thousands of People Online

A series of engineered animals, one a robotic swarm and another a single organism, controlled by the invisible workers behind Amazon Mechanical Turk and Twitter sentiment analysis, respectively. Animating ambiguous, emergent life forms from the immaterial. A kinetic sculpture investigating the questions raised by semi-conscious AI, by humans hidden behind an internet facade, by technology that is increasingly integrated with our bodies and minds. What is conscious, what is collective, what is organic, and what is purely technological?

For the Signature Hack of MIT Hacking Arts 2018 I assembled a dream team of friends from MIT to work with the amazing artist Agnieszka Kurant and execute this project overnight. See it all here! Read more about our team and work here! Shown at Nicolas Bourriad's La Panacée, SFMOMA's online collection, and at Harvard metaLAB's AI + Art Group. Living on in the ether evermore. Many thanks to CAMIT for their support. 

 

Hacking Arts Signature Hack 2

I assembled a team from across MIT to work with Professor Ian Condry on a piece which used 360 audio and distributed cellphone speakers to create a performance piece representing the democratic characteristics of sound. The multi-channel audio installation sampled and abstracted audio excerpts from recent presidential inaugural addresses, then blended them with breathing sounds that the team recorded from a live audience. Building on this soundtrack, two team members acted as event DJs, instructing the audience to hum and breathe in unison, while their phones — controlled by an app created for the hackathon — played additional breathing and humming sounds. The work was covered in MIT News here!