100 minutes of immersive terror

I interviewed Carl Guyenette, the creative intelligence behind The War Of The Worlds: The Immersive Experience, for New Scientist, 23 May 2019.

It’s six years since the Martian invaders succumbed to a microbial infection, leaving us once again in possession of our planet. Carl Guyenette has repaired to The Spirit of Man to raise a glass to Earth’s victory, and to take stock of his new production, a 100 minute-long immersion in Jeff Wayne’s The War of the Worlds.

The pub, which is part of the set and an integral part of the show, comes with its own meticulous backstory. On its walls, animated paintings record famous scenes from humanity’s first interstellar conflict. Remnants of Martian technology loom over the patrons. The effect is amusing for the first few minutes, but the aura of threat is unmistakable: pleasingly, the guts from one of the invaders’ war machines turn out to have been re-engineered to dispense gin.

Wayne’s musical retelling of H G Wells’s sci-fi shocker was released as a double album in 1978, and remains a hit, having sold over 2.5 million copies in the UK alone. There have been spin-offs a-plenty: video games, DVDs, stage shows, live tours. Nothing quite like this, though: “When I’ve been trying to explain this show to people,” says Guyenette, “I say it’s like walking into a cinema, except that once you’re there, you just keep on walking, into the screen. Into the movie itself.”

The full effect of Guyenette’s experiment in “layered reality” can only really be experienced at first hand. Nothing stays still, and neither does the audience, as it moves in groups of a dozen through over 2000 square metres of unlikely theatrical space – two floors of the old Metal Exchange in the City of London.

Visiting this venue mid-development, it had looked like somebody’s open-plan office: bad fluorescent lighting, grey carpet tiles; bins full of sandwich-shop litter; plastic water bottles in cardboard trays; laptops everywhere. Now, as the cast and crew set about unkinking the show’s phenomenally complicated logistics, the space is coming alive, fully dressed in both real and virtual light. Everything trembles. Everything moves, especially the air. Everything has a temperature. Everything has a smell.

Some of the experiences on offer in this show use VR headsets. Others use projection mapping. Some involve puppetry. Almost all manage to work in one of eight different holographic effects. Reality intrudes on the virtual world in unsettling and shocking ways. Things grab you – things you had thought were only in the headset. In VR, meanwhile, figures that seem to be fellow theatre-goers are plucked into the sky by Martian harvesting machines, their eyes meeting those of the participants (thanks to a neat eye-tracking algorithm) as they rise and perish.

Carl Guyenette talks about how he created the show.

New Scientist: What do they call you here?

Carl Guyenette: My job description’s a nuisance. When I called myself the CTO, the technologists on the show insisted I was actually the creative director. Then the creative people told me I’m a technologist. What I actually do is bring things together and makes new things out of them. So I suppose I’m an inventor.

NS: How did you come to work in theatre?

CG: I studied computer science, then joined the film visual effects industry, compositing for big Hollywood films. From there I moved on to making creative technological applications for the British Museum and other venues and festivals. I worked on Viens!, a virtual-reality piece by Michel Reilhac, which then went to Sundance and Cannes. This shot me into the centre of things. And now with the production company dotdotdot I’m trying to bring new media technologies and general audiences together through immersive theatrical experiences like this one. Not that we’ve worked at quite this scale before.

NS: Which of these new media are making the biggest impact on live performance at the moment? 

CG: Projection mapping is really interesting. There are systems now that will project images and textures over objects even as you move them. This is edging us towards VR experiences that won’t require us to wear headsets. And there are domes which you can projection-map from the inside which give you immersive video experiences. There’s a massive one that is going up in Madison Square Garden in 2020 which has a capacity of around 20,000 people: that’s going to be fun!

NS: How did you select the technologies for War of the Worlds?

Stability was essential. Because we’re splitting the audience up into groups of a dozen, and leading them from set to set, and from experience to experience, we’re effectively putting on 70 shows a day. The bottom line is, you want to be using kit that doesn’t break or fall over, so we’re using the HTC Vive Pro. We try out more exotic machinery in our prototyping and experimental work — everything from Hololens to Magic Leap, which I’d dearly like to use in a theatrical setting. But augmented reality systems are still a generation behind VR in terms of stability.

NS: Even with a workhorse VR platform, you’ve been able to mix the real and the virtual in clever ways. Was achieving that mix always an important aspect of the production?

CG: More important for us was to make sure that the technologies worked well with the storytelling. At one point we place our audience in a small boat and set them afloat on a computer-generated sea. The graphics are just one element to the experience. The mechanisms that move the boat, the breeze, the drop in temperature: these elements are just as important. And timing’s the most vital element of all, not just to provide seamless experiences, but also to give the audience breathing space between experiences.

NS: A lot of the technology you’re using is old…

CG: I wanted this show to be an homage to old media: Pepper’s Ghost illusions, and zoopraxiscopes, pyrotechnics and animatronics. It’s a show set over a hundred years ago, after all, at the birth of photography and cinema. In The War of the Worlds, all these technologies feel new.

NS: VR was said to be a medium that would isolate us from each other but you’ve used it to create a social experience. Is this the future of VR?

CG: I think there’s still money to be made from the home VR market. But building something big, in a spacious venue, layering technologies together so you can let audiences do things they couldn’t do anywhere else, means that you can also add a social dimension to the experience. There are not many places where you can be with 12 people in the same room in VR, firing cannon at Martian invaders, fighting off tentacles, befriending and losing people as you struggle through a besieged city.

“Bloody useless at objects. Bloody brilliant at space”

Playing Lunatick by Antony Gormley and Priyamvada Natarajan for New Scientist, 27 March 2019

VISIT The Store X, a venue for art and design in London’s West End, and you are in for quite a journey. Wearing an HTC Vive headset, you are given an island to explore in Lunatick, a glossy, game-like virtual-reality experience that starts at Kiribati in Micronesia. For a while, you have the run of the place by means of hand controllers, although producers Acute Art plans to use EEG to let you control it with your thoughts.

Don’t get too comfortable. Wandering past a stone platform triggers the space elevator. It lifts you gently off your feet, then propels you through the stratosphere. This long, beautiful and increasingly uncanny transit carries you into the void between the moon and Earth.

Take a breath. Look around you. The geometrical relationship between the sun, Earth and its moon unwinds around you as time skews and the moon swells. Before you know it, you are skating around lunar crater rims, plummeting into craters, flying high, until, losing control again, you are flung into the sun.

Lunatick is the first joint work by British artist Antony Gormley and astrophysicist Priyamvada Natarajan from Yale University. Natarajan visualises the accretion history of black holes, and maps the granularity of dark matter by studying the way it bends light – a phenomenon called gravitational lensing. But she couldn’t resist the idea of giving space a sensual dimension by making the vastness and loneliness of the cosmos tangible.

Artist and scientist bonded over their early love of science fiction. H. G. Wells’s 1901 novel The First Men in the Moonwas Natarajan’s contribution: a fictional journey powered by the mysterious gravity-less mineral cavorite. Gormley, in his turn, recalled C. S. Lewis’s space trilogy that began with Out of the Silent Planet, in which a man travels the solar system pinned in a coffin.

Both influences emerge clearly enough in Lunatick, but the real star of the show isn’t fictional: it is the flyable lunar terrain wrestled into shape by Rodrigo Marques, Acute Art’s chief technology officer, from data sent back by NASA’s Lunar Reconnaissance Orbiter. “There’s something both moving and funny about skating over the surface of the moon,” says Gormley. “I’ve got very fond of skiing along the ridges then down into the crater bottoms.”

Gormley’s art is popular globally, not least because people find it easy to grasp. Never mind the cosmological and philosophical dimensions: what strikes the viewer is how he renders, in solid matter, the building blocks of our lives.

Gormley has been making sculptural art out of wireframes and voxels (three-dimensional pixels), even as architects and games designers having been moving away from model-making into a purely virtual 3D design space. “Until recently, I had no idea what a voxel was,” says Gormley, who has spent more than five years making oddly expressive low-resolution sculptures assembled from cubes and cuboids. His wireframe experiments (assembled from real wires and rods) are older still, dating back to the late 1990s.

Why has Gormley chosen to enter the virtual realm now? First, Lunatick was a chance to explore a medium that, he says, is “bloody useless at objects and bloody brilliant at space”. Objects, ultimately, are bodies: VR is hobbled because it can’t convey their mass and tactility. But space is different. We perceive space primarily through seeing, which means VR can convey scale and immensity to a sublime degree.

But why should an artist best known for exploring the sculptural possibilities of the human body (particularly his own) be keen on disembodied space? The image of body-as-spaceship crops up intermittently in Gormley’s work, but rarely so urgently. He says he is haunted by an image of long-haul flight, where the shutters are down and everyone is watching movies: virtual versions of human life.

“I want this piece to say to people, ‘Break out!’,” he says. “Of course we get very obsessed with human matters. But there are bigger affairs out there. Recognise your cosmic identity!”

Teeth and feathers

Visiting T. rex: The ultimate predator at the American Museum of Natural History in New York, for New Scientist, 13 March 2019

PALAEONTOLOGY was never this easy. Reach into a bin and pick up a weightless fossil bone hardly smaller than you are. Fling it into the air, in roughly the direction indicated by the glowing orange light. It fixes in place above you with a satisfying click. Add more bones. You are recreating the head of the most fearsome predator known to natural history: Tyrannosaurus rex. Once it is complete, the skull you have made makes this point nicely – by coming after you.

The American Museum of Natural History is 150 years old this year. One of its collectors, Barnum Brown, discovered the first fossil remains of the predator in Montana in 1902. So the museum has made T. rex the subject of its first exhibition celebrating the big anniversary.

The game I was playing, T. rex: Skeleton crew, is also the museum’s first foray into virtual reality. It is a short, sweet, multiplayer game that, if it doesn’t convey much scientific detail, nonetheless gives the viewer a glimpse of the first great puzzle palaeontologists confront: how to put scattered remains together. It also gives a real sense of the beast’s size: a fully grown T. rex (and they could live into their late 20s) was more than 12 metres long and weighed 15 tonnes.

An extended version of this game, for home use, will feature a full virtual gallery tour. It has been put together by the Vive arm of tech firm HTC. Early on, in its project to establish a name in the cultural sector, the company decided not to compete with the digital realm’s top dog, Google Arts & Culture. Google, at least until recently, has tended to brand its efforts quite heavily because it brings a wealth of big data to its projects.

HTC Vive, by contrast, works behind the scenes with museums, cultural organisations and artists to realise relatively modest projects. By letting the client take the lead, it is learning faster than most what the VR medium can do. It cut its teeth on an explorable 3D rendering of Modigliani’s studio in the Tate Modern in London in 2017 and created an immersive exploration of Claude Monet’s approach to painting in The Water Lily Obsession, now a permanent feature at the Musée de l’Orangerie in Paris.

The trick, it seems, is to focus, to make immersion and physical sensation the point of each piece. Above all, the idea is to slow down. VR isn’t a traditional teaching aid. The Monet project in particular revealed how good VR is at conveying craft knowledge.

But if, instead, a VR installation delivers a brief, memorable, even magical experience, this, too, has value. Skeleton crew is a powerful prompt to the imagination. It isn’t, and isn’t meant to be, the star of this show. The models are the real draw here. Traditionally fashioned life-size renderings of tyrannosaurs big and small, scaled, tufted and sometimes fully feathered, their variety reflecting the explosion of palaeobiological research that has transformed our understanding of millions of years of Mesozoic fauna over the past 20 years. We can now track trace chemicals in the material surrounding a fossil so precisely that we even know the colour of some species’ eggs.

T. rex: The ultimate predator certainly delivers on its brash, child-friendly title. Terrifying facts abound. T. rex‘s jaws had a maximum bite force 10 times that of an alligator – enough not just to break bone, but to burst it into swallowable splinters.

But the really impressive thing about the show are the questions it uses to convey the sheer breadth of palaeontology. What did T. rex sound like? No one knows, but here are a mixing desk and some observations about how animals vocalise: go figure. Is this fossil a juvenile T. rex or a separate species? Here is a summary of the arguments: have a think.

Staged in a huge cavern-like hall, with shadow-puppet predators and prey battling for dear life, T. rex: The ultimate predator will wow families. Thankfully, it is also a show that credits their intelligence.

Plastered

Stereo recordings, Imax movies, Steam-generated games: for over a century we’ve relied on innovations in media to deliver us a taste of the sublime. As the pioneering pilot Antoine de Saint-Exupéry put it, “The machine does not isolate man from the great problems of nature but plunges him more deeply into them.”

But even if that was true in 1939, I’m not sure it’s true any longer — not now we live enmired in media, and, indeed, have to make special efforts to escape them.

For me, 2017 ended with a couple of little disappointments both involving virtual reality.
Having wandered disembodied around Amedeo Modigliani’s “Ochre Atelier“ (an international curatorial effort which took five months to construct) I left London’s Tate Modern feeling as though I’d been transported, not to Paris, but to some Pixar-driven studio circa 1995. Virtual environments can actually render grit, dirt and water stains to a high creepiness level — but not on the sort of budget the Tate can command. The Ochre Atelier was a cartoon.

Not long after, I was prevented from wandering around Yinka Shonibare’s reconfigured, virtual version of the “Townley Venus” at the Royal Academy. The invigilator had saved me from walking into a wall, but I stalked out of that overclean, over-smooth virtual interior convinced that current museum technology can only insulate us from the sublime and the beautiful.
The technologies of digital reproduction are doing to objects exactly what they have done for books and music. They’re are making everything simultaneously accessible and boring.

A misplaced compulsion to entertain is only part of this story, and not the most interesting part. For cultural institutions, the chief promise of digital is its ability to reproduce and preserve. The effort to copy and disseminate objects of antiquity is both venerable and important. Indeed Mari Lending’s new book, Plaster Monuments: Architecture and the Power of Reproduction, reveals the role of casting in preserving entire architectural epochs. She recounts how in 1857 and on a visit to the Paris International Exhibition, the V&A’s first director Henry Cole persuaded 15 European princes to sign up to an ‘International Convention of promoting universally Reproductions of Works of Art’.

In Cole’s day, the technologies of reproduction consisted of photography, electrotyping and plastercasting. Casting, alas, proved far more damaging than was at first realised. Giovanni Battista Belzoni, the former circus strongman who conveyed the 7-ton bust of Pharaoh Ramesses II out of Luxor (it’s now in the British Museum) was no vandal, but he still managed to to take all the paint off when he cast the figure of Anubis in the tomb of Seti I, in the Valley of the Kings.

Worse followed: Belzoni’s facsimile of Seti’s tomb, shown at the Egyptian Halls in London in 1821, was one of several events that led Thomas Cook to form his travel agency. This in turn triggered arguably history’s greatest cultural catastrophe, since tourism is predicated on the idea that original objects, buildings and places can be made more or less universally accessible. Suddenly the traces of past ages were fixed in place, and people travelled to them. The damage Belzoni did to the tomb of Seti I pales in comparison to the wholesale dilapidation visitors have visited upon the entire complex at Luxor.

Many heritage sites are now permanently closed. Those wanting to explore Seti’s 3,300-year-old tomb will have to make do with a trip to Basel and a 100-micron-resolution facsimile of two chambers, courtesy of the Theban Necropolis Preservation Initiative. This project, run by Factum Foundation in collaboration with the University of Basel and under contract to the Egyptian Ministry of Antiquities, is embarked on nothing less than the wholesale recreation of the treasures of the Valley of the Kings.

Multi-spectral composite photography; 3D scanning; additive 3D printing; high-resolution CNC routing: the technologies employed and built from scratch by the technicians and artisans of “digital mediators” Factum Arte allow us to copy originals without ever touching them. But there is something kitschy about Seti’s immaculate new e-tomb: what it has gained in freshness, it has lost in dignity.

Similarly when I visited the RA, though I enjoyed the way Shonibare had repurposed the Roman Venus statue, I found its VR setting (based on a Gavin Hamilton painting) profoundly dispiriting. he hadn’t just brought an old painting to life. He’d done something far worse. He’d cleaned it up. He’d made it good as new.

The eighteenth-century culture in which Belzoni operated – doing an honest but far from perfect job of reproducing the treasures of his day — can, I think, point us in a healthier direction.

Belzoni’s was a world where no one worried much when a nose or a leg fell off an old statue. They simply patched them up. This honourable practice goes back at least as far as the 16th-century sculptor Benvenuto Cellini, who argued that the ravaged sculptures of antiquity ”were screaming for his help”.

The poster-child of that era (at least for Londoners, who can visit the museum he made of his house in Lincoln’s Inn Fields) is the architect Sir John Soane. Soane, born 1753, once grubbed up the garden of Pitzhanger, his country manor, to plant imaginary ruins. This was not (or not entirely) a Romantic indulgence. He used them mostly to tease antiquarians. He would take shards and fashion them into whole items, not willy-nilly, but on the basis of diligent study and educated taste. He knew the past was gone, and approximations were necessary. Lacking today’s precision instruments and the over-weaning, millennial rhetoric that accompanies them, he never once imagined that the past could be entirely recouped, or that the present moment might be made to last forever. One design for his Bank of England (a masterpiece now largely destroyed) showed how it would look in ruins.

Soane was rather fond of imagining what posterity might make of his museum. In 1812, and in the voice of an archaeologist of the future, he wrote:

“It is difficult to determine for what purposes such a strange and mixed assemblage of ancient works or rather [plaster] copies of them, for many are not of stone or marble, have been brought together…”

Not every young man could afford a Grand Tour of Europe. Soane considered his collection of copies the next best thing. He was right.

It was the stubborn whiteness of plastercasts, perhaps, the lack of an accurate means of coloration, not to mention a growing awareness of the damage being done by even the most careful copying — that saved Soane and his generation from imagining that they could render objects as they had been at their moment of completion.

Today we are armed with technology that can copy the surface appearance and even some of the structural elements of originals to perfection, or as close to perfection as makes no odds. This inadvertently puts us in the position of Lord Duveen, the British art dealer who, believing that authenticity was but a bathtime away, saw to the scrubbing of the Elgin Marbles. Nor was he the first restorer to kill through kindness. Writers Edmond and Jules de Goncourt were already complaining about this sort of thing in 1851, after seeing a newly cleaned Rubens: “It is,” they wrote, “like a piece of music from which all the half-tones have been removed: everything screams and bellows like earthenware gone mad.”

Nothing material will be damaged by today’s digital scrubbing — indeed, much valuable information will be uncovered and saved. But if we think that returning objects to their pristine state brings us closer to an authentic view of the past, then we are seriously short-changing our imaginations.

Soane and his fellows knew what we affect to deny: that time is real, and that everything ages and crumbles and dies. That’s why they considered art even more important than artworks.

As he reflected on the potential of scanning and 3D printing, Tristram Hunt, the current director of the V&A, pointed out in a recent article that “the lost artefacts of the ancient past have never felt more tangible, and less controllable.” Factum Arte founder Adam Lowe goes further. In the future, he says, we will handle each object “like an interpretation of a musical score. There will be different performances of the same object. They can be compared, discussed and understood.”

I would like to think the future will be full of John Soanes as, with a clear conscience, we start patching together bits of the past into new objects: not at random, not out of mischief, but as Soane and his contemporaries did — out of an imaginative engagement with a past we accept is irretrievably lost.

Technology vs observation

Losing my rag at the Royal Academy for New Scientist, 13 December 2017

When the schools of London’s Royal Academy of Arts were opened in 1769, life drawing — the business of sketching either live models or the plaster casts of worthy sculptures — was an essential component of an artist’s training.

As I wandered around From Life, an exhibition devoted to the history as well as the future of the practice, I overheard a curator explaining that, now life drawing is no longer obligatory in Royal Academy art courses, a new generation of artists are approaching the practice in a “more expressive” way. The show’s press release claims even more: that life drawing is evolving “as technology opens up new ways of creating and visualising artwork”.

There was little of this in evidence when I visited, however: two-and-a-half of the three virtual-reality experiences on offer had broken down. Things break down when the press turns up – you might even say it’s a rule. Still, given their ubiquity, I’m beginning to wonder whether gallery-based VR malfunctions are not a kind of mischievous artwork in their own right. In place of a virtual sketch, a message in an over-friendly font asks: “Have you checked your internet connection?” At least Swiss artist Jean Tinguely’s wild mobiles of the 1960s had the decency to catch fire.

How can new technologies like Google’s Tilt Brush and HTC’s Vive VR platform bring artists into a more intimate relation to their subject — more intimate than might be achieved by, say, standing a metre away from a naked stranger armed only with a bit of charcoal?

Jonathan Yeo has had a stab at the problem, using Tilt Brush’s 3D painting tech to fashion a sculptural self-portrait. The outsize bronze 3D print of his effort — an assemblage of short, wide, hesitant virtual “brushstrokes” — has a curiously dated feel and wouldn’t look out of place in a group retrospective of 20th-century British sculpture. As an advert for a technology that prides itself on its expressivity (videos of the platform at work usually resemble explosions in a paint factory), it’s a curiously laborious piece.

On a nearby wall hang Gillian Wearing’s photographic self-portraits, manipulated using the sort of age-progression technology employed by forensic artists. In this way, Wearing has captured her appearance 10, 20, 30 years into her future. It’s an undeniably moving display, and undeniably off the point: life drawing is about capturing the present moment, which leaves Wearing’s contribution resembling those terms and conditions that appear at the bottom of TV advertisements – Other Moments Are Available.

Yinka Shonibare (best known for his ship-in-a-bottle sculpture on the fourth plinth of Trafalgar Square) comments on the show, rather than contributes to it, with a 3D VR conceptualisation of a painting by the 18th-century Scots artist and dealer Gavin Hamilton.

Hamilton once sold a Roman sculpture to a collector. Shonibare has scanned a plaster cast of this Townley Venus, then placed it on a plinth in a largely imaginary VR garden (you catch only a glimpse of this space in Hamilton’s painting). He has covered its plaster-white surface with batik designs (referring to common sub-Saharan African fabric, though it was originally a Dutch export) and as a coup de grâce, he has stuck a globe on Venus’s torso in place of her head. The point is that we can never copy something without to some degree appropriating it. Whether you like what he’s done will depend on whether you like art that makes a primarily intellectual point.

In a gallery environment increasingly besotted by (and bested by) technology, such acts of cultural orienteering may be necessary; they’re certainly inevitable. The new work gracing From Life at least attempts to address the theme of the show, and its several failures are honest and interesting.

Still, I keep coming back to the historical half of the exhibition — to the casts, the drawings, the portraits of struggling young artists from 1769 to now. Life drawing is not obligatory for artists? It should be obligatory for everyone. If we never learn to observe honestly, what the devil will we ever have to be expressive about?