When robots start caring

The question is not whether we should employ robots. Given the lousiness of some institutions, why on earth wouldn’t we? The question is whether the robots we employ will be any good, and whether we can accept them as substitute humans. We’d like to think not, but there’s evidence to suggest that we’ll bond with even a basic machine far more easily than we’d like to believe.

A glimpse into our Eloi future for the Spectator, 3 February 2018

There’s also a podcast

 

Plastered

Stereo recordings, Imax movies, Steam-generated games: for over a century we’ve relied on innovations in media to deliver us a taste of the sublime. As the pioneering pilot Antoine de Saint-Exupéry put it, “The machine does not isolate man from the great problems of nature but plunges him more deeply into them.”

But even if that was true in 1939, I’m not sure it’s true any longer — not now we live enmired in media, and, indeed, have to make special efforts to escape them.

For me, 2017 ended with a couple of little disappointments both involving virtual reality.
Having wandered disembodied around Amedeo Modigliani’s “Ochre Atelier“ (an international curatorial effort which took five months to construct) I left London’s Tate Modern feeling as though I’d been transported, not to Paris, but to some Pixar-driven studio circa 1995. Virtual environments can actually render grit, dirt and water stains to a high creepiness level — but not on the sort of budget the Tate can command. The Ochre Atelier was a cartoon.

Not long after, I was prevented from wandering around Yinka Shonibare’s reconfigured, virtual version of the “Townley Venus” at the Royal Academy. The invigilator had saved me from walking into a wall, but I stalked out of that overclean, over-smooth virtual interior convinced that current museum technology can only insulate us from the sublime and the beautiful.
The technologies of digital reproduction are doing to objects exactly what they have done for books and music. They’re are making everything simultaneously accessible and boring.

A misplaced compulsion to entertain is only part of this story, and not the most interesting part. For cultural institutions, the chief promise of digital is its ability to reproduce and preserve. The effort to copy and disseminate objects of antiquity is both venerable and important. Indeed Mari Lending’s new book, Plaster Monuments: Architecture and the Power of Reproduction, reveals the role of casting in preserving entire architectural epochs. She recounts how in 1857 and on a visit to the Paris International Exhibition, the V&A’s first director Henry Cole persuaded 15 European princes to sign up to an ‘International Convention of promoting universally Reproductions of Works of Art’.

In Cole’s day, the technologies of reproduction consisted of photography, electrotyping and plastercasting. Casting, alas, proved far more damaging than was at first realised. Giovanni Battista Belzoni, the former circus strongman who conveyed the 7-ton bust of Pharaoh Ramesses II out of Luxor (it’s now in the British Museum) was no vandal, but he still managed to to take all the paint off when he cast the figure of Anubis in the tomb of Seti I, in the Valley of the Kings.

Worse followed: Belzoni’s facsimile of Seti’s tomb, shown at the Egyptian Halls in London in 1821, was one of several events that led Thomas Cook to form his travel agency. This in turn triggered arguably history’s greatest cultural catastrophe, since tourism is predicated on the idea that original objects, buildings and places can be made more or less universally accessible. Suddenly the traces of past ages were fixed in place, and people travelled to them. The damage Belzoni did to the tomb of Seti I pales in comparison to the wholesale dilapidation visitors have visited upon the entire complex at Luxor.

Many heritage sites are now permanently closed. Those wanting to explore Seti’s 3,300-year-old tomb will have to make do with a trip to Basel and a 100-micron-resolution facsimile of two chambers, courtesy of the Theban Necropolis Preservation Initiative. This project, run by Factum Foundation in collaboration with the University of Basel and under contract to the Egyptian Ministry of Antiquities, is embarked on nothing less than the wholesale recreation of the treasures of the Valley of the Kings.

Multi-spectral composite photography; 3D scanning; additive 3D printing; high-resolution CNC routing: the technologies employed and built from scratch by the technicians and artisans of “digital mediators” Factum Arte allow us to copy originals without ever touching them. But there is something kitschy about Seti’s immaculate new e-tomb: what it has gained in freshness, it has lost in dignity.

Similarly when I visited the RA, though I enjoyed the way Shonibare had repurposed the Roman Venus statue, I found its VR setting (based on a Gavin Hamilton painting) profoundly dispiriting. he hadn’t just brought an old painting to life. He’d done something far worse. He’d cleaned it up. He’d made it good as new.

The eighteenth-century culture in which Belzoni operated – doing an honest but far from perfect job of reproducing the treasures of his day — can, I think, point us in a healthier direction.

Belzoni’s was a world where no one worried much when a nose or a leg fell off an old statue. They simply patched them up. This honourable practice goes back at least as far as the 16th-century sculptor Benvenuto Cellini, who argued that the ravaged sculptures of antiquity ”were screaming for his help”.

The poster-child of that era (at least for Londoners, who can visit the museum he made of his house in Lincoln’s Inn Fields) is the architect Sir John Soane. Soane, born 1753, once grubbed up the garden of Pitzhanger, his country manor, to plant imaginary ruins. This was not (or not entirely) a Romantic indulgence. He used them mostly to tease antiquarians. He would take shards and fashion them into whole items, not willy-nilly, but on the basis of diligent study and educated taste. He knew the past was gone, and approximations were necessary. Lacking today’s precision instruments and the over-weaning, millennial rhetoric that accompanies them, he never once imagined that the past could be entirely recouped, or that the present moment might be made to last forever. One design for his Bank of England (a masterpiece now largely destroyed) showed how it would look in ruins.

Soane was rather fond of imagining what posterity might make of his museum. In 1812, and in the voice of an archaeologist of the future, he wrote:

“It is difficult to determine for what purposes such a strange and mixed assemblage of ancient works or rather [plaster] copies of them, for many are not of stone or marble, have been brought together…”

Not every young man could afford a Grand Tour of Europe. Soane considered his collection of copies the next best thing. He was right.

It was the stubborn whiteness of plastercasts, perhaps, the lack of an accurate means of coloration, not to mention a growing awareness of the damage being done by even the most careful copying — that saved Soane and his generation from imagining that they could render objects as they had been at their moment of completion.

Today we are armed with technology that can copy the surface appearance and even some of the structural elements of originals to perfection, or as close to perfection as makes no odds. This inadvertently puts us in the position of Lord Duveen, the British art dealer who, believing that authenticity was but a bathtime away, saw to the scrubbing of the Elgin Marbles. Nor was he the first restorer to kill through kindness. Writers Edmond and Jules de Goncourt were already complaining about this sort of thing in 1851, after seeing a newly cleaned Rubens: “It is,” they wrote, “like a piece of music from which all the half-tones have been removed: everything screams and bellows like earthenware gone mad.”

Nothing material will be damaged by today’s digital scrubbing — indeed, much valuable information will be uncovered and saved. But if we think that returning objects to their pristine state brings us closer to an authentic view of the past, then we are seriously short-changing our imaginations.

Soane and his fellows knew what we affect to deny: that time is real, and that everything ages and crumbles and dies. That’s why they considered art even more important than artworks.

As he reflected on the potential of scanning and 3D printing, Tristram Hunt, the current director of the V&A, pointed out in a recent article that “the lost artefacts of the ancient past have never felt more tangible, and less controllable.” Factum Arte founder Adam Lowe goes further. In the future, he says, we will handle each object “like an interpretation of a musical score. There will be different performances of the same object. They can be compared, discussed and understood.”

I would like to think the future will be full of John Soanes as, with a clear conscience, we start patching together bits of the past into new objects: not at random, not out of mischief, but as Soane and his contemporaries did — out of an imaginative engagement with a past we accept is irretrievably lost.

Cosmoscope

We’ve learned much in the half-millennium since Leonardo declared Man “the measure of all things.”, and seen the human species relegated to a footnote in the cosmological story.
Now we’re beginning to see that humanity maybe does sit at the heart of the universe. At no other scale but ours does the universe attain such complexity.

Exhibited at the London Lumiere festival in January 2018, Simeon Nelson’s 3.3 metre-high singing, flashing sculpture, is an enormous puzzle in structural engineering, sound and software design. It’s also a homage to cosmological models of the past, especially Leonardo’s “Vitruvian Man”, drawn around 1490.

I interviewed the makers for this video by David Stock.

What’s the Russian for Eastbourne?

This is a journey through an exotic world conjured into being by the Military Topographic Directorate of the General Staff of the Soviet Army. Tasked by Stalin during the Second World War with accurately and secretly mapping the Soviet Union, its Eastern European allies, its Western adversaries, and the rest of the world, the Directorate embarked on the largest mapping effort in history.

Reading Davies and Kent’s Red Atlas for the Telegraph, 13 January 2018

A glimpse at time

Time is difficult to talk about – the show’s cumbersome title, Life Time: Biological clocks of the universe, is proof enough of that. Even the gallery’s lucid handout by William Myers, a curator based in Amsterdam, labours under the title “A Non-Circadian Cadence”. But the show itself does much better, embracing a wide swathe of temporal landscape, from the universal to the personal and from the cellular to the geological.

Visiting MU Artspace, Eindhoven for New Scientist, 20 January 2018

Music and art stop dementia from stealing everything we cherish

According to Greek legend, 49 of King Danaus’s 50  daughters were mariticidal, and condemned to fill a leaky bath in hell, and their lot is an apt metaphor for the human condition. However much we fill our lives, our lives still dribble away. We experience, we learn – but we also forget. 

A choir and a De Kooning show inspired this piece for New Scientist, 4 January 2018

Just experience it

According to the curators, these exquisite allegorical frescoes by 18th-century artist Johann Wenzel Bergl are “recognizable as strategies of absolutist picture propaganda”. And Mark Dion’s installation capturing “the lifestyle and self-image of the prototypical ethnographer of colonial times”, isn’t even that, but alludes “to our own imagination of that ethnographer”. I left feeling rather as Lewis Carroll’s Alice might have felt if, instead of freely stepping through the mirror, she had been shoved through it from behind by a gang of goonish anthropologists.

Visiting mumok, Vienna’s museum of contemporary art, for New Scientist, 23 December 2017

Technology vs observation

Things break down when the press turns up – you might even say it’s a rule. Still, given their ubiquity, I’m beginning to wonder whether gallery-based VR malfunctions are not a kind of mischievous artwork in their own right. In place of a virtual sketch, a message in an over-friendly font asks: “Have you checked your internet connection?” At least Swiss artist Jean Tinguely’s wild mobiles of the 1960s had the decency to catch fire.

Losing my rag at the Royal Academy for New Scientist, 13 December 2017

It’s coming at you!

OUTSIDE Dimension Studios in Wimbledon, south London, is one of those tiny wood-framed snack bars that served commercial travellers in the days before motorways. The hut is guarded by old shop dummies dressed in fishnet tights and pirate hats. If the UK made its own dilapidated version of Westworld, the cyborg rebellion would surely begin here.

Steve Jelley orders us breakfast. Years ago he left film production to pursue a career developing new media. He’s of the generation for whom the next big thing is always just around the corner. Most of them perished in the dot-com bust of 2001, but Jelley clung to the dream, and now Microsoft has come calling.

His company, Hammerhead, makes 360-degree videos for commercial clients. Its partner in this current venture, Timeslice Films, is best known for volumetric capture of still images – the business of cinematographically recording forms in three dimensions – a practice that goes back to founder Tim MacMillan’s art-school experiments of the early 1980s.

Steve Sullivan, director of the Holographic Video initiative at Microsoft, is fusing both companies’ technical expertise to create volumetric video: immersive entertainment that’s indistinguishable from reality.

There are only three studios in the world that can do this with any degree of conviction, and Wimbledon is the only one outside the US. Still, I’m sceptical. It has been clear for a while that truly immersive media won’t spring from a single “light-bulb” moment. The technologies involved are, in conceptual terms, surprisingly old. Volumetric capture is a good example.

MacMillan is considered the godfather of this tech, having invented the “bullet time” effect central to The Matrix. But The Matrix is 18 years old, and besides, MacMillan reckons that pioneer photographer Eadweard Muybridge got to the idea years before him – in fact, decades before cinema was invented.

“Engineer Masahiro Mori says his ‘uncanny valley’ idea was never meant to be taken scientifically”

Then there’s motion capture (or mocap): recording the movement of points attached to an actor, and from those points, constructing the performance of a three-dimensional model. The pioneering Soviet physiologist Nikolai Bernstein invented the technique in the early 1920s, while developing training programmes for factory workers.

Truly immersive media will be achieved not through magic bullets, but through thugging – the application of ever more computer power, and the ever-faster processing of more and more data points. Impressive, but where’s the breakthrough?

“Well,” Jelley begins, handing me what may be the largest bacon sandwich in London, “you know this business of the ‘uncanny valley’…?” My heart sinks slightly.

Most New Scientist readers will be familiar with Masahiro Mori’s concept of the uncanny valley. It’s a curiously anglophone obsession. In the 30 years since the Japanese engineer published his paper in 1970, it has been referred to in Japanese academic literature only once. Mori himself says the idea was never meant to be taken scientifically. He was merely warning robot designers at a time when humanoid robots didn’t exist that the closer their works came to resemble people, the creepier we would find them.

In the West, discussions of the uncanny valley have grown to a sizeable cottage industry. There have been expensive studies done with PET scans to prove the existence of the effect. But as Mori commented in an interview in 2012: “I think that the brainwaves act that way because we feel eerie. It still doesn’t explain why we feel eerie to begin with.”

Our discomfort extends beyond encounters with physical robots to include some cinematic experiences. Many are the animated movies that have employed mocap to achieve something like cinematic realism, only to plummet without trace into the valley.

Elsewhere, actor Andy Serkis famously uses mocap to transform himself into characters like Gollum in The Lord of the Rings, or the chimpanzee Caesar in Rise of the Planet of the Apes, and we are carried along well enough by these films. The one creature this technology can’t emulate, however, is Serkis himself. Though mocap now renders human body movement with impressive realism, the human face remains a machine far too complex to be seamlessly emulated even by the best system.

Jelley reckons he and his partners have “solved the problem” of the uncanny valley. He leads me into the studio. There’s a small, circular, curtained-off area – a sort of human-scale birdcage. Rings of lights and cameras are mounted on scaffolds and hang from a moveable and very heavy-looking ceiling rig.

There are 106 cameras: half of them recording in the infrared spectrum to capture depth information, half of them recording visible light. Plus, a number of ultraviolet cameras. “We use ultraviolet paint to mask areas for effects work,” Jelley explains, “so we record the UV spectrum, too. Basically we use every glimmer of light we can get short of asking you to swallow radium.”

The cameras shoot between 30 and 60 times a second. “We have a directional map of the configuration of those cameras, and we overlay that with a depth map that we’ve captured from the IR cameras. Then we can do all the pixel interpolation.”

This is a big step up from mocap. Volumetric video captures real-time depth information from surfaces themselves: there are no fluorescent sticky dots or sliced-through ping-pong balls attached to actors here. As far as the audience is concerned, volumetric video is essentially just that, video, and as close to a true record as anything piped through a basement full of computers is ever going to get.

So what kind of films are made in such studios? Right now, the education company Pearson is creating virtual consultations for trainee nurses. Fashion brands and car companies have shot adverts here. TV companies want to use them for fully immersive and interactive dramas.

“I know she’s not real, but my body doesn’t. Every bit of me has fallen for this super-real gymnast”

On a table nearby, a demo is ready to watch on a Vive VR headset. There are three sets of performances for me to observe, all looping in a grey, gridded, unadorned virtual space: the digital future as a filing cabinet. There are two experiments from Sullivan’s early days at Microsoft. Thomas Jefferson is pure animatronic; the two Maori haka dancers are engaging, if unhuman. The circus gymnast swinging on her hoop is different. I recognise her, or think I do. My body-language must be giving the game away, because Jelley laughs.

“Go up to her,” he says. I can’t place where I’ve seen her before. I try and catch her eye. “Closer.”

I’m invading her space, and I’m not comfortable with this. I can see the individual threads, securing the sequins to her costume. More than that: I can smell her. I can feel the heat coming from her skin.

I know she’s not real, but my body doesn’t. Every bit of me that might have rejected a digitised face as uncanny has fallen hook, line and sinker for this super-real gymnast. And this, presumably, is why the bit of my mind that enables me to communicate freely and easily with my fellow humans is in overdrive, trying to plug the gaps in my experience, as if to say, “Of course her skin is hot. Of course she has a scent.”

Mori’s uncanny valley effect is not quantifiable, and I don’t suppose my experience is any more measurable than the one Mori identified. But I’d bet the farm that, had you scanned me, you would have seen all manner of pretty lights. This hasn’t been an eerie experience. Quite the reverse. It’s terrifyingly ordinary. Almost, I might say, human.

Jelley walks me back to the main road. Neither of us says a word. He knows what he has. He knows what he has done.

Outside the snack shack, three shop dummies in pirate gear wobble in the wind.