Technology vs observation

Losing my rag at the Royal Academy for New Scientist, 13 December 2017

When the schools of London’s Royal Academy of Arts were opened in 1769, life drawing — the business of sketching either live models or the plaster casts of worthy sculptures — was an essential component of an artist’s training.

As I wandered around From Life, an exhibition devoted to the history as well as the future of the practice, I overheard a curator explaining that, now life drawing is no longer obligatory in Royal Academy art courses, a new generation of artists are approaching the practice in a “more expressive” way. The show’s press release claims even more: that life drawing is evolving “as technology opens up new ways of creating and visualising artwork”.

There was little of this in evidence when I visited, however: two-and-a-half of the three virtual-reality experiences on offer had broken down. Things break down when the press turns up – you might even say it’s a rule. Still, given their ubiquity, I’m beginning to wonder whether gallery-based VR malfunctions are not a kind of mischievous artwork in their own right. In place of a virtual sketch, a message in an over-friendly font asks: “Have you checked your internet connection?” At least Swiss artist Jean Tinguely’s wild mobiles of the 1960s had the decency to catch fire.

How can new technologies like Google’s Tilt Brush and HTC’s Vive VR platform bring artists into a more intimate relation to their subject — more intimate than might be achieved by, say, standing a metre away from a naked stranger armed only with a bit of charcoal?

Jonathan Yeo has had a stab at the problem, using Tilt Brush’s 3D painting tech to fashion a sculptural self-portrait. The outsize bronze 3D print of his effort — an assemblage of short, wide, hesitant virtual “brushstrokes” — has a curiously dated feel and wouldn’t look out of place in a group retrospective of 20th-century British sculpture. As an advert for a technology that prides itself on its expressivity (videos of the platform at work usually resemble explosions in a paint factory), it’s a curiously laborious piece.

On a nearby wall hang Gillian Wearing’s photographic self-portraits, manipulated using the sort of age-progression technology employed by forensic artists. In this way, Wearing has captured her appearance 10, 20, 30 years into her future. It’s an undeniably moving display, and undeniably off the point: life drawing is about capturing the present moment, which leaves Wearing’s contribution resembling those terms and conditions that appear at the bottom of TV advertisements – Other Moments Are Available.

Yinka Shonibare (best known for his ship-in-a-bottle sculpture on the fourth plinth of Trafalgar Square) comments on the show, rather than contributes to it, with a 3D VR conceptualisation of a painting by the 18th-century Scots artist and dealer Gavin Hamilton.

Hamilton once sold a Roman sculpture to a collector. Shonibare has scanned a plaster cast of this Townley Venus, then placed it on a plinth in a largely imaginary VR garden (you catch only a glimpse of this space in Hamilton’s painting). He has covered its plaster-white surface with batik designs (referring to common sub-Saharan African fabric, though it was originally a Dutch export) and as a coup de grâce, he has stuck a globe on Venus’s torso in place of her head. The point is that we can never copy something without to some degree appropriating it. Whether you like what he’s done will depend on whether you like art that makes a primarily intellectual point.

In a gallery environment increasingly besotted by (and bested by) technology, such acts of cultural orienteering may be necessary; they’re certainly inevitable. The new work gracing From Life at least attempts to address the theme of the show, and its several failures are honest and interesting.

Still, I keep coming back to the historical half of the exhibition — to the casts, the drawings, the portraits of struggling young artists from 1769 to now. Life drawing is not obligatory for artists? It should be obligatory for everyone. If we never learn to observe honestly, what the devil will we ever have to be expressive about?

It’s coming at you!

OUTSIDE Dimension Studios in Wimbledon, south London, is one of those tiny wood-framed snack bars that served commercial travellers in the days before motorways. The hut is guarded by old shop dummies dressed in fishnet tights and pirate hats. If the UK made its own dilapidated version of Westworld, the cyborg rebellion would surely begin here.

Steve Jelley orders us breakfast. Years ago he left film production to pursue a career developing new media. He’s of the generation for whom the next big thing is always just around the corner. Most of them perished in the dot-com bust of 2001, but Jelley clung to the dream, and now Microsoft has come calling.

His company, Hammerhead, makes 360-degree videos for commercial clients. Its partner in this current venture, Timeslice Films, is best known for volumetric capture of still images – the business of cinematographically recording forms in three dimensions – a practice that goes back to founder Tim MacMillan’s art-school experiments of the early 1980s.

Steve Sullivan, director of the Holographic Video initiative at Microsoft, is fusing both companies’ technical expertise to create volumetric video: immersive entertainment that’s indistinguishable from reality.

There are only three studios in the world that can do this with any degree of conviction, and Wimbledon is the only one outside the US. Still, I’m sceptical. It has been clear for a while that truly immersive media won’t spring from a single “light-bulb” moment. The technologies involved are, in conceptual terms, surprisingly old. Volumetric capture is a good example.

MacMillan is considered the godfather of this tech, having invented the “bullet time” effect central to The Matrix. But The Matrix is 18 years old, and besides, MacMillan reckons that pioneer photographer Eadweard Muybridge got to the idea years before him – in fact, decades before cinema was invented.

“Engineer Masahiro Mori says his ‘uncanny valley’ idea was never meant to be taken scientifically”

Then there’s motion capture (or mocap): recording the movement of points attached to an actor, and from those points, constructing the performance of a three-dimensional model. The pioneering Soviet physiologist Nikolai Bernstein invented the technique in the early 1920s, while developing training programmes for factory workers.

Truly immersive media will be achieved not through magic bullets, but through thugging – the application of ever more computer power, and the ever-faster processing of more and more data points. Impressive, but where’s the breakthrough?

“Well,” Jelley begins, handing me what may be the largest bacon sandwich in London, “you know this business of the ‘uncanny valley’…?” My heart sinks slightly.

Most New Scientist readers will be familiar with Masahiro Mori’s concept of the uncanny valley. It’s a curiously anglophone obsession. In the 30 years since the Japanese engineer published his paper in 1970, it has been referred to in Japanese academic literature only once. Mori himself says the idea was never meant to be taken scientifically. He was merely warning robot designers at a time when humanoid robots didn’t exist that the closer their works came to resemble people, the creepier we would find them.

In the West, discussions of the uncanny valley have grown to a sizeable cottage industry. There have been expensive studies done with PET scans to prove the existence of the effect. But as Mori commented in an interview in 2012: “I think that the brainwaves act that way because we feel eerie. It still doesn’t explain why we feel eerie to begin with.”

Our discomfort extends beyond encounters with physical robots to include some cinematic experiences. Many are the animated movies that have employed mocap to achieve something like cinematic realism, only to plummet without trace into the valley.

Elsewhere, actor Andy Serkis famously uses mocap to transform himself into characters like Gollum in The Lord of the Rings, or the chimpanzee Caesar in Rise of the Planet of the Apes, and we are carried along well enough by these films. The one creature this technology can’t emulate, however, is Serkis himself. Though mocap now renders human body movement with impressive realism, the human face remains a machine far too complex to be seamlessly emulated even by the best system.

Jelley reckons he and his partners have “solved the problem” of the uncanny valley. He leads me into the studio. There’s a small, circular, curtained-off area – a sort of human-scale birdcage. Rings of lights and cameras are mounted on scaffolds and hang from a moveable and very heavy-looking ceiling rig.

There are 106 cameras: half of them recording in the infrared spectrum to capture depth information, half of them recording visible light. Plus, a number of ultraviolet cameras. “We use ultraviolet paint to mask areas for effects work,” Jelley explains, “so we record the UV spectrum, too. Basically we use every glimmer of light we can get short of asking you to swallow radium.”

The cameras shoot between 30 and 60 times a second. “We have a directional map of the configuration of those cameras, and we overlay that with a depth map that we’ve captured from the IR cameras. Then we can do all the pixel interpolation.”

This is a big step up from mocap. Volumetric video captures real-time depth information from surfaces themselves: there are no fluorescent sticky dots or sliced-through ping-pong balls attached to actors here. As far as the audience is concerned, volumetric video is essentially just that, video, and as close to a true record as anything piped through a basement full of computers is ever going to get.

So what kind of films are made in such studios? Right now, the education company Pearson is creating virtual consultations for trainee nurses. Fashion brands and car companies have shot adverts here. TV companies want to use them for fully immersive and interactive dramas.

“I know she’s not real, but my body doesn’t. Every bit of me has fallen for this super-real gymnast”

On a table nearby, a demo is ready to watch on a Vive VR headset. There are three sets of performances for me to observe, all looping in a grey, gridded, unadorned virtual space: the digital future as a filing cabinet. There are two experiments from Sullivan’s early days at Microsoft. Thomas Jefferson is pure animatronic; the two Maori haka dancers are engaging, if unhuman. The circus gymnast swinging on her hoop is different. I recognise her, or think I do. My body-language must be giving the game away, because Jelley laughs.

“Go up to her,” he says. I can’t place where I’ve seen her before. I try and catch her eye. “Closer.”

I’m invading her space, and I’m not comfortable with this. I can see the individual threads, securing the sequins to her costume. More than that: I can smell her. I can feel the heat coming from her skin.

I know she’s not real, but my body doesn’t. Every bit of me that might have rejected a digitised face as uncanny has fallen hook, line and sinker for this super-real gymnast. And this, presumably, is why the bit of my mind that enables me to communicate freely and easily with my fellow humans is in overdrive, trying to plug the gaps in my experience, as if to say, “Of course her skin is hot. Of course she has a scent.”

Mori’s uncanny valley effect is not quantifiable, and I don’t suppose my experience is any more measurable than the one Mori identified. But I’d bet the farm that, had you scanned me, you would have seen all manner of pretty lights. This hasn’t been an eerie experience. Quite the reverse. It’s terrifyingly ordinary. Almost, I might say, human.

Jelley walks me back to the main road. Neither of us says a word. He knows what he has. He knows what he has done.

Outside the snack shack, three shop dummies in pirate gear wobble in the wind.

The genius of making a little go a long way

Visiting Illuminating India at London’s Science Museum for New Scientist, 10 October 2017

One can taste the boosterism in the air at London’s Science Museum as it introduces its two-gallery exhibition, Illuminating India.

There is a cafe serving excellent Indian street food. Someone next to me used the word “Commonwealth” without irony. Would there have been such a spirit without Brexit? Probably not: this is a show about the genius of another country that very much wants to project Britain’s own global aspirations. Any historian of Anglo-British relations will give a sardonic smile at this.

When you visit (and you should), try to look around the smaller, artefacts-driven gallery first.

This room tells the stories of Indian science – stories plural because there can never be one, linear account of how such dissimilar and contesting cultures struggled and more or less succeeded in understanding and exploiting a space of such extraordinary complexity.

Naturally, since India has a past to boast of, pride of place goes to its indigenous cultures. It was the Indus valley civilisation, after all, whose peoples fashioned standardised weights around 4000 years ago: items that indicate high levels of arithmetical literacy, communication and trade.

And there are reconstructions of Ayurvedic surgical instruments described in records dating back to around 500 BC. Also on show is a 1800-year-old document containing the first example of the use of zero. Wonderfully, radiocarbon dating pushed the document’s age back by 500 years just before the exhibition opened.

It is a measure of the wisdom of the curators that such an illustrious past isn’t allowed to overshadow India’s more recent achievements. For example, Jagadish Chandra Bose’s early-20th-century crescograph, designed to observe plant growth at a magnification of 10,000 times, reminds us why he is often called the father of modern Indian science.

Another winning object is Chandrasekhara Raman’s spectrometer. Raman was the first Indian to win a Nobel prize, for physics, in 1930.

And what of that other great empire far to the north? Well, there is a map of George Everest’s career-defining Great Trigonometrical Survey of India – the teamwork of 70 years distilled on a single, meticulously drawn map. And nearby are details of a recent collaboration between Surrey Satellite Technology in the UK and the Indian Space Research Organisation on the Earth-surveying NovaSAR satellite.

Some of the deeper, darker questions about Anglo-Indian relations are posed in the second, photographic half of the exhibition.

There, the anthropometric photographs of Maurice Portman make a depressingly silly impression next to the respectful, revealing and entirely unlicentious photographs Ram Singh took of the women of his own harem: powerful political players all, who shaped the country through marriage and allied treaties.

It is hard to say why the split nature of Illuminating India works as well as it does. It has something to do with the way the rooms handle political power.

India’s science, from its ancient stepwells that gathered monsoon waters to the bureaucratic and algorithmic marvel that is today’s tiffin tin-based lunch delivery system, has been driven by the complex needs of a massive population making a living.

Similarly, its doing-more-with-less style of innovation is reflected in everything from the world’s cheapest artificial leg (the Jaipur leg, made of rubber, plastic and wood) to the world’s cheapest Mars-orbiting camera.

Visitors to Illuminating India will leave thinking that technology may, after all, be making the world a better place, and that what people do is ultimately more influential than who they are.

When art and technology pull each other to bits

ars-electronica

Visiting Ars Electronica for New Scientist, 21 September 2016

In a disused mail sorting office in Linz, Austria, an industrial robot twice my height has got hold of Serbian-born artist Dragan Ilic and is wiping him over a canvas-covered wall. Ilic is clutching pencils, and as the robot twirls and dabs him against the wall, the artist makes his own frantic marks – a sort of sentient brush head. These performances are billed as a collaboration between artist and machine. All I see is the user getting used.

Every September in Linz, the Ars Electronica festival tries to marry technology and art. Andy Warhol took up screen-printing in the 1960s, and a whole generation of gallery-goers have since grown up with the notion that this match is easy.

Indeed, the coupling of art and technology has become a solid pillar of art education, especially now that so much funding comes from IT big hitters such as Sony and private institutions like the Wellcome Trust.

But if the venerable Ars Electronica has demonstrated anything in its 37-year history – beyond the ability of the arts to remake and rejuvenate a city – it is that technology and art make astonishingly unhappy bedfellows.

This year, for example, Swiss artist Daniel Boschung used an industrial robot controlled by bespoke software to take 900-million-pixel portraits of people – forensic surveys so detailed that they drained all emotion from their subjects’ faces.

Not far away, another robot, Davide Quayola’s Sculpture Factory, chiselled through partially completed life-size stone renditions of Michelangelo’s David. The attention was directed to the pixelated nature of 3D scanning, which smeared, spread and tessellated the biblical giant-killer to the point of incoherence: here a limb, there an eye.

Walking the corridor towards the current sculpture-in-progress, one passed attempt after attempt of clone Davids peeking in more or less agonised fashion from their cuboid stone prisons, in a sort of mineral retread of that infamous scene from Alien: Resurrection.

The provoking thing about Ars Electronica is that it jams together boutique displays of the latest technology, trenchant criticisms of the post-industrial project, jokes and honest failures. It is a gargantuan vessel powered by enthusiasm, but steered by nothing remotely resembling taste.

Eventually, the visitor comes to rest against one of the more obvious wins. Boris Labbé’s film Rhizome, which netted the festival’s annual animation prize, unites watercolour illustration and digital effects to tell an epic tale of evolution, civilisation and cosmic transformation in one steady, heart-stopping reverse zoom.

And then there is Frank Kolkman’s OpenSurgery, which combines 3D printing and laser-cutting with hacked surgical pieces and components bought online. This is a DIY robot that can perform laparoscopic surgery – a terrifying comment on the way that hacking and “making” are increasingly expected to stand in for the real thing.

You’ll need a drink after all that, so head for Max Dovey’s A Hipster Bar. And good luck – this genuine pop-up drinking hole will, in true neo-liberal fashion, keep the gate shut unless its face-recognition system considers you “90 per cent hipster”.

As you sip, ponder this: it was the assertion of the Romantic movement that art makes us appreciate the beauty, richness and sheer size of the world. And technology, used appropriately, brings us closer to that sublime. As the Romantics’ acolyte, the writer and pioneering pilot Antoine de Saint-Exupéry, put it: “The machine does not isolate man from the great problems of nature but plunges him more deeply into them.”

Even if that was true in 1939, it’s not true now: not now our drones do our flying for us; not now our technology has got away from us to the point where large portions of nature are being erased; not now we live mired in media and, indeed, have to make special efforts to escape it.

Naturally, artists want to explore the new technology of their day, but these days the best results seem to come when we misappropriate the machines and kick them into new shapes, or simply point and laugh.

Putting the wheel in its place

wheel
Mary Evans / Grenville Collins

 

What made the rickshaw so different from a wagon or an ox-cart and, in the eyes of many Westerners, so cruel, was the idea of it being pulled by a man instead of a farm animal. Pushing wheelchairs and baby carriages posed no problem, but pulling turned a man into a beast.
for New Scientist, 20 January 2016