Cosmoscope

We’ve learned much in the half-millennium since Leonardo declared Man “the measure of all things.”, and seen the human species relegated to a footnote in the cosmological story.
Now we’re beginning to see that humanity maybe does sit at the heart of the universe. At no other scale but ours does the universe attain such complexity.

Exhibited at the London Lumiere festival in January 2018, Simeon Nelson’s 3.3 metre-high singing, flashing sculpture, is an enormous puzzle in structural engineering, sound and software design. It’s also a homage to cosmological models of the past, especially Leonardo’s “Vitruvian Man”, drawn around 1490.

I interviewed the makers for this video by David Stock.

What’s the Russian for Eastbourne?

This is a journey through an exotic world conjured into being by the Military Topographic Directorate of the General Staff of the Soviet Army. Tasked by Stalin during the Second World War with accurately and secretly mapping the Soviet Union, its Eastern European allies, its Western adversaries, and the rest of the world, the Directorate embarked on the largest mapping effort in history.

Reading Davies and Kent’s Red Atlas for the Telegraph, 13 January 2018

A glimpse at time

Time is difficult to talk about – the show’s cumbersome title, Life Time: Biological clocks of the universe, is proof enough of that. Even the gallery’s lucid handout by William Myers, a curator based in Amsterdam, labours under the title “A Non-Circadian Cadence”. But the show itself does much better, embracing a wide swathe of temporal landscape, from the universal to the personal and from the cellular to the geological.

Visiting MU Artspace, Eindhoven for New Scientist, 20 January 2018

Music and art stop dementia from stealing everything we cherish

According to Greek legend, 49 of King Danaus’s 50  daughters were mariticidal, and condemned to fill a leaky bath in hell, and their lot is an apt metaphor for the human condition. However much we fill our lives, our lives still dribble away. We experience, we learn – but we also forget. 

A choir and a De Kooning show inspired this piece for New Scientist, 4 January 2018

Just experience it

According to the curators, these exquisite allegorical frescoes by 18th-century artist Johann Wenzel Bergl are “recognizable as strategies of absolutist picture propaganda”. And Mark Dion’s installation capturing “the lifestyle and self-image of the prototypical ethnographer of colonial times”, isn’t even that, but alludes “to our own imagination of that ethnographer”. I left feeling rather as Lewis Carroll’s Alice might have felt if, instead of freely stepping through the mirror, she had been shoved through it from behind by a gang of goonish anthropologists.

Visiting mumok, Vienna’s museum of contemporary art, for New Scientist, 23 December 2017

Technology vs observation

Things break down when the press turns up – you might even say it’s a rule. Still, given their ubiquity, I’m beginning to wonder whether gallery-based VR malfunctions are not a kind of mischievous artwork in their own right. In place of a virtual sketch, a message in an over-friendly font asks: “Have you checked your internet connection?” At least Swiss artist Jean Tinguely’s wild mobiles of the 1960s had the decency to catch fire.

Losing my rag at the Royal Academy for New Scientist, 13 December 2017

It’s coming at you!

OUTSIDE Dimension Studios in Wimbledon, south London, is one of those tiny wood-framed snack bars that served commercial travellers in the days before motorways. The hut is guarded by old shop dummies dressed in fishnet tights and pirate hats. If the UK made its own dilapidated version of Westworld, the cyborg rebellion would surely begin here.

Steve Jelley orders us breakfast. Years ago he left film production to pursue a career developing new media. He’s of the generation for whom the next big thing is always just around the corner. Most of them perished in the dot-com bust of 2001, but Jelley clung to the dream, and now Microsoft has come calling.

His company, Hammerhead, makes 360-degree videos for commercial clients. Its partner in this current venture, Timeslice Films, is best known for volumetric capture of still images – the business of cinematographically recording forms in three dimensions – a practice that goes back to founder Tim MacMillan’s art-school experiments of the early 1980s.

Steve Sullivan, director of the Holographic Video initiative at Microsoft, is fusing both companies’ technical expertise to create volumetric video: immersive entertainment that’s indistinguishable from reality.

There are only three studios in the world that can do this with any degree of conviction, and Wimbledon is the only one outside the US. Still, I’m sceptical. It has been clear for a while that truly immersive media won’t spring from a single “light-bulb” moment. The technologies involved are, in conceptual terms, surprisingly old. Volumetric capture is a good example.

MacMillan is considered the godfather of this tech, having invented the “bullet time” effect central to The Matrix. But The Matrix is 18 years old, and besides, MacMillan reckons that pioneer photographer Eadweard Muybridge got to the idea years before him – in fact, decades before cinema was invented.

“Engineer Masahiro Mori says his ‘uncanny valley’ idea was never meant to be taken scientifically”

Then there’s motion capture (or mocap): recording the movement of points attached to an actor, and from those points, constructing the performance of a three-dimensional model. The pioneering Soviet physiologist Nikolai Bernstein invented the technique in the early 1920s, while developing training programmes for factory workers.

Truly immersive media will be achieved not through magic bullets, but through thugging – the application of ever more computer power, and the ever-faster processing of more and more data points. Impressive, but where’s the breakthrough?

“Well,” Jelley begins, handing me what may be the largest bacon sandwich in London, “you know this business of the ‘uncanny valley’…?” My heart sinks slightly.

Most New Scientist readers will be familiar with Masahiro Mori’s concept of the uncanny valley. It’s a curiously anglophone obsession. In the 30 years since the Japanese engineer published his paper in 1970, it has been referred to in Japanese academic literature only once. Mori himself says the idea was never meant to be taken scientifically. He was merely warning robot designers at a time when humanoid robots didn’t exist that the closer their works came to resemble people, the creepier we would find them.

In the West, discussions of the uncanny valley have grown to a sizeable cottage industry. There have been expensive studies done with PET scans to prove the existence of the effect. But as Mori commented in an interview in 2012: “I think that the brainwaves act that way because we feel eerie. It still doesn’t explain why we feel eerie to begin with.”

Our discomfort extends beyond encounters with physical robots to include some cinematic experiences. Many are the animated movies that have employed mocap to achieve something like cinematic realism, only to plummet without trace into the valley.

Elsewhere, actor Andy Serkis famously uses mocap to transform himself into characters like Gollum in The Lord of the Rings, or the chimpanzee Caesar in Rise of the Planet of the Apes, and we are carried along well enough by these films. The one creature this technology can’t emulate, however, is Serkis himself. Though mocap now renders human body movement with impressive realism, the human face remains a machine far too complex to be seamlessly emulated even by the best system.

Jelley reckons he and his partners have “solved the problem” of the uncanny valley. He leads me into the studio. There’s a small, circular, curtained-off area – a sort of human-scale birdcage. Rings of lights and cameras are mounted on scaffolds and hang from a moveable and very heavy-looking ceiling rig.

There are 106 cameras: half of them recording in the infrared spectrum to capture depth information, half of them recording visible light. Plus, a number of ultraviolet cameras. “We use ultraviolet paint to mask areas for effects work,” Jelley explains, “so we record the UV spectrum, too. Basically we use every glimmer of light we can get short of asking you to swallow radium.”

The cameras shoot between 30 and 60 times a second. “We have a directional map of the configuration of those cameras, and we overlay that with a depth map that we’ve captured from the IR cameras. Then we can do all the pixel interpolation.”

This is a big step up from mocap. Volumetric video captures real-time depth information from surfaces themselves: there are no fluorescent sticky dots or sliced-through ping-pong balls attached to actors here. As far as the audience is concerned, volumetric video is essentially just that, video, and as close to a true record as anything piped through a basement full of computers is ever going to get.

So what kind of films are made in such studios? Right now, the education company Pearson is creating virtual consultations for trainee nurses. Fashion brands and car companies have shot adverts here. TV companies want to use them for fully immersive and interactive dramas.

“I know she’s not real, but my body doesn’t. Every bit of me has fallen for this super-real gymnast”

On a table nearby, a demo is ready to watch on a Vive VR headset. There are three sets of performances for me to observe, all looping in a grey, gridded, unadorned virtual space: the digital future as a filing cabinet. There are two experiments from Sullivan’s early days at Microsoft. Thomas Jefferson is pure animatronic; the two Maori haka dancers are engaging, if unhuman. The circus gymnast swinging on her hoop is different. I recognise her, or think I do. My body-language must be giving the game away, because Jelley laughs.

“Go up to her,” he says. I can’t place where I’ve seen her before. I try and catch her eye. “Closer.”

I’m invading her space, and I’m not comfortable with this. I can see the individual threads, securing the sequins to her costume. More than that: I can smell her. I can feel the heat coming from her skin.

I know she’s not real, but my body doesn’t. Every bit of me that might have rejected a digitised face as uncanny has fallen hook, line and sinker for this super-real gymnast. And this, presumably, is why the bit of my mind that enables me to communicate freely and easily with my fellow humans is in overdrive, trying to plug the gaps in my experience, as if to say, “Of course her skin is hot. Of course she has a scent.”

Mori’s uncanny valley effect is not quantifiable, and I don’t suppose my experience is any more measurable than the one Mori identified. But I’d bet the farm that, had you scanned me, you would have seen all manner of pretty lights. This hasn’t been an eerie experience. Quite the reverse. It’s terrifyingly ordinary. Almost, I might say, human.

Jelley walks me back to the main road. Neither of us says a word. He knows what he has. He knows what he has done.

Outside the snack shack, three shop dummies in pirate gear wobble in the wind.

Future by design

The Second Digital Turn: Design beyond intelligence
Mario Carpo
MIT Press

THE Polish futurist Stanislaw Lem once wrote: “A scientist wants an algorithm, whereas the technologist is more like a gardener who plants a tree, picks apples, and is not bothered about ‘how the tree did it’.”

For Lem, the future belongs to technologists, not scientists. If Mario Carpo is right and the “second digital turn” described in his extraordinary new book comes to term, then Lem’s playful, “imitological” future where analysis must be abandoned in favour of creative activity, will be upon us in a decade or two. Never mind our human practice of science, science itself will no longer exist, and our cultural life will consist of storytelling, gesture and species of magical thinking.

Carpo studies architecture. Five years ago, he edited The Digital Turn in Architecture 1992-2012, a book capturing the curvilinear, parametric spirit of digital architecture. Think Frank Gehry’s Guggenheim Museum in Bilbao – a sort of deconstructed metal fish head – and you are halfway there.

Such is the rate of change that five years later, Carpo has had to write another book (the urgency of his prose is palpable and thrilling) about an entirely different kind of design. This is a generative design powered by artificial intelligence, with its ability to thug through digital simulations (effectively, breaking things on screen until something turns up that can’t be broken) and arriving at solutions that humans and their science cannot better.

This kind of design has no need of casts, stamps, moulds or dies. No costs need be amortised. Everything can be a one-off at the same unit cost.

Beyond the built environment, it is the spiritual consequences of this shift that matter, for by its light Carpo shows all cultural history to be a gargantuan exercise in information compression.

Unlike their AIs, human beings cannot hold much information at any one time. Hence, for example, the Roman alphabet: a marvel of compression, approximating all possible vocalisations with just 26 characters. Now that we can type and distribute any glyph at the touch of a button, is it any wonder emojis are supplementing our tidy 26-letter communications?

Science itself is simply a series of computational strategies to draw the maximum inference from the smallest number of precedents. Reduce the world to rules and there is no need for those precedents. We have done this for so long and so well some of us have forgotten that “rules” aren’t “real” rules, they are just generalisations.

AIs simply gather or model as many precedents as they wish. Left to collect data according to their own strengths, they are, Carpo says, “postscientific”. They aren’t doing science we recognise: they are just thugging.

“Carpo shows all cultural history to be a gargantuan exercise in information compression”

Carpo foresees the “separation of the minds of the thinkers from the tools of computation”. But in that alienation, I think, lies our reason to go on. Because humans cannot handle very much data at any one time, sorting is vital, which means we have to assign meaning. Sorting is therefore the process whereby we turn data into knowledge. Our inability to do what computers can do has a name already: consciousness.

Carpo’s succinctly argued future has us return to a tradition of orality and gesture, where these forms of communication need no reduction or compression since our tech will be able to record, notate, transmit, process and search them, making all cultural technologies developed to handle these tasks “equally unnecessary”. This will be neither advance nor regression. Evolution, remember, is maddeningly valueless.

Could we ever have evolved into Spock-like hyper-rationality? I doubt it. Carpo’s sincerity, wit and mischief show that Prospero is more the human style. Or Peter Pan, who observed: “You can have anything in life, if you will sacrifice everything else for it.”

 

The sooner we pave over this lot, the better

Venom: Killer and cure ran at London’s Natural History Museum to 13 May 2018…

Londoners! This holiday season, why not take the children along to the Natural History Museum? Its new exhibition Venom: Killer and cure brims over with fascinating and entertaining stories.

Have you heard about the emerald cockroach wasp (Ampulex compressa), which zombifies its cockroach prey with its sting before laying an egg on it that hatches into a larva that eats the cockroach alive while knowing, somehow, to leave its vital organs till last?

Too strong? Then how about the paralysis-inducing bites of the marine bloodworm (Glycera dibranchiata), whose copper-reinforced teeth are one of the toughest known structures in the natural world?

Oh, dear. There must be something child-friendly round here… How about the deer fly (Chrysops sp.)? The males feed exclusively on nectar! Unfortunately, the females feed exclusively on blood and have evolved an anticoagulant venom to keep their meals flowing.

Nods to some ingenious medicine aside, Venom seems hell-bent on convincing visitors that “nature” is a state of perpetual, terrible and gruesome conflict, and that – if your environmental competitors have their way – your whole lived experience is going to be filled with excruciating pain.

Those with strong enough stomachs will marvel at the ingenuity of nature’s torturers. Even the Iberian ribbed newt (Pleurodeles waltl), which hardly sounds the fiercest animal in the pantheon, has ribs which burst out through its poisonous skin to deter predators.

Those of a philosophic bent will appreciate the show’s underlying narrative, explaining how human cunning makes us the most efficient, though by no means the only, harvester of venom. There’s a sea swallow (Glaucus atlanticus) here, in the form of an extraordinarily delicate and beautiful glass model made by Leopold and Rudolf Blaschka. This pretty sea slug, about 2.5-centimetres long, eats Portuguese man-of-war (Physalia physalis) and collects their venom in its own tentacles, which it fires at predators to defend itself.

The fine-art crowd will thrill to artist Steve Ludwin’s 30-year project of no certain purpose: injecting himself with snake venom. Those of a literary bent, meanwhile, will savour the elegant phrasing of Justin Schmidt’s sting pain scale. Of the Western yellow jacket wasp (Vespula pensylvanica) he writes: “Hot and smoky, almost irreverent. Imagine W. C. Fields extinguishing a cigar on your tongue.”

Venom shows London’s Natural History Museum at its best: the exhibition is intimate, but not claustrophobic; unafraid of detail, but eminently accessible; visually arresting, but not exhausting.

I left trembling, angry and depressed. Had the show let me down? Quite the contrary: if anything, it had over-delivered.

How long, I wondered, must we put up with this ghastly horror-show world of ours? Why should we have to tolerate the way competing slow lorises (Nycticebus sp.) inflict festering wounds on each other, and male emperor scorpions (Pandinus imperator) feel the need to sting their females before they dare broach the subject of sex?

Venom has convinced me that nature is vile. It is pitiless and disgusting, and the sooner we pave over it the better.

The boring beasts that changed the world

Some animals are so familiar, we barely see them. If we think of them at all, we categorise them according to their role in our lives: as pests or food; as unthinking labourers or toy versions of ourselves. If we looked at them as animals – non-human companions riding with us on our single Earth – what would we make of them? Have we raised loyal subjects, or hapless victims, or monsters?

Visiting the Museum of Ordinary Animals exhibition for New Scientist, 4 November 2017