Liquid Crystal Display: Snap judgements

Lead_Waad-AlBawardi,-The-Hidden-Life-of-Crystals,

Visiting Liquid Crystal Display at SITE Gallery, Sheffield, for New Scientist, 31 October 2018

Untitled Gallery was founded in Sheffield in 1979. It specialised in photography. In 1996 it was renamed Site Gallery and steadily expanded its remit to cover the intersection between science and art. Nearly 30 years and a £1.7million refit later, Site Gallery is the new poster child of Sheffield’s Cultural Industries Quarter, with an exhibition, Liquid Crystal Display, that cleverly salutes its photographic past.

Most shows about art value the results over the ingredients. The picture matters more than the paint. The statue matters more than the stone. Exhibitions about photography give rather more space to process because photography’s ingredients are so involved and fascinating.

Liquid Crystal Display follows this photographic logic to its end. This is a show about the beauty, weight and messiness of materials we notice only when they’ve stopped working. It’s about the beauty created by a broken smartphone screen, a corroded battery, a cracked lens.

Site Gallery’s new exhibition – a cabinet of curiosities if ever there was one – collides science and art, the natural and the manufactured, the old and the new. It puts the exquisite sketches of 19th-century Scottish chemist and photographer Mungo Ponton (detailing his observations of how crystals polarise light), next to their nearest contemporary equivalent: microscopic studies (pictured) of liquid crystals caught in the process of self-organisation by Waad AlBawardi, a Saudi molecular biologist who’s currently in Edinburgh, researching the structure of DNA organisation inside cells.

This provocative pairing of the relatively simple and the manifestly complex is repeated several times. Near a selection of crystals from John Ruskin’s mineral collection sit the buckets, burners and batteries of Jonathan Kemp, Martin Howse and Ryan Jordan’s The Crystal World project, a tabletop installation recording their hot, smelly, borderline-hazardous effort to extract the original minerals from bits of scavenged computers. Curated by Laura Sillars, assisted by Site Gallery’s own Angelica Sule, Liquid Crystal Display reveals the material, mineral reality behind our oh-so-weightless holographic world of digital imagery. “Liquid crystals polarise light, produce colour and yet, as a material form, recede into the background of technology,” Sillars wrote in the catalogue to this show.

JulesLister_24Sept2018Large_4024-Large

This awareness is not new, of course. In the 1960s, liquid crystals were being burned on overhead projectors to create psychedelic light shows. J G Ballard’s novel The Crystal World (1966) concocted a paranoid vision of a world and a civilisation returned (literally) to its mineral roots. That story receives a handsome homage here from the scifi-obsessed Norwegian artist Anne Lislegaard, whose stark monochrome animation (above) turns the sharp shadows and silhouettes cast by contemporary domestic furniture into insidious crystalline growths.

Arrayed within Anna Barham’s peculiar hexagonal cabinetwork, a gigantic piece of display furniture that is itself an artwork, the pictures, objects, films and devices in Liquid Crystal Display speak to pressing topical worries – resource depletion, environmental degradation, the creeping uncanny of digital experience – while at the same time evoking a peculiar nostalgia for our photochemical past.

The exhibition lacks one large signature object against which visitors can take selfies. A peculiar omission in a show that’s relaunching a gallery. And a bit of a shame for an exhibition that, in its left-field way, has handsomely captured the philosophical essence of photography.

Tomás Saraceno: Beneath an ocean of air

Visiting Tomás Saraceno’s Berlin studio for New Scientist, 13 October 2018

THE Argentine-born artist Tomás Saraceno maintains a studio in Berlin – if you can call a disused chemicals factory a studio. There is nothing small about this operation. Saraceno, who trained as an architect in Buenos Aires, now employs hundreds of people, with specialisms ranging from art history and architecture to biology and anthropology. If you’re serious about saving the world, you need this kind of cross-disciplinary team, I suppose.

Though Saraceno hasn’t exactly promised to save the world, he has been dropping some big hints. His utopian installations include Cloud Cities at Hamburger Bahnhof, Berlin, in 2011 – a collection of geometric, inflated shapes. Even by the time of his Observatory/Air-Port-City show at London’s Hayward Gallery in 2008, these shapes contained autonomous residential units. A network of habitable cells floated in the air, combining and recombining like clouds.

A year later at the Metropolitan Museum in New York, gallery-goers got to explore these spaces via 16 interconnected modules made up of glass segments held in place by steel cables. And in June 2013, the K21 gallery in Düsseldorf invited visitors to wander more than 25 metres above the gallery’s piazza across a web dotted with inflated PVC spheres.

This is Saraceno’s answer to our global problems: he wants us to take to the air. That’s why he coined the term “Aerocene” for one of his projects. He wants people to think of climate change in terms of possibility, playfulness and, yes, escape. “We live beneath an ocean of air,” he once wrote, as he sketched his utopian vision of a city in the clouds. “But we’ve yet to find a way to inhabit it.”

Near his Berlin studio is a scruffy public park. Part of it is marked out for football. Behind one goal stands a graffitied stretch of the Berlin Wall. Today there’s another attraction: two men are running back and forth, trying to fill a black bag as big as a minivan with air. It is a fine, windless day; the air in the bag heats up quickly, and once it is sealed, the container rises into the sky. A bag no longer, it is clearly recognisable as one of Saraceno’s signature tetrahedral solar balloons.

These black balloons have been plying the skies since 2007. They are mascots of the artist’s multi-stranded effort to combine engineering, architecture and the natural sciences to create a new, democratic kind of environmental art, made of bubbles and aerial platforms and webs. An art that mitigates climate change, he says, and makes the sky habitable, by establishing a modular, transnational settlement in the skies through solar balloons that require no fuel at all. An art that ushers in utopia.

Could it be that this chap is just playing about with balloons? Trying to calculate Saraceno’s level of seriousness is half the fun. Over lunch, for instance, he tells me that he wants to return us “to a sort of Mayan sensitivity towards celestial mechanics”.

But some of his efforts are admirably practical. The balloon I’d just seen being demonstrated was an Aerocene Explorer: it comes in a backpack complete with instructions on how to create and fly lightweight sensors. Any data collected can be uploaded and shared with Aerocene’s online community, via a website where participants from all over the world are sharing their experiments and innovations.

Practicalities aside, much of Saraceno’s work is simply beautiful. For a show opening at the Palais de Tokyo in Paris on 17 October, the team is busy building playful orreries, mechanical models of the solar system that combine planetary orbits with the physics of soap bubbles and webs spun by his pet Cyrtophora citricola spiders.

These unbelievably delicate confections will be on show with some mirrored umbrellas that also double as solar cookers. When arranged in concentric circles, Saraceno imagines that in the manner of a solar thermal power plant, the umbrellas might even concentrate enough heat to inflate a large balloon. He hopes to try out the idea when Audemars Piguet – a Swiss watch manufacturer that has recent form in backing innovative science-inflected art – takes parts of his sprawling Aerocene endeavour to Miami this December for the Art Basel fair.

Meanwhile, there are myriad things to organise for Paris: workshops, concerts, public symposiums uniting scientific institutions, researchers, activists, local communities, musicians and philosophers. As he says: “People aren’t very interested in simple ideas. You have to give things a little bit of complication to get the audience to engage.”

balloons

He found this out the moment he started using solar balloons. The balloons, which work by simply zipping up some air in a heat-absorbing bag, have been around since the 1970s. His own projects have demonstrated their usefulness in meteorology, pollution monitoring, even passenger transport. In 2015, he flew in a tethered solar balloon over the dunes of White Sands in New Mexico, where the US launched its first rockets and where the world’s first tourist spaceport is located. The Massachusetts Institute of Technology got in on the act, and created technology so that you can use the Aerocene.org website to plan a meteorologically feasible journey, by balloon, from Point A to Point B, anywhere on Earth.

“Rats saved at the point of giving up fought for life 240 times longer when returned to danger”

Here’s the paradox. Saraceno’s work has always been playful, and part of the game, he explains, has been “trying to sell this work as some sort of global solution to something”. But while his visions of an airborne utopia remain as remote as ever, his Aerocene project has spawned a foundation that uses lightweight balloons for climate activism and pollution monitoring. And even the absurd spectacle of someone jetting from country to country to fly fuel-less balloons has become part of the art, as Saraceno’s studio begins to record his own carbon footprint.

Saraceno makes an important point about how we address climate change in our lives. The trick, he says, is not to let the perfect get in the way of the good. Escapism is fine. He has no time for the way so many artists and pundits are ringing humanity’s death knell. He has a special contempt for the lazy way the word Anthropocene crops up now in every climate conversation, as if, with the advent of this putative new era, our doom was sealed. “What a great way for a small number of people to disempower and demotivate us,” he says.

Given the seriousness of our environmental bind, isn’t escapism a bit irresponsible? Saraceno points me to a 1957 paper by psychobiologist Curt Richter. His gruesome experiments left rats to drown in water-filled containers from which they could not escape. But if he briefly rescued rats at the point they gave up swimming, and then returned them to the water, those rats continued to fight for life 240 times longer. Richter concluded that they had learned that there was hope. Faced with challenges on a planetary scale, we are scrambling for our lives, and can see no way out. “We need the energy those rats got when they saw some small hope,” says Saraceno.

I hadn’t expected our conversation to take this dark turn, but creating such small glimmers of hope is his business. If he is a joker, then he is one in the best sense of the word.

Should we take Saraceno’s work seriously? I was doubtful, but now I think, why look a gift horse in the mouth? He enthuses people. He gets us thinking. And he is right: a little hope goes a long way.

Pierre Huyghe: Digital canvases and mind-reading machines

Visiting UUmwelt, Pierre Huyghe’s show at London’s Serpentine Gallery, for the Financial Times, 4 October 2018

On paper, Pierre Huyghe’s new exhibition at the Serpentine Gallery in London is a rather Spartan effort. Gone are the fictional characters, the films, the drawings; the collaborative manga flim-flam of No Ghost Just a Shell; the nested, we’re not-in-Kansas-any-more fictions, meta-fictions and crypto-documentaries of Streamside Day Follies. In place of Huyghe’s usual stage blarney come five large LED screens. Each displays a picture that, as we watch, shivers through countless mutations, teetering between snapshot clarity and monumental abstraction. One display is meaty; another, vaguely nautical. A third occupies a discomforting interzone between elephant and milk bottle.

Huyghe has not abandoned all his old habits. There are smells (suggesting animal and machine worlds), sounds (derived from brain-scan data, but which sound oddly domestic: was that not a knife-drawer being tidied?) and a great many flies. Their random movements cause the five monumental screens to pause and stutter, and this is a canny move, because without that  arbitrary grammar, Huyghe’s barrage of visual transformations would overwhelm us, rather than excite us. There is, in short, more going on here than meets the eye. But that, of course, is true of everywhere: the show’s title nods to the notion of “Umwelt” coined by the zoologist Jacob von Uexküll in 1909, when he proposed that the significant world of an animal was the sum of  things to which it responds, the rest going by virtually unnoticed. Huyghe’s speculations about machine intelligence are bringing this story up to date.

That UUmwelt turns out to be a show of great beauty as well; that the gallery-goer emerges from this most abstruse of high-tech shows with a re-invigorated appetite for the arch-traditional business of putting paint on canvas: that the gallery-goer does all the work, yet leaves feeling exhilarated, not exploited — all this is going to require some explanation.

To begin at the beginning, then: Yukiyasu Kamitani , who works at Kyoto University in Japan, made headlines in 2012 when he fed the data from fMRI brain scans of sleeping subjects into neural networks. These computer systems eventually succeeded in capturing shadowy images of his volunteers’ dreams. Since then his lab has been teaching computers to see inside people’s heads. It’s not there yet, but there are interesting blossoms to be plucked along the way.

UUmwelt is one of these blossoms. A recursive neural net has been shown about a million pictures, alongside accompanying fMRI data gathered from a human observer. Next, the neural net has been handed some raw fMRI data, and told to recreate the picture the volunteer was looking at.

Huyghe has turned the ensuing, abstruse struggles of the Kamitani Lab’s unthinking neural net into an exhibition quite as dramatic as anything he has ever made. Only, this time, the theatrics are taking place almost entirely in our own heads. What are we looking at here? A bottle. No, an elephant, no, a Francis Bacon screaming pig, goose, skyscraper, mixer tap, steam train mole dog bat’s wing…

The closer we look, the more engaged we become, the less we are able to describe what we are seeing. (This is literally true, in fact, since visual recognition works just that little bit faster than linguistic processing.) So, as we watch these digital canvases, we are drawn into dreamlike, timeless lucidity: a state of concentration without conscious effort that sports psychologists like to call “flow”. (How the Serpentine will ever clear the gallery at the end of the day I have no idea: I for one was transfixed.)

UUmwelt, far from being a show about how machines will make artists redundant, turns out to be a machine for teaching the rest of us how to read and truly appreciate the things artists make. It exercises and strengthens that bit of us that looks beyond the normative content of images and tries to make sense of them through the study of volume, colour, light, line, and texture. Students of Mondrian, Duffy and Bacon, in particular, will lap up this show.

Remember those science-fictional devices and medicines that provide hits of concentrated education? Quantum physics in one injection! Civics in a pill! I think Huyghe may have come closer than anyone to making this silly dream a solid and compelling reality. His machines are teaching us how to read pictures, and they’re doing a good job of it, too.

Jenna Sutela: Mars in a dish

Don’t call this an AI, whatever you do. Jenna Sutela’s mentor on this project, Memo Atkin, has issued a public warning that “every time someone personifies this stuff, every time someone talks about ‘the AI’, a kitten is strangled.”

Watching Jenna Sutela’s art-video nimiia cétiï for New Scientist, 11 September 2018

Parade of the Possible

Watching the Elfwegentocht parade spool by for New Scientist, 18 July 2018

Astronaut André Kuipers has enjoyed his share of travel, and has no doubt racked up some air miles. Who better, then, to wave the start flag for a parade of futuristic vehicles?

Spooling along at a sedate 30 miles an hour down the motorway from Drachten to Leeuwarden, this year’s European Capital of Culture, they lacked a certain Mad Max flair. But that’s Friesland for you: a land of 60 languages and 128 nationalities, birthplace of Mata Hari – and when you drive through there’s nothing to see but cows.

The Parade was the festive conclusion of the Elfwegentocht: for two weeks, people have got about the region without using a drop of fossil fuel. “And now we’ve shown it’s possible,” says Bouwe de Boer, the municipality’s energy coordinator at the municipality of Leeuwarden, “we’ve shown that it is possible also for the rest of the Netherlands.”

De Boer is now project leader of Fossylfrij Fryslân, the fossil-free movement in Friesland, bringing disparate environmental campaigns and start-up technologies together to achieve real goals in tiny time frames. Electric vehicles dominate the parade but as de Boer points out, there are other ways to drive fossil-free. “Think of trucks and buses on hydrogen, cars on blue diesel, buses on green gas, Segways, bicycles, mobility scooters, go-karts…”

Go-karts? It’s a gimcrack future, this – but then, what else can the future ever be but an amalgam of new and old, complex and homespun?

The two big innovative technologies on display here aren’t actually “on display” in a physical sense. You’ll have to take my word for it that the “Blauwe Diesel” manufactured by Neste in Rotterdam from restaurant waste and residues is, indeed, satisfyingly blue. It’s a pure HVO (“hydrotreated vegetable oil” to you), low on emissions and so similar to regular diesel in the way it behaves that it requires no modifications to existing diesel engines or distribution systems. At a pump near you – assuming you live in this go-ahead region of the Netherlands – it could be the saving of an industry that some manufacturers and governments have already written off. Meanwhile Neste is trying to make its blue diesel from other sources, including old car tires, waste paper and algae.

Elsewhere in the parade, under the bonnets of a handful of electric cars, sit batteries from MG Energy Systems. These are the batteries you most often find in racing cars and speedboats, and they’re the brainchild of local entrepreneurs Gerard van der Schaar and Mark Scholten, whose first project, back in 2006, was a vessel to compete in the world’s first solar boat race (another de Boer initiative).

They quickly discovered that batteries were the boats’ Achilles’ heel. There was simply no good battery management system available. A little over a decade later their products power the Furia solar boat, which has finished first in just about every international solar boat event; Solarwave 62, the first hybrid yacht with electric propulsion to cross the Atlantic Ocean; the Elektra One Solar, the first electric-and-solar aircraft to fly over the Alps; and the Nuna7 car, winner of the World Solar Challenge in 2013, having achieved an average speed of 90.71 kmh for over 30 hours.

De Boer is proud of his region’s achievements but he has his eye on the bigger picture, too. As of 27 June the Netherlands has set in train one of the world’s most ambitious climate laws, which if it’s finalised in 2019, will set a target to reduce greenhouse gas emissions by 95 per cent by 2050, with the introduction of a carbon-neutral electricity system. (The UK’s mandated 2050 emissions target is an 80 per cent reduction. Sweden and Norway are set to go carbon neutral by 2045 and 2050 respectively)

De Boer talks excitedly about Friesland’s circular energy economy. Cleaning up waste water in this region generates methane which is being harvested to boost biogas production. He talks excitedly about advances in renewable energy. Solar panels power MG’s entire battery factory. He talks excitedly about everything, quite frankly. But it’s an incidental detail which captures my attention: fruit, I am told, of another one of de Boer’s endless stream of friendly, chivvying phone-calls.

The police looking after the parade are riding electric bikes.

Technology vs observation

Losing my rag at the Royal Academy for New Scientist, 13 December 2017

When the schools of London’s Royal Academy of Arts were opened in 1769, life drawing — the business of sketching either live models or the plaster casts of worthy sculptures — was an essential component of an artist’s training.

As I wandered around From Life, an exhibition devoted to the history as well as the future of the practice, I overheard a curator explaining that, now life drawing is no longer obligatory in Royal Academy art courses, a new generation of artists are approaching the practice in a “more expressive” way. The show’s press release claims even more: that life drawing is evolving “as technology opens up new ways of creating and visualising artwork”.

There was little of this in evidence when I visited, however: two-and-a-half of the three virtual-reality experiences on offer had broken down. Things break down when the press turns up – you might even say it’s a rule. Still, given their ubiquity, I’m beginning to wonder whether gallery-based VR malfunctions are not a kind of mischievous artwork in their own right. In place of a virtual sketch, a message in an over-friendly font asks: “Have you checked your internet connection?” At least Swiss artist Jean Tinguely’s wild mobiles of the 1960s had the decency to catch fire.

How can new technologies like Google’s Tilt Brush and HTC’s Vive VR platform bring artists into a more intimate relation to their subject — more intimate than might be achieved by, say, standing a metre away from a naked stranger armed only with a bit of charcoal?

Jonathan Yeo has had a stab at the problem, using Tilt Brush’s 3D painting tech to fashion a sculptural self-portrait. The outsize bronze 3D print of his effort — an assemblage of short, wide, hesitant virtual “brushstrokes” — has a curiously dated feel and wouldn’t look out of place in a group retrospective of 20th-century British sculpture. As an advert for a technology that prides itself on its expressivity (videos of the platform at work usually resemble explosions in a paint factory), it’s a curiously laborious piece.

On a nearby wall hang Gillian Wearing’s photographic self-portraits, manipulated using the sort of age-progression technology employed by forensic artists. In this way, Wearing has captured her appearance 10, 20, 30 years into her future. It’s an undeniably moving display, and undeniably off the point: life drawing is about capturing the present moment, which leaves Wearing’s contribution resembling those terms and conditions that appear at the bottom of TV advertisements – Other Moments Are Available.

Yinka Shonibare (best known for his ship-in-a-bottle sculpture on the fourth plinth of Trafalgar Square) comments on the show, rather than contributes to it, with a 3D VR conceptualisation of a painting by the 18th-century Scots artist and dealer Gavin Hamilton.

Hamilton once sold a Roman sculpture to a collector. Shonibare has scanned a plaster cast of this Townley Venus, then placed it on a plinth in a largely imaginary VR garden (you catch only a glimpse of this space in Hamilton’s painting). He has covered its plaster-white surface with batik designs (referring to common sub-Saharan African fabric, though it was originally a Dutch export) and as a coup de grâce, he has stuck a globe on Venus’s torso in place of her head. The point is that we can never copy something without to some degree appropriating it. Whether you like what he’s done will depend on whether you like art that makes a primarily intellectual point.

In a gallery environment increasingly besotted by (and bested by) technology, such acts of cultural orienteering may be necessary; they’re certainly inevitable. The new work gracing From Life at least attempts to address the theme of the show, and its several failures are honest and interesting.

Still, I keep coming back to the historical half of the exhibition — to the casts, the drawings, the portraits of struggling young artists from 1769 to now. Life drawing is not obligatory for artists? It should be obligatory for everyone. If we never learn to observe honestly, what the devil will we ever have to be expressive about?

It’s coming at you!

OUTSIDE Dimension Studios in Wimbledon, south London, is one of those tiny wood-framed snack bars that served commercial travellers in the days before motorways. The hut is guarded by old shop dummies dressed in fishnet tights and pirate hats. If the UK made its own dilapidated version of Westworld, the cyborg rebellion would surely begin here.

Steve Jelley orders us breakfast. Years ago he left film production to pursue a career developing new media. He’s of the generation for whom the next big thing is always just around the corner. Most of them perished in the dot-com bust of 2001, but Jelley clung to the dream, and now Microsoft has come calling.

His company, Hammerhead, makes 360-degree videos for commercial clients. Its partner in this current venture, Timeslice Films, is best known for volumetric capture of still images – the business of cinematographically recording forms in three dimensions – a practice that goes back to founder Tim MacMillan’s art-school experiments of the early 1980s.

Steve Sullivan, director of the Holographic Video initiative at Microsoft, is fusing both companies’ technical expertise to create volumetric video: immersive entertainment that’s indistinguishable from reality.

There are only three studios in the world that can do this with any degree of conviction, and Wimbledon is the only one outside the US. Still, I’m sceptical. It has been clear for a while that truly immersive media won’t spring from a single “light-bulb” moment. The technologies involved are, in conceptual terms, surprisingly old. Volumetric capture is a good example.

MacMillan is considered the godfather of this tech, having invented the “bullet time” effect central to The Matrix. But The Matrix is 18 years old, and besides, MacMillan reckons that pioneer photographer Eadweard Muybridge got to the idea years before him – in fact, decades before cinema was invented.

“Engineer Masahiro Mori says his ‘uncanny valley’ idea was never meant to be taken scientifically”

Then there’s motion capture (or mocap): recording the movement of points attached to an actor, and from those points, constructing the performance of a three-dimensional model. The pioneering Soviet physiologist Nikolai Bernstein invented the technique in the early 1920s, while developing training programmes for factory workers.

Truly immersive media will be achieved not through magic bullets, but through thugging – the application of ever more computer power, and the ever-faster processing of more and more data points. Impressive, but where’s the breakthrough?

“Well,” Jelley begins, handing me what may be the largest bacon sandwich in London, “you know this business of the ‘uncanny valley’…?” My heart sinks slightly.

Most New Scientist readers will be familiar with Masahiro Mori’s concept of the uncanny valley. It’s a curiously anglophone obsession. In the 30 years since the Japanese engineer published his paper in 1970, it has been referred to in Japanese academic literature only once. Mori himself says the idea was never meant to be taken scientifically. He was merely warning robot designers at a time when humanoid robots didn’t exist that the closer their works came to resemble people, the creepier we would find them.

In the West, discussions of the uncanny valley have grown to a sizeable cottage industry. There have been expensive studies done with PET scans to prove the existence of the effect. But as Mori commented in an interview in 2012: “I think that the brainwaves act that way because we feel eerie. It still doesn’t explain why we feel eerie to begin with.”

Our discomfort extends beyond encounters with physical robots to include some cinematic experiences. Many are the animated movies that have employed mocap to achieve something like cinematic realism, only to plummet without trace into the valley.

Elsewhere, actor Andy Serkis famously uses mocap to transform himself into characters like Gollum in The Lord of the Rings, or the chimpanzee Caesar in Rise of the Planet of the Apes, and we are carried along well enough by these films. The one creature this technology can’t emulate, however, is Serkis himself. Though mocap now renders human body movement with impressive realism, the human face remains a machine far too complex to be seamlessly emulated even by the best system.

Jelley reckons he and his partners have “solved the problem” of the uncanny valley. He leads me into the studio. There’s a small, circular, curtained-off area – a sort of human-scale birdcage. Rings of lights and cameras are mounted on scaffolds and hang from a moveable and very heavy-looking ceiling rig.

There are 106 cameras: half of them recording in the infrared spectrum to capture depth information, half of them recording visible light. Plus, a number of ultraviolet cameras. “We use ultraviolet paint to mask areas for effects work,” Jelley explains, “so we record the UV spectrum, too. Basically we use every glimmer of light we can get short of asking you to swallow radium.”

The cameras shoot between 30 and 60 times a second. “We have a directional map of the configuration of those cameras, and we overlay that with a depth map that we’ve captured from the IR cameras. Then we can do all the pixel interpolation.”

This is a big step up from mocap. Volumetric video captures real-time depth information from surfaces themselves: there are no fluorescent sticky dots or sliced-through ping-pong balls attached to actors here. As far as the audience is concerned, volumetric video is essentially just that, video, and as close to a true record as anything piped through a basement full of computers is ever going to get.

So what kind of films are made in such studios? Right now, the education company Pearson is creating virtual consultations for trainee nurses. Fashion brands and car companies have shot adverts here. TV companies want to use them for fully immersive and interactive dramas.

“I know she’s not real, but my body doesn’t. Every bit of me has fallen for this super-real gymnast”

On a table nearby, a demo is ready to watch on a Vive VR headset. There are three sets of performances for me to observe, all looping in a grey, gridded, unadorned virtual space: the digital future as a filing cabinet. There are two experiments from Sullivan’s early days at Microsoft. Thomas Jefferson is pure animatronic; the two Maori haka dancers are engaging, if unhuman. The circus gymnast swinging on her hoop is different. I recognise her, or think I do. My body-language must be giving the game away, because Jelley laughs.

“Go up to her,” he says. I can’t place where I’ve seen her before. I try and catch her eye. “Closer.”

I’m invading her space, and I’m not comfortable with this. I can see the individual threads, securing the sequins to her costume. More than that: I can smell her. I can feel the heat coming from her skin.

I know she’s not real, but my body doesn’t. Every bit of me that might have rejected a digitised face as uncanny has fallen hook, line and sinker for this super-real gymnast. And this, presumably, is why the bit of my mind that enables me to communicate freely and easily with my fellow humans is in overdrive, trying to plug the gaps in my experience, as if to say, “Of course her skin is hot. Of course she has a scent.”

Mori’s uncanny valley effect is not quantifiable, and I don’t suppose my experience is any more measurable than the one Mori identified. But I’d bet the farm that, had you scanned me, you would have seen all manner of pretty lights. This hasn’t been an eerie experience. Quite the reverse. It’s terrifyingly ordinary. Almost, I might say, human.

Jelley walks me back to the main road. Neither of us says a word. He knows what he has. He knows what he has done.

Outside the snack shack, three shop dummies in pirate gear wobble in the wind.

The genius of making a little go a long way

Visiting Illuminating India at London’s Science Museum for New Scientist, 10 October 2017

One can taste the boosterism in the air at London’s Science Museum as it introduces its two-gallery exhibition, Illuminating India.

There is a cafe serving excellent Indian street food. Someone next to me used the word “Commonwealth” without irony. Would there have been such a spirit without Brexit? Probably not: this is a show about the genius of another country that very much wants to project Britain’s own global aspirations. Any historian of Anglo-British relations will give a sardonic smile at this.

When you visit (and you should), try to look around the smaller, artefacts-driven gallery first.

This room tells the stories of Indian science – stories plural because there can never be one, linear account of how such dissimilar and contesting cultures struggled and more or less succeeded in understanding and exploiting a space of such extraordinary complexity.

Naturally, since India has a past to boast of, pride of place goes to its indigenous cultures. It was the Indus valley civilisation, after all, whose peoples fashioned standardised weights around 4000 years ago: items that indicate high levels of arithmetical literacy, communication and trade.

And there are reconstructions of Ayurvedic surgical instruments described in records dating back to around 500 BC. Also on show is a 1800-year-old document containing the first example of the use of zero. Wonderfully, radiocarbon dating pushed the document’s age back by 500 years just before the exhibition opened.

It is a measure of the wisdom of the curators that such an illustrious past isn’t allowed to overshadow India’s more recent achievements. For example, Jagadish Chandra Bose’s early-20th-century crescograph, designed to observe plant growth at a magnification of 10,000 times, reminds us why he is often called the father of modern Indian science.

Another winning object is Chandrasekhara Raman’s spectrometer. Raman was the first Indian to win a Nobel prize, for physics, in 1930.

And what of that other great empire far to the north? Well, there is a map of George Everest’s career-defining Great Trigonometrical Survey of India – the teamwork of 70 years distilled on a single, meticulously drawn map. And nearby are details of a recent collaboration between Surrey Satellite Technology in the UK and the Indian Space Research Organisation on the Earth-surveying NovaSAR satellite.

Some of the deeper, darker questions about Anglo-Indian relations are posed in the second, photographic half of the exhibition.

There, the anthropometric photographs of Maurice Portman make a depressingly silly impression next to the respectful, revealing and entirely unlicentious photographs Ram Singh took of the women of his own harem: powerful political players all, who shaped the country through marriage and allied treaties.

It is hard to say why the split nature of Illuminating India works as well as it does. It has something to do with the way the rooms handle political power.

India’s science, from its ancient stepwells that gathered monsoon waters to the bureaucratic and algorithmic marvel that is today’s tiffin tin-based lunch delivery system, has been driven by the complex needs of a massive population making a living.

Similarly, its doing-more-with-less style of innovation is reflected in everything from the world’s cheapest artificial leg (the Jaipur leg, made of rubber, plastic and wood) to the world’s cheapest Mars-orbiting camera.

Visitors to Illuminating India will leave thinking that technology may, after all, be making the world a better place, and that what people do is ultimately more influential than who they are.

Stanisław Lem: The man with the future inside him

lem

From the 1950s, science fiction writer Stanisław Lem began firing out prescient explorations of our present and far beyond. His vision is proving unparalleled.
For New Scientist, 16 November 2016

“POSTED everywhere on street corners, the idiot irresponsibles twitter supersonic approval, repeating slogans, giggling, dancing…” So it goes in William Burroughs’s novel The Soft Machine (1961). Did he predict social media? If so, he joins a large and mostly deplorable crowd of lucky guessers. Did you know that in Robert Heinlein’s 1948 story Space Cadet, he invented microwave food? Do you care?

There’s more to futurology than guesswork, of course, and not all predictions are facile. Writing in the 1950s, Ray Bradbury predicted earbud headphones and elevator muzak, and foresaw the creeping eeriness of today’s media-saturated shopping mall culture. But even Bradbury’s guesses – almost everyone’s guesses, in fact – tended to exaggerate the contemporary moment. More TV! More suburbia! Videophones and cars with no need of roads. The powerful, topical visions of writers like Frederik Pohl and Arthur C. Clarke are visions of what the world would be like if the 1950s (the 1960s, the 1970s…) went on forever.

And that is why Stanisław Lem, the Polish satirist, essayist, science fiction writer and futurologist, had no time for them. “Meaningful prediction,” he wrote, “does not lie in serving up the present larded with startling improvements or revelations in lieu of the future.” He wanted more: to grasp the human adventure in all its promise, tragedy and grandeur. He devised whole new chapters to the human story, not happy endings.

And, as far as I can tell, Lem got everything – everything – right. Less than a year before Russia and the US played their game of nuclear chicken over Cuba, he nailed the rational madness of cold-war policy in his book Memoirs Found in a Bathtub (1961). And while his contemporaries were churning out dystopias in the Orwellian mould, supposing that information would be tightly controlled in the future, Lem was conjuring with the internet (which did not then exist), and imagining futures in which important facts are carried away on a flood of falsehoods, and our civic freedoms along with them. Twenty years before the term “virtual reality” appeared, Lem was already writing about its likely educational and cultural effects. He also coined a better name for it: “phantomatics”. The books on genetic engineering passing my desk for review this year have, at best, simply reframed ethical questions Lem set out in Summa Technologiae back in 1964 (though, shockingly, the book was not translated into English until 2013). He dreamed up all the usual nanotechnological fantasies, from spider silk space-elevator cables to catastrophic “grey goo”, decades before they entered the public consciousness. He wrote about the technological singularity – the idea that artificial superintelligence would spark runaway technological growth – before Gordon Moore had even had the chance to cook up his “law” about the exponential growth of computing power. Not every prediction was serious. Lem coined the phrase “Theory of Everything”, but only so he could point at it and laugh.

He was born on 12 September 1921 in Lwów, Poland (now Lviv in Ukraine). His abiding concern was the way people use reason as a white stick as they steer blindly through a world dominated by chance and accident. This perspective was acquired early, while he was being pressed up against a wall by the muzzle of a Nazi machine gun – just one of several narrow escapes. “The difference between life and death depended upon… whether one went to visit a friend at 1 o’clock or 20 minutes later,” he recalled.

Though a keen engineer and inventor – in school he dreamed up the differential gear and was disappointed to find it already existed – Lem’s true gift lay in understanding systems. His finest childhood invention was a complete state bureaucracy, with internal passports and an impenetrable central office.

He found the world he had been born into absurd enough to power his first novel (Hospital of the Transfiguration, 1955), and might never have turned to science fiction had he not needed to leap heavily into metaphor to evade the attentions of Stalin’s literary censors. He did not become really productive until 1956, when Poland enjoyed a post-Stalinist thaw, and in the 12 years following he wrote 17 books, among them Solaris (1961), the work for which he is best known by English speakers.

Solaris is the story of a team of distraught experts in orbit around an inscrutable and apparently sentient planet, trying to come to terms with its cruel gift-giving (it insists on “resurrecting” their dead). Solaris reflects Lem’s pessimistic attitude to the search for extraterrestrial intelligence. It’s not that alien intelligences aren’t out there, Lem says, because they almost certainly are. But they won’t be our sort of intelligences. In the struggle for control over their environment they may as easily have chosen to ignore communication as respond to it; they might have decided to live in a fantastical simulation rather than take their chances any longer in the physical realm; they may have solved the problems of their existence to the point at which they can dispense with intelligence entirely; they may be stoned out of their heads. And so on ad infinitum. Because the universe is so much bigger than all of us, no matter how rigorously we test our vaunted gift of reason against it, that reason is still something we made – an artefact, a crutch. As Lem made explicit in one of his last novels, Fiasco (1986), extraterrestrial versions of reason and reasonableness may look very different to our own.

Lem understood the importance of history as no other futurologist ever has. What has been learned cannot be unlearned; certain paths, once taken, cannot be retraced. Working in the chill of the cold war, Lem feared that our violent and genocidal impulses are historically constant, while our technical capacity for destruction will only grow.

Should we find a way to survive our own urge to destruction, the challenge will be to handle our success. The more complex the social machine, the more prone it will be to malfunction. In his hard-boiled postmodern detective story The Chain of Chance (1975), Lem imagines a very near future that is crossing the brink of complexity, beyond which forms of government begin to look increasingly impotent (and yes, if we’re still counting, it’s here that he makes yet another on-the-money prediction by describing the marriage of instantly accessible media and global terrorism).

Say we make it. Say we become the masters of the universe, able to shape the material world at will: what then? Eventually, our technology will take over completely from slow-moving natural selection, allowing us to re-engineer our planet and our bodies. We will no longer need to borrow from nature, and will no longer feel any need to copy it.

At the extreme limit of his futurological vision, Lem imagines us abandoning the attempt to understand our current reality in favour of building an entirely new one. Yet even then we will live in thrall to the contingencies of history and accident. In Lem’s “review” of the fictitious Professor Dobb’s book Non Serviam, Dobb, the creator, may be forced to destroy the artificial universe he has created – one full of life, beauty and intelligence – because his university can no longer afford the electricity bills. Let’s hope we’re not living in such a simulation.

Most futurologists are secret utopians: they want history to end. They want time to come to a stop; to author a happy ending. Lem was better than that. He wanted to see what was next, and what would come after that, and after that, a thousand, ten thousand years into the future. Having felt its sharp end, he knew that history was real, that the cause of problems is solutions, and that there is no perfect world, neither in our past nor in our future, assuming that we have one.

By the time he died in 2006, this acerbic, difficult, impatient writer who gave no quarter to anyone – least of all his readers – had sold close to 40 million books in more than 40 languages, and earned praise from futurologists such as Alvin Toffler of Future Shock fame, scientists from Carl Sagan to Douglas Hofstadter, and philosophers from Daniel Dennett to Nicholas Rescher.

“Our situation, I would say,” Lem once wrote, “is analogous to that of a savage who, having discovered the catapult, thought that he was already close to space travel.” Be realistic, is what this most fantastical of writers advises us. Be patient. Be as smart as you can possibly be. It’s a big world out there, and you have barely begun.

 

When art and technology pull each other to bits

ars-electronica

Visiting Ars Electronica for New Scientist, 21 September 2016

In a disused mail sorting office in Linz, Austria, an industrial robot twice my height has got hold of Serbian-born artist Dragan Ilic and is wiping him over a canvas-covered wall. Ilic is clutching pencils, and as the robot twirls and dabs him against the wall, the artist makes his own frantic marks – a sort of sentient brush head. These performances are billed as a collaboration between artist and machine. All I see is the user getting used.

Every September in Linz, the Ars Electronica festival tries to marry technology and art. Andy Warhol took up screen-printing in the 1960s, and a whole generation of gallery-goers have since grown up with the notion that this match is easy.

Indeed, the coupling of art and technology has become a solid pillar of art education, especially now that so much funding comes from IT big hitters such as Sony and private institutions like the Wellcome Trust.

But if the venerable Ars Electronica has demonstrated anything in its 37-year history – beyond the ability of the arts to remake and rejuvenate a city – it is that technology and art make astonishingly unhappy bedfellows.

This year, for example, Swiss artist Daniel Boschung used an industrial robot controlled by bespoke software to take 900-million-pixel portraits of people – forensic surveys so detailed that they drained all emotion from their subjects’ faces.

Not far away, another robot, Davide Quayola’s Sculpture Factory, chiselled through partially completed life-size stone renditions of Michelangelo’s David. The attention was directed to the pixelated nature of 3D scanning, which smeared, spread and tessellated the biblical giant-killer to the point of incoherence: here a limb, there an eye.

Walking the corridor towards the current sculpture-in-progress, one passed attempt after attempt of clone Davids peeking in more or less agonised fashion from their cuboid stone prisons, in a sort of mineral retread of that infamous scene from Alien: Resurrection.

The provoking thing about Ars Electronica is that it jams together boutique displays of the latest technology, trenchant criticisms of the post-industrial project, jokes and honest failures. It is a gargantuan vessel powered by enthusiasm, but steered by nothing remotely resembling taste.

Eventually, the visitor comes to rest against one of the more obvious wins. Boris Labbé’s film Rhizome, which netted the festival’s annual animation prize, unites watercolour illustration and digital effects to tell an epic tale of evolution, civilisation and cosmic transformation in one steady, heart-stopping reverse zoom.

And then there is Frank Kolkman’s OpenSurgery, which combines 3D printing and laser-cutting with hacked surgical pieces and components bought online. This is a DIY robot that can perform laparoscopic surgery – a terrifying comment on the way that hacking and “making” are increasingly expected to stand in for the real thing.

You’ll need a drink after all that, so head for Max Dovey’s A Hipster Bar. And good luck – this genuine pop-up drinking hole will, in true neo-liberal fashion, keep the gate shut unless its face-recognition system considers you “90 per cent hipster”.

As you sip, ponder this: it was the assertion of the Romantic movement that art makes us appreciate the beauty, richness and sheer size of the world. And technology, used appropriately, brings us closer to that sublime. As the Romantics’ acolyte, the writer and pioneering pilot Antoine de Saint-Exupéry, put it: “The machine does not isolate man from the great problems of nature but plunges him more deeply into them.”

Even if that was true in 1939, it’s not true now: not now our drones do our flying for us; not now our technology has got away from us to the point where large portions of nature are being erased; not now we live mired in media and, indeed, have to make special efforts to escape it.

Naturally, artists want to explore the new technology of their day, but these days the best results seem to come when we misappropriate the machines and kick them into new shapes, or simply point and laugh.