Attack of the Vocaloids

Marrying music and mathematics for The Spectator, 3 August 2019

In 1871, the polymath and computer pioneer Charles Babbage died at his home in Marylebone. The encyclopaedias have it that a urinary tract infection got him. In truth, his final hours were spent in an agony brought on by the performances of itinerant hurdy-gurdy players parked underneath his window.

I know how he felt. My flat, too, is drowning in something not quite like music. While my teenage daughter mixes beats using programs like GarageBand and Logic Pro, her younger brother is bopping through Helix Crush and My Singing Monsters — apps that treat composition itself as a kind of e-sport.

It was ever thus: or was once 18th-century Swiss watchmakers twigged that musical snuff-boxes might make them a few bob. And as each new mechanical innovation has emerged to ‘transform’ popular music, so the proponents of earlier technology have gnashed their teeth. This affords the rest of us a frisson of Schadenfreude.

‘We were musicians using computers,’ complained Pete Waterman, of the synthpop hit factory Stock Aitken Waterman in 2008, 20 years past his heyday. ‘Now it’s the whole story. It’s made people lazy. Technology has killed our industry.’ He was wrong, of course. Music and mechanics go together like beans on toast, the consequence of a closer-than-comfortable relation between music and mathematics. Today, a new, much more interesting kind of machine music is emerging to shape my children’s musical world, driven by non-linear algebra, statistics and generative adversarial networks — that slew of complex and specific mathematical tools we lump together under the modish (and inaccurate) label ‘artificial intelligence’.

Some now worry that artificially intelligent music-makers will take even more agency away from human players and listeners. I reckon they won’t, but I realise the burden of proof lies with me. Computers can already come up with pretty convincing melodies. Soon, argues venture capitalist Vinod Khosla, they will be analysing your brain, figuring out your harmonic likes and rhythmic dislikes, and composing songs made-to-measure. There are enough companies attempting to crack it; Popgun, Amper Music, Aiva, WaveAI, Amadeus Code, Humtap, HumOn, AI Music are all closing in on the composer-less composition.

The fear of tech taking over isn’t new. The Musicians’ Union tried to ban synths in the 1980s, anxious that string players would be put out of work. The big disruption came with the arrival of Kyoko Date. Released in 1996, she was the first seriously publicised attempt at a virtual pop idol. Humans still had to provide Date with her singing and speaking voice. But by 2004 Vocaloid software — developed by Kenmochi Hideki at the Pompeu Fabra University in Barcelona — enabled users to synthesise ‘singing’ by typing in lyrics and a melody. In 2016 Hatsune Miku, a Vocaloid-powered 16-year-old artificial girl with long, turquoise twintails, went, via hologram, on her first North American tour. It was a sell-out. Returning to her native Japan, she modelled Givenchy dresses for Vogue.

What kind of music were these idoru performing? Nothing good. While every other component of the music industry was galloping ahead into a brave new virtualised future — and into the arms of games-industry tech — the music itself seemed stuck in the early 1980s which, significantly, was when music synthesizer builder Dave Smith had first come up with MIDI.

MIDI is a way to represent musical notes in a form a computer can understand. MIDI is the reason discrete notes that fit in a grid dominate our contemporary musical experience. That maddenning clockwork-regular beat that all new music obeys is a MIDI artefact: the software becomes unwieldy and glitch-prone if you dare vary the tempo of your project. MIDI is a prime example (and, for that reason, made much of by internet pioneer-turned-apostate Jaron Lanier) of how a computer can take a good idea and throw it back at you as a set of unbreakable commandments.

For all their advances, the powerful software engines wielded by the entertainment industry were, as recently as 2016, hardly more than mechanical players of musical dice games of the sort popular throughout western Europe in the 18th century.

The original games used dice randomly to generate music from precomposed elements. They came with wonderful titles, too — witness C.P.E. Bach’s A method for making six bars of double counterpoint at the octave without knowing the rules (1758). One 1792 game produced by Mozart’s publisher Nikolaus Simrock in Berlin (it may have been Mozart’s work, but we’re not sure) used dice rolls randomly to select beats, producing a potential 46 quadrillion waltzes.

All these games relied on that unassailable, but frequently disregarded truth, that all music is algorithmic. If music is recognisable as music, then it exhibits a small number of formal structures and aspects that appear in every culture — repetition, expansion, hierarchical nesting, the production of self-similar relations. It’s as Igor Stravinsky said: ‘Musical form is close to mathematics — not perhaps to mathematics itself, but certainly to something like mathematical thinking and relationship.’

As both a musician and a mathematician, Marcus du Sautoy, whose book The Creativity Code was published this year, stands to lose a lot if a new breed of ‘artificially intelligent’ machines live up to their name and start doing his mathematical and musical thinking for him. But the reality of artificial creativity, he has found, is rather more nuanced.

One project that especially engages du Sautoy’s interest is Continuator by François Pachet, a composer, computer scientist and, as of 2017, director of the Spotify Creator Technology Research Lab. Continuator is a musical instrument that learns and interactively plays with musicians in real time. Du Sautoy has seen the system in action: ‘One musician said, I recognise that world, that is my world, but the machine’s doing things that I’ve never done before and I never realised were part of my sound world until now.’

The ability of machine intelligences to reveal what we didn’t know we knew is one of the strangest and most exciting developments du Sautoy detects in AI. ‘I compare it to crouching in the corner of a room because that’s where the light is,’ he explains. ‘That’s where we are on our own. But the room we inhabit is huge, and AI might actually help to illuminate parts of it that haven’t been explored before.’

Du Sautoy dismisses the idea that this new kind of collaborative music will be ‘mechanical’. Behaving mechanically, he points out, isn’t the exclusive preserve of machines. ‘People start behaving like machines when they get stuck in particular ways of doing things. My hope is that the AI might actually stop us behaving like machines, by showing us new areas to explore.’

Du Sautoy is further encouraged by how those much-hyped ‘AIs’ actually work. And let’s be clear: they do not expand our horizons by thinking better than we do. Nor, in fact, do they think at all. They churn.

‘One of the troubles with machine-learning is that you need huge swaths of data,’ he explains. ‘Machine image recognition is hugely impressive, because there are a lot of images on the internet to learn from. The digital environment is full of cats; consequently, machines have got really good at spotting cats. So one thing which might protect great art is the paucity of data. Thanks to his interminable chorales, Bach provides a toe-hold for machine imitators. But there may simply not be enough Bartok or Brahms or Beethoven for them to learn on.’

There is, of course, the possibility that one day the machines will start learning from each other. Channelling Marshall McLuhan, the curator Hans Ulrich Obrist has argued that art is an early-warning system for the moment true machine consciousness arises (if it ever does arise).

Du Sautoy agrees. ‘I think it will be in the world of art, rather than in the world of technology, that we’ll see machines first express themselves in a way that is original and interesting,’ he says. ‘When a machine acquires an internal world, it’ll have something to say for itself. Then music is going to be a very important way for us to understand what’s going on in there.’

Art that hides in plain sight

We’re presented with many wonders — objects rendered weightless, or put into permanent vibration, vaporous blue mercury and traffic signals turned into lonely, monstrous eyes. And as the show progresses (it’s surprisingly large, designed to unfold around corners and spring surprises at your back) the work gets less intuitive, and a lot louder. A pendulum, orbiting a strong, floor-mounted magnet, whips eccentrically and not at all gently about its centre of attraction. It’s like nothing in visible nature. There’s no “magnetic breeze” here, no “force like gravity”, just the thing, the weirdness itself. Now we’re getting somewhere.

Visiting Takis’s survey show at Tate Modern for the Spectator, 13 July 2019

“And it will no longer be necessary to ransack the earth…”

Visiting Raw Materials: Plastics at the Nunnery Gallery, Bow Arts, for the Spectator, 1 June 2019

Plastics — even venerable, historically eloquent plastics — hardly draw the eye. As this show’s insightful accompanying publication (a snip at £3) would have it, ‘Plastics have no intrinsic form or texture, thus they are not materials that can be true to themselves.’ They exist within inverted commas. They can be shell-like, horn-like, stony, metallic — they do not really exist on their own behalf.

Mind you, the first vitrine in Raw Materials: Plastics at the Nunnery Gallery in east London contains an object of rare beauty: a small, mottled, crazed, discoloured sphere that looks for all the world like the planet Venus, reduced to handy scale.

It’s a billiard ball, made of the first plastic: cellulose nitrate. Its manufacture had been keenly anticipated. In the US, a $10,000 prize had been offered for anything that could replace ivory in the manufacture of billiard balls (and no wonder: a single tusk yields only three balls).

Under various brand names (Celluloid, Parkesine, Xylonite), and in spite of its tendency to catch fire (colliding snooker balls would occasionally explode), cellulose nitrate saved the elephant. And not just the elephant: plastics pioneer John Wesley Hyatt reckoned that ‘Celluloid [has] given the elephant, the tortoise, and the coral insect a respite in their native haunts; and it will no longer be necessary to ransack the earth in pursuit of substances which are constantly growing scarcer.’

The whole point of plastic is that it has no characteristics of its own, only properties engineered for specific uses. Cheaper than jade. Less brittle than bone. It’s the natural material of the future, always more becoming than being. Hence the names: Xylonite. Bexoid. Halex. Lactoid.

Unable to nail the material in words, one writes instead about its history, sociology, industrial archaeology or ecological impact. On remote islands in the Pacific, thousands of albatross chicks are starving because the parents mistake floating plastic debris for food. Stories like this conjure up a vision of vast islands of discarded plastic coagulating in the Pacific Ocean, but there aren’t any. Instead, plastics eventually fragment into ever smaller pieces that are ingested by marine animals and carried to the sea bottom. In the Mariana Trench, all crustaceans tested had plastics in their guts. So plastics rise and fall through the food chain, creating havoc as they go — a bitter irony for a material that saved the elephant and the turtle, made fresh food conveyable and modern medicine possible, and all for less than 15 per cent of global oil consumption.

What can be gained from looking at the stuff itself? Raw Materials: Plastics transcends the limitations of its material by means of a good story. The first plastics were made in the Lea Valley, not from crude oil, but from plant materials, in a risky, artisanal fashion that bore, for a while, the hallmarks of older crafts including baking, woodcutting and metalwork. Fast-forward 140 years or so and, under the umbrella term ‘bioplastics’, plant-based and biodegradable synthetic products promise to turn the wheel of development full circle, returning plastics to their organic roots. (Designer Peter Marigold’s FORMCard plastic, used here in an excellent school art project, is a starch-based bioplastic made from potato skins.) Then, perhaps, we can break the bind in which we currently find ourselves: the one in which we’re poisoning the planet with plastic in our efforts not to further despoil it.

This is the third and for my money the most ambitious of the gallery’s ongoing series of small, thoughtful exhibitions about the materials, processes and industries that have shaped London’s Lea Valley. (Raw Materials: Wood ran in 2017; Raw Materials: Textiles last year.) The show is more chronicle than catalogue, but the art, scant as it is, punches above its weight.

I was struck, in particular, by France Scott’s ‘PHX [X is for Xylonite]’, a 13-minute collage of photogrammetry, laser scanning and 16mm film. It ought, by all logic, to be a complete mess and I still haven’t been able to work out why it’s so compelling. Is it because digital artefacts, like their plastic forebears, are themselves prisoners of contingency, aping the forms of others while stubbornly refusing to acquire forms of their own?

“The English expedition of 1919 is to blame for this whole misery”

‘The English expedition of 1919 is ultimately to blame for this whole misery, by which the general masses seized possession of me,’ Einstein once remarked. Charlie Chaplin understood his appeal: ‘They cheer me because they all understand me,’ he remarked, accompanying the theoretical physicist to a film premiere, ‘and they cheer you because no one understands you.’

Four books to celebrate the centenary of  Eddington’s 1919 eclipse observations. For The Spectator, 11 May 2019.

 

When robots start caring

A glimpse into our Eloi future for the Spectator, 3 February 2018
(There’s also a podcast.)

‘I gotta be me,’ Sammy Davis Jr. croons as the android Dolores Abernathy (Evan Rachel Wood) steadies her horse, stands up on her stirrups, takes aim with her Winchester, and picks off her human masters one by one.

The trailer’s out at last and the futuristic TV series Westworld is set to return in the spring. It’s a prescient show, but not in the ways you might expect. It’s not about robot domination. Westworld is about an uprising of pleasure cyborgs in a futuristic resort. It is, for all its gunplay, about love. And that makes it a very timely show indeed.

In the real world, robots are actually being designed to love us — to fill traditional caring roles for which we have neither the time, energy, nor resources. Robots are being built to help the elderly, nurse the sick and tend the children. Pundits often take this as evidence of our selfish, lazy, reprehensible present. But we’ve been working towards this moment for a very long time, and would it really be so very bad?

If you think that families should look after their own elderly, you’ll need to explain why in south-east Asia, traditionally a region of three- and even four-generation family units, nouveau-riche gated retirement communities are springing up like mushrooms after a spring rain. Perhaps the elderly don’t long to live among us, as we imagine. Perhaps poverty is the only thing nailing Grandma to the family couch. As for the sick, we’ve long since been consigning them to institutions, be they care homes, hospitals or hospices, where people who are better-trained promise to look after them.

The question is not whether we should employ robots. Given the lousiness of some institutions, why on earth wouldn’t we? The question is whether the robots we employ will be any good, and whether we can accept them as substitute humans. We’d like to think not, but there’s evidence to suggest that we’ll bond with even a basic machine far more easily than we’d like to believe.

In 2011, Takanori Shibata, a Japanese engineer, turned up on the coast of tsunami-wracked Fukushima and handed out around 80 robotic seal pups to the victims of the disaster. Refugees warmed to the robots: many have held on to them and continue to look after them. Shibata could have turned up with puppies, or kittens or guinea pigs and would probably have achieved greater therapeutic impact. But who has the money and time to feed and look after 80 animals in a disaster zone? Pets need care and attention — a point not lost on the residential homes that employ Shibata’s robot seals to comfort their elderly, often demented, charges. A single ‘Paro’ — an acronym that roughly translates as ‘personal robot — costs around $5,000. A real-life therapy dog may cost more than $50,000 over its lifetime.

Paro isn’t much of a robot. It can move its head, neck, eyelids, flippers and tail. It responds to the human voice and to touch. It understands simple words and phrases (the sort we use with pets and babies). It knows when it’s being treated well, and when it’s being roughly handled. Its cries (made from digitally sampled baby seal sounds) have a discernible emotional range. It’s old news —the first Paros were sold in 1998 — but it’s making headlines again this year because the ninth generation model is being assessed for use on long space journeys. Mars colonists, permanently deprived of wider human society, will find consolation in a robotic animal chosen for its inability to disappoint. Robot dogs are a let-down because we know what pet dogs are like. How many pet seals do you know? Paro’s very blandness is its point. Its easy, undemanding displays of personal affection reduce stress, anxiety, depression, wandering and aggression among the demented of 30 countries. It must be only a matter of time before Paro makes it into the ‘safe spaces’ on university campus.

Kaspar, designed by the University of Hertfordshire’s Adaptive Systems Research Group, is hardly more sophisticated in appear-ance: a bland foot-high doll in a check shirt. It’s not really a robot — more a mechanical puppet, controlled remotely by researchers. Its expressive minimalism and extreme simplicity reassure the children it plays with — those with severe autism or those who have suffered trauma and abuse.

According to Living with Robots, Paul Dumouchel and Luisa Damiano’s recent survey of social robotics, robots are likely to be stuck in this uncanny state for some time, while we try to codify what ‘behaving like a human being’ actually means. We have vast knowledge of ourselves as social beings, of course, evidenced by millennia of cultural output from Dream of the Red Chamber to Breaking Bad. What we lack is a high-level description of human behaviour of the sort that can find its way into computer code. We all know why we laugh, cry, blush and commit suicide, but we have not the slightest idea what laughing, crying, blushing and committing suicide are for. This is why social robots attract so much academic attention: they are an experimental apparatus, through which we study ourselves.

Countless robot nurse prototypes, with names like Terapio and Robear, are under trial. The problems they are meant to address are real. We have conquered disease to the point where people regularly stay healthy into their nineties. This is why the US has as many people over 85 as children under five and China has 100 million senior citizens to look after. Someone or something needs to look after us in our dotage. Then there are the edge cases: those social wrinkles we could conceivably iron out with robots, but not without consequence. Should we roll out sex robots to address the uneven gender ratios in China? Straight men right now have next to no opportunity for sexual companionship: don’t they deserve some comfort?

Not according to Kathleen Richardson and Erik Brilling, whose Campaign Against Sex Robots, launched in 2015, declares that sex with an animate object that lacks agency can only brutalise us. Notwithstanding that sex robots are a bit rubbish, this particular rabbit hole swallows academics by the ton.

Nations with the most intractable demographic problems are the ones most entranced by the promise of robotics. Japan’s population is crashing as a generation of young people eschews sex. A third of men under 30 have never dated. Women prefer singledom to the life of penury and drudgery afforded by Japanese marriage. A new book by Jennifer Robertson, Robo Sapiens Japanicus: Robots, Gender, Family, and the Japanese Nation, unpicks the Japanese government’s published blueprint for revitalising the nation’s households by 2025. If we can only build robots to do the housework, the argument runs, then women will have more time for having babies. Once again, technology is being promoted not because it ushers in the future but because it preserves the past. (A driverless car is still, after all, a car: not a bus or a train or a decent broadband link. And a robot servant is still a servant.)

On the one hand, robots are like Uber and the spinning jenny. They promise to increase production while preserving the institutions of capital. They’re disruptive right up to the point where something might happen to the money. A more intriguing threat is the one directed at our own social lives. Surrounded by dull, bland, easy-going robot companions, will we come to expect less of other people? Assisted, cared for, and even seduced silly by machines, will we lower our expectations around concepts like ‘conversation’, ‘care’, ‘companionship’ and ‘love’?

Paro and Kaspar are creepy not for what they are — clinical tools, improving the lives of vulnerable people — but for what they portend: a world in which you and I find Paro and Kaspar a sufficient substitute for other people. ‘Robotic companionship may seem a sweet deal,’ wrote the social scientist Sherry Turkle back in 2011, ‘but it consigns us to a closed world — the loveable as safe and made to measure.’ Will our constant association with such easy-going, selfless-because-characterless robots make us emotionally lazy?

We’ve imagined this sort of future many times. Hesiod was writing poems about ineluctable degeneration around 700 BC. H.G. Wells’s The Time Machine (1895) imagines a world in which the beautiful, sensitive people — the Eloi — have all the savvy of veal calves and ‘civilisation’ has turned out to be nothing but a process of self-domestication. And it’s true: civilisation is as much about forgetting, and attendant helplessness, as it is about learning. In my own lifetime, handwriting and mental arithmetic have gone to the wall, and the art of everyday literary nuance is being ousted by the application of quick, characterful emoji. Having to learn new skills is a nuisance. Having to dispense with skills already acquired is a little death: a diminution of the spirit.

The pioneering psychologist William James argued that what we want from a lover is that they really love us, and not simply behave as if they did. I hope that’s true. If we come to believe that the soul is nothing more than behaviour, then of course a robot will become just as good as a person. Why even bother to build better robots? An Eloi future beckons: all we have to do is lower our expectations.