No experiment guarantees a successful outcome; if it did then it wouldn't be an experiment. Lee Gamble comes face to face with this as he takes the first sip of his mocha, picked on a whim from the blackboard of the Soho café we're in. He practically gags before slowly putting the thing back on its saucer as if it's evidence of some terrible crime. My black tea, meanwhile, has been served in its constituent parts. Maybe there was doubt as to where exactly I'd want to put the teabag.

An uninspiring start to an interview, you might think, but it doesn't take much to get Gamble's synapses firing. The tea, he explains, is an example of how capitalism stimulates consumption by presenting the same products to us in novel ways. 15 minutes later he's dropping beautiful words like "rhizomatic" and "bifurcations" and fiddling with the asymmetric collar of his Maharishi raincoat. He wanders into an extended metaphor involving the Voyager probe and space debris. "I don't know whether that just sounds utterly pretentious. At the end of the day, we're in the arts, aren't we? Allow it, man. It's fucking dull otherwise. I mean, it's not dull, but you know what I mean!"

If you had to explain Gamble's interview manner using the language he often applies to his own music—that of outer space—the phrase would be "orbital decay." His thoughts seem like unrelated satellites at first, but they whizz around each other in ever tighter circles, eventually colliding with his point. This afternoon there are two recurring themes. One is to do with his alleged pretentiousness. Knee-deep in press for his stunning new album, Mnestic Pressure, Gamble is keen to defend his love of big words, big ideas and bug-eyed readings of hardcore continental theory. The other is about politics.

"You get it so much on Twitter or something," he says. "Someone will say, 'Oh, keep politics out of music.' I understand what they mean—I get the escapism, and the need to, like, get the fuck out of that shit for a bit, with art or music or something. But look, man, you only have to take a few layers off to find that it's in there. You're listening to fucking techno in Berlin, tough shit. It's in there, mate.

"Especially now, you've got the rise of the fucking right, this neo-fascism, and you've got these so-called smart people talking about fucking eugenics and shit. The fact that these musics and this culture have developed out of systems that are not about what this alt-right shit is about—are the antithesis of it—has to be brought out. We need to see that these things, these cultural clashes, these borderless ideas are positive. It's a good argument against this new idea of nationalism and walls and borders."

This is a new line for Gamble. The Birmingham producer's past music has been nakedly escapist, reaching a peak in 2013's mind-bending double LP, KOCH. The title is pronounced "cotch"—as in the British English colloquial verb meaning to chill out, usually with a spliff—and the contents exploded techno's stable timeclock to induce a brain-fuddled state outside of the daily grind.

Mnestic Pressure is different. As early as the second track, "Istian," the dreamy fog is being swept away by a panicked electro beat and smears of bright melody. The album is tense and restless, and routinely goes for the throat—in the drum salvos of "UE8" and "Ignition Lockoff," the almost-sentimental melody in "A tergo Real," and "Ghost," which is pretty much straight-up jungle.

As Gamble likes to say, it's as if the interstellar bodies fleetingly captured on his past releases have come crashing down to earth. "The time where I was making this record, Brexit was happening, we had Trump happening," he tells me. "The idea of making a record that pulls away from that just didn't feel right. I thought, I need to be back in this, in the mix. Rather than hiding away smoking a spliff somewhere. You can't really be silent on these things."

It's not just this directness that makes Mnestic Pressure a watershed release for Gamble. The record is his first with London's Hyperdub label, after a long association with PAN. As he stoically drinks his mocha, I start to wonder how much his new home has rubbed off on him. It's not just his Maharishi getup that reminds me of Hyperdub founder Kode9; when Gamble calls his music "science fiction," or describes Jamaican soundsystem culture as a "virus," he's singing almost verbatim from the Hyperdub hymn sheet.

But this is nothing new: Gamble's love affair with these ideas goes way back. In the late '90s Kode9 was in the CCRU, a renegade academic unit whose associates included the philosopher Nick Land, now an alt-right poster-boy, but also key dance music theorists Mark Fisher and Kodwo Eshun. The CCRU was at Warwick University, just down the road from Birmingham, where the young Gamble, having "fucked about" at school, was coiling hosepipes by day and considering his next step.

"I wanted to leave—there was fuck all there, not for me. I started looking at doing a degree. I was at my aunt's house, using her internet, and I was like, 'Fucking hell, who are these people chatting about [French philosopher] Deleuze and Metalheadz at the same time?'"

Gamble, like Kode9, was a jungle obsessive, and the CCRU embodied something he had intuitively grasped: that this music wasn't mindless entertainment, but a node in a wider matrix of ideas.

"Goldie and Gilles Deleuze [next to one another] allows you to go, 'OK, well I know about half of it, so maybe the other bit is OK.' There are no steps of, 'Well, this is the intelligentsia and these are the pill-head idiots.' Metalheadz was put alongside these people, equally as valid, equally as vibrant an idea, even if it hadn't been necessarily quantified that way."

Gamble never made it to the Warwick campus, but this worldview still guides him. In conversation he cuts between brain-tickling concepts in the same way that an in-the-zone DJ throws on records. Sometimes the blends work and sometimes they don't, but the momentum never dips.

He compares his study regime to crate-digging. He'll listen to lectures while travelling, or spend an afternoon pursuing some topic of interest in the British Library, jotting down thoughts in an ever-growing pile of notebooks ("I've got boxes of stuff"). These notes form the conceptual kern of his releases, as hinted at in their titles, which touch on numerous fields: topography ("Quadripoints"), premodern musical notation ("Nueme"), graphical modelling ("Voxel City Spirals"), taxonomy ("Caudata"), classical mechanics ("Chain Kinematics"), and of course the CCRU ("Barker Spirals," named for a fictitious Massachusetts professor).

This makes the music feel not like a destination, but a point of departure: a foggy porthole into other worlds. "I think it's right that dance music culture should develop, not just as a form of escapism on a dance floor on a Friday night. Of course it serves that purpose as well, but it's more than that, surely, man. Surely it's deeper than that."

In this regard, the dance music world has slowly caught up with Gamble. Back in the 2000s, when he was making austere "sculptural" computer music and struggling to find an audience, the idea of approaching rave with an academic slant was mostly confined to labels like Mego and Entr'acte. Now it's everywhere: in the thriving experimental festival circuit, the vigorous reappraisal of IDM, and the wealth of cross-disciplinary forms being unlocked by the internet.

Gamble caught this wave early with his breakout releases, 2012's Diversions 1994-1996 and Dutch Tvashar Plumes. Now he finds himself in the centre of the action. He promotes new music via his DJing, his NTS show and his UIQ label, which he describes as "viral" in nature, and which showcases talented newcomers like Zuli, Renick Bell and Lanark Artefax. The amount of new music he receives is "insane," he says. Since he switched to CDJs, he's never played remotely the same DJ set twice.

These experimental boom times are great news for an artist trying to make a living. But they bring the pressures of the market with them. Just as tea-drinkers want their cuppa sold to them in new ways, so too do listeners expect a degree of novelty from a producer they've followed for half a decade.

"To be an artist now, it's not the same as it was in the years of [avant-garde composer] Xenakis, where you were funded from academic institutions. Now everyone can make music—everyone. So to have this longevity, one of the things that you have to play with is that reinvention, that malleability that's required. So what do you do? Do you reject this? Or do you go, 'Alright, there's some interest in it,' and try and reinvent your own shit? Which in a sense means that you can delve further into your ideas. That's not a bad thing, I don't think."

At this point Gamble's whizzing digressions begin to orbit around a new gravitational centre. An old friend from the experimental world had, after hearing the new album, accused him of "going pop." Gamble sees this as another hierarchy to be broken down: "back to the Metalheadz-Deleuze thing: so what? I can do that and that."

But he also takes a pragmatist's approach to compromise. Reinventing himself keeps his audience interested, which means he can keep earning a living from music, and keep exploring his ideas. "Man, or what? Coil hosepipes? The compromise is coiling hosepipes—I'm not compromised because I'm being required to push my work. It might lead to some shit records—whatever, we'll live with that."

Mnestic Pressure isn't shit. It might even be Gamble's best record, which is saying something. It's certainly his most ambitious. Aside from the music itself, the project includes a live show, for which Gamble is working with two programmers, Hyperdub's in-house designer Optigram (also responsible for the album's amazing sleeve), and lighting designer Florence To. There will be two versions of the show, one a "high contrast attack on the senses," the other "smoked-out" and "sensory deprived." The music will draw on the album alongside other parts of Gamble's decade-deep discography. "I have never done anything like this before live," he says. "So it's a steep learning curve, I guess you'd call it."

It's not the only one he's scaling: Gamble is also working on two short pieces for the London Chamber Orchestra, based on the idea of "emergence." The ensemble has form for working with electronic producers. But unlike Actress, who brought his gear to a recent collaboration, Gamble is working solely with orchestral instruments. He has no knowledge of classical notation, but it doesn't seem to be holding him back. "They're used to having notes," he says, "and I was giving them concepts, I was giving them adjectives, I drew these geometric forms."

Gamble's readiness for these challenges gets at why Mnestic Pressure is such a success. He's five years into a full-time music career, which is about the point when he might be getting comfortable. But on some level he's still the hosepipe-coiling autodidact, Yahoo-searching "Deleuze" on his aunt's dial-up. In other words, he's still curious.

Mocha vanquished, we head around the corner to an exhibition by the Mexican-born artist Gabriel Kuri. We take a circuit of the room, peering at replica lettuce leaves and bread rolls arranged on chrome cubes, and Gamble cracks a joke about Heston Blumenthal. Before long those restless synapses are firing again—maybe it's the title of the show, "Afterthought Is Never Binary," which sets them off—and we pick up the train of thought later on Skype.

"That's sort of where the term 'Mnestic Pressure' came from. These points, Trump, Brexit, the DUP thing, where all of a sudden you're expected to make this binary choice: red or blue, you know? The result is this kind of shock moment. As we've seen now with Trump, it's insane really, what we've been bequeathed from this pressure to choose between a couple of things. They're not really binary choices, are they? They don't feel like that nowadays."

These shock moments can be felt in the album's bold dynamics and blasts of kinetic rhythm. They'll be written into the live show, too, as sudden bursts of high-contrast imagery. "Like a zap to your brain," Gamble says—events that arrive, unexpected and unavoidable, and make an indelible impression on your senses.

"I'm not gonna flash up pictures of politicians or anything like that," he reassures me. "I know there's always that debate about politics in work, and I understand the arguments, but I can't personally feel outside of those things just because I'm making some music. I'd have to be a better person than I am to be able to switch all of that off."