On globalism: “… It denies and destroys other cultures and civilisations, specifically those that threaten the universalist values said to be ‘Western’, that in reality exist simply for the benefit of globalising markets and ‘democracy’, summed up in the triptych: fun, sex, and money.[4] It is clear to us, of course, that this globalist pretention rallies the resistance against itself, and even the revolt of peoples who refuse it.”
p15 This universalist belief is also dangerous for those of us in Europe. It stunts our ability to comprehend that other men do not feel, think, or live the same way we do. It is dangerous because it acts destructively upon our own identity. After having colonised other peoples in the name of universalism, Europeans are now in the process of being colonised in the name of the very same principle against which they do not know how to defend themselves: if all men are brothers, nothing can stop the arrival of others on our doorstep.
“The Congolese girls of Leopoldville of the “Parisienne” type are exceptional in every respect. They are completely westernised in their dress and manners and altogether charming. They are well turned out, polite and well acquainted with the social graces. In fact, they are the exact equivalent of the courtesan of Marie Antoinette’s day. They fill a much needed want in this respect. Many of the Congolese politicians have advanced so speedily that their wives are still in the mud-hut stage and are totally incapable of behaving in public according to western standards, and the sight of a table laid with an array of knives and forks is known to send them into panic. Not so the courtesan. She has studied this situation. I was not surprised then to learn that these girls are greatly in demand by the recently arrived and that formal invitations to official functions even made provision for this fact, by acknowledging that they will be welcome as guests in default of a wife.”
On comforts in the congo:
“The food,” said one such disappointed warrior, “was terrible.” “There was,” pointed out a second would-be saviour of the Congo, “no beer.” “We were,” declared yet another intrepid irregular, “getting hacked to pieces.”
Concerning the spoils of war:
A white soldier (not my unit) tried the handle of an upstairs bedroom. It was locked. He shot out the lock and bashed in the door with his boot. Inside he found a young Congolese girl, hiding in the shower cubicle. He stripped off her clothes till she was naked. He liked what he saw. “Shower,” he ordered her, “then lie on that bed.” Without a word she obeyed. He raped her. Then he ordered her downstairs with the other prisoners and marched her to the river’s edge some sixty yards away. A small pier ran out ten yards into the fast flowing water. “Walk down that,” he commanded. She knew she was going to die. With the impulse of revenge, tinged with a spark of genius, she turned and screamed at the sadist, words which would last him the rest of his life … “You don’t know how to make love … you’re too small!” With a derisory laugh she faced her death. It came a second later. Two shots rang out and with a pitiful “Oh!” in sad contrast to her brave speech, her body disappeared into the Congo for ever.
On the barbarism of the natives:
The Provincial President, a man known to be loyal to Mr. Tshombe’s Government, was executed in front of hundreds of jeering rebels with ritual bestiality, as a warning to all others. The ritual followed the age-old custom. First his tongue was cut out. Then his ears, his hands and his feet were hacked off with razor-sharp pangas. Finally, a bamboo stake was driven into his rectum. He lasted fifteen minutes, watched by the insane mob of hooligans. The savages had not moved one inch towards civilisation in the last eighty years, despite the noble self-sacrifice of hundreds of missionaries.
Trial by “acclamation”:
On a flood-lit platform a tribunal was in session. It was a “trial by acclamation.” As I watched it, I realised the clock had been put back two thousand years. A rebel was paraded on the dais and his name announced over the public address system. If he was cheered, he was released. If he was hissed, he was taken out and shot.
A typical Christmas (1964) celebration amongst mercenaries in the Congo:
During the course of the dinner, however, a shot rang out in the dining-hall. He could contain his curiosity no longer. “It is perhaps a normal occurrence?” he said, without raising his voice a semitone. Alastair investigated. It was nothing, he said, just a little good-natured buffoonery. One man had shot another by mistake. The wounded volunteer was removed and the happy meal went on.
…The African, generally, has not the makings of a good soldier and lacks the necessary self-discipline and courage essential to the task… in the long term I see no probability of vast African armies rampaging up and down the continent, if only for the basic reason that the average African at heart is not a soldier. Economically, one imagines, it will not be possible for many African countries to maintain large standing armies, but in any case the idea of taking arms to redress their wrongs is not one which is likely to prove attractive to many Africans…
…During my campaigning I came to meet a large number of Congolese officers and civilians with whom I was able to discuss intimate matters of this nature quite objectively. One of the favourite subjects for debate was the difference which exists between the European and African character. The Congolese conceded that they did not understand the meaning of chivalry, in fact there is no word for it in Swahili or Lingala, and the concept of sportsmanship, the son of chivalry, was completely unknown to them. These were distinctly European attributes for which they could see little use in the African context. Gratitude was another, but I agreed readily that their sense of loyalty and devotion to their family unit far surpassed anything of which we were capable…
…my Congolese friends came to understand how shocked the European mind can be at cruelty, although this is something which is accepted as quite normal up and down the African Continent. I recall seeing a soldier of the A.N.C. pluck the feathers from a living dove and then throw the naked bird, still alive, on to a bed of red-hot coals to cook it. He was genuinely unable to understand my rancour…On the other hand I was genuinely unable to understand their attitude to ritual torture, something which had been handed down to them through the centuries. A prisoner of war must be killed after ritual torture, it was always thus, and nobody expected anything different…
Politics in the Congo is very similar to that in the U.S. today:
The Army was fortunate in that it had an efficient administration, free from political interference. Its clear-cut methods had been handed down to it by the old Force Publique and a chain of command, good system of communications, and a reasonable standard of discipline obtained throughout. The Army represented, in effect, the only system of administration which had shown itself capable of government, the civilian system having broken down under the strain of events and the junketings of unscrupulous politicians. To make matters worse, the civilian machine had been fraught with office seekers, opportunists, financial mendicants, and politicians whose sole aim in life was not service, but personal aggrandisement….In fairness to the politicians, it could be said that no pattern of behaviour existed for them to copy, they had received literally no training for executive positions or administrative matters at the higher levels, and to them political appointment represented the ultimate in African sophistication, coupled, as it was seen to be, with instant acclaim, great wealth, and fantastic power.
We (U.S.) need Belgian type retribution applied to corrupt politicians:
As part of his general overhaul of the Army during the last fourteen months, the Commander-in-Chief had introduced a Judge Advocate General’s Department and placed it under a distinguished Belgian Officer, Colonel Van Hallowen. It was a brilliant move. In the first few months of its existence the J.A.G.’s Department tightened up control throughout the Army and court martialled several highly placed officers and Military Governors for defalcations of large sums of money, many of whom were sentenced to terms of imprisonment exceeding eight years.
…The salvation of the Congo, as I see it, will be the reintroduction of as many Europeans as are prepared to emigrate to the country to become part of the fabric of the Congo, to help the Congolese on the road to political maturity and to teach them the skills of commerce and administration. These immigrants must come with a new mind—not as “agents sous contrats”, the iniquitous Belgian system1—but as settlers, as white Congolese, who will take a pride in their adopted country and who will come not with any superior colonialistic ideas, but with the genuine desire to help the Congolese help themselves…
On brotherhood:
--Why, the isolation that prevails everywhere, above all in our age- it has not fully developed, it has not reached its limit yet. For everyone strives to keep his individuality as apart as possible, wishes to secure the greatest possible fullness of life for himself; but meantime all his efforts result not in attaining fullness of life but self-destruction, for instead of self-realisation he ends by arriving at complete solitude. All mankind in our age have split up into units, they all keep apart, each in his own groove; each one holds aloof, hides himself and hides what he has, from the rest, and he ends by being repelled by others and repelling them. He heaps up riches by himself and thinks, 'How strong I am now and how secure,' and in his madness he does not understand that the more he heaps up, the more he sinks into self-destructive impotence. For he is accustomed to rely upon himself alone and to cut himself off from the whole; he has trained himself not to believe in the help of others, in men and in humanity, and only trembles for fear he should lose his money and the privileges that he has won for himself. Everywhere in these days men have, in their mockery, ceased to understand that the true security is to be found in social solidarity rather than in isolated individual effort. But this terrible individualism must inevitably have an end, and all will suddenly understand how unnaturally they are separated from one another. It will be the spirit of the time, and people will marvel that they have sat so long in darkness without seeing the light. And then the sign of the Son of Man will be seen in the heavens.... But, until then, we must keep the banner flying. Sometimes even if he has to do it alone, and his conduct seems to be crazy, a man must set an example, and so draw men's souls out of their solitude, and spur them to some act of brotherly love, that the great idea may not die.
On the Comanches at their peak (early 19th century): …warriors and women, lived a short, perilous, brutish existence, full of pain and sorrow, but also filled with marvelous exhilaration and exultation. The patterns of their life, with the male population kept low by hunting and warfare and the females kept barren by constant riding and heavy labors, assured that they would never overpopulate the plains. The Comanches could not have destroyed the buffalo, their staff of life, in a million years of this closed cycle. They lived, or seemed to live, in a world without end, and it was the only world they knew or wanted to know…
On Anglo-American children captured during Camanche raids: …Such children quickly forgot their native language and old associations and strove to become accepted Comanches. The girls looked forward to becoming the wives of great warriors. The boys dreamed of the day when they would become an honored member of the warriors’ circle, or lead their own war band. The children absorbed the beliefs and drives of the People, and many who returned or were rescued admitted candidly that they had learned to love the wild life on the plains, with its thrills, raw dangers, exaltations, and leisure that no civilized existence could match. Once children adjusted to Comanche life they were almost never able to readjust again to civilization. They became as culturally wild and stubborn as born Comanches. The process seemingly could not be reversed…
On the chasm of world views: …But the Amerindians were isolated from all Europeans by world-views and cultural divergences that had been widening for at least four thousand years. To become “civilized” like Anglo-Americans (or Spaniards), the truly primitive tribesman had to do more than learn a new language and pick up a few new techniques. He had to betray his whole concept of the world and man’s role in it, and destroy all his cultural instincts and laws and beliefs—everything that to him seemed natural or sacred. The Amerindian spirit world and the European universe of cause and effect did not just exist on higher and lower technical planes. They were utterly disparate and innately hostile; the freedoms of one were the abject tyrannies of the other, and vice versa. The Amerindian adjusted and attuned to nature, gaining an enormous spiritual intoxication but little power over the real or physical world. The Indo-Europeans who reached America eschewed spiritual intoxication and even spiritual peace while mastering the physical world—even if they had to destroy it—by appalling, incessant labor. Even in the hungriest bands, no Amerindian warrior accepted such crushing tyranny. The Indian, as an Indian, could not go very far down any white man’s road to civilization…
On conflicts of interest: …the British had had no experience with a primeval wilderness. The first settlers could not have survived had they not learned agronomy and woods lore from the tribes that received them with the hospitality reserved for distant travelers. Once ensconced, the planters and Pilgrims were compelled to attack and destroy the wilderness in order to create civilization as they knew it. They felled trees, made roads, put up fences, and erected permanent structures; they carved out fields and made smoke rise from a thousand raw clearings. A mere handful of industrious white men in a region soon utterly changed the face and nature of the land forever, ruining it for the ancient uses of the Amerindians…
On history repeating itself: “All men from Washington are liars” –Spotted Tail, Dakota war chief
…First contacts between the strangers were usually peaceful. The Indians lacked hatred for strangers; their energies were engaged in enmities with neighboring tribes. …
…The bloodiest Indian wars were actually fought along the Atlantic seaboard in the seventeenth century, never on the western plains. Here, the whites suffered their greatest losses in proportion to total numbers. Thousands of English colonists were killed from the Virginia tidewater to New England when at last the various eastern confederacies took the war trail in despair, to throw the encroaching newcomers back into the ocean….
On feminism: …The individual family groups were ruled by a tyranny of biology and circumstance. Only males had the power to hunt and fight, the two roles that ensured immediate survival. All other work logically fell to females, with one symbolic exception—men made weapons. The males being stronger, though not necessarily more aggressive, they could enforce their dominion over females and family. The male-female roles had long been set in hardening custom-cement. Females had become inferior and subordinate. Among the Nermernuh as among most other Amerindians, women had become almost chattels, as the ancient practice of wife-immolation revealed…
High Speed Rail (HSR) has been in the news, with a recent New York Times article listing some of the reasons that the California HSR project seems unlikely to ever be completed. Quite aside from California’s development quagmire and the article’s author’s unstated involvement in the story, there are a series of much deeper, physical reasons why HSR hasn’t really caught on. I haven’t seen these developed in accessible blog form so I thought I would write this brief note on the topic.
[Edit: This blog generated more controversy than usual, and a thread on HN. I was surprised to see how readily some of the core ideas were misunderstood, though it is true that I haven’t written as much recently as I used to, and I’m almost certainly less sharp as a result. This blog contains generalizations and inaccuracies – it’s intended as a jumping off point for the interested reader. One of them (Ashton) wrote a rebuttal! The bottom line is that California high speed rail won’t work for dozens of reasons, and I wrote about a few relatively obscure but quite fundamental ones.]
My personal background on trains is that I love them! I’ve taken hundreds of trains across multiple timezones, in China, Japan, Mongolia, Russia, all over Europe, Vietnam, Cuba, Australia, New Zealand, and the US. I also worked on track-based transport as a levitation engineer at Hyperloop between 2015 and 2018, so I have some appreciation for the art. I also spent a bunch of time on transport economics at Hyperloop, which taught me a lot about why my frustrations with rail’s failure to take over actually occurred, as well as some of the deeper reasons why Hyperloop is a rather challenged idea. For those who want more depth, parts of this blog derive from academic studies on wheel-rail dynamic vibration and rail spalling, more rolling contact fatigue, and Japanese shinkansen maintenance analysis.
Despite my ongoing enthusiasm for HSR, I have to concede that it is not a universal panacea. It remains relatively niche and relatively undeveloped. It is possible that its failure to be deployed everywhere since it was first developed 50 years ago is due to short-sighted governments and cost disease (the tendency for modern construction within developed cities to cost much much more than the original project did historically), but other factors also contribute to its lack of competitiveness.
Despite decades of development, only a handful of routes in Europe operate at anything like airplane-competitive speeds, which for all but the shortest routes, require > 300 km/h or > 185 mph.
Even turboprop aircraft, which are relatively slow, cruise at > 500 km/h, while jets cruise at > 800 km/h. Over longer routes, the relative hassle of getting to and from an airport instead of an HSR rail terminal is eroded by the higher speed of aircraft, even in places where the rail terminal is in a densely populated city center and the airport is way outside. Comfort and convenience are other important factors, but there aircraft are also quite competitive.
In general, the number of >300 km/h HSR networks on Earth can be counted on one hand, and the total track length is a minuscule and mostly static fraction of global rail network length, which itself is shrinking at a reasonably rapid pace.
Japan, a linear archipelago, famously developed the high speed Shinkansen but like recent Chinese development, it must be seen in the context of very heavy-handed government subsidies and a response to geographic and structural factors that inhibited the development of airports. For example, while the US has more than 15,000 airports (most of which are untowered paved strips), Japan and much of China is relatively mountainous, historically relatively poor, and historically beset by relatively poor transport networks. Add to that various Japanese prohibitions on certain weapons technology in the post war period and high speed rail served as a government imposed solution to mass transportation.
It did not come without cost, however. Japan’s ostensibly private rail companies have gone bankrupt and been bailed out so many times I’ve lost count, racking up billion dollar yearly deficits year after year. Indeed, as far as I know there isn’t a single HSR route anywhere on Earth that operates profitably on ticketing revenue, and so operation always requires substantial subsidies. I should probably mention here that actual ridership and thus fare revenue is, as a rule of thumb, typically around a third of the projections used to justify initial development.
One can argue that aviation and any other kind of transport also benefits from various subsidies, such as expenditure on the US Navy guaranteeing freedom of navigation, without which the global oil trade wouldn’t work. Or that CO2 emissions from aircraft are an unpriced externality that HSR partly alleviates. But if we care about HSR and its ability to enable people to travel with fewer emissions or lower costs, we need to understand why it’s so expensive, no matter who is building it or where.
Why is HSR so expensive?
I will discuss three main groups of reasons: rail is suboptimal, HSR grading requirements are really tough, and steel-on-steel rolling is less perfect than you might think.
Rail is kind of obsolete
The first set of reasons are common to all kinds of rail. As mentioned in my post on traffic congestion:
There are a few reasons. Some are similar to car economic problems, with peak and average demand variation, particularly for commuter services. But I think the fundamental reason is that compact diesel engines got, if not good, then acceptable, in the 1930s. After that, shippers could move freight in almost any form factor between any two points directly. Even in 2022, freight by rail is much slower as rail cars must wait in yards for trains to be assembled.
There is another direct issue with trains, which is that rail systems are, by their nature, one dimensional. Any disruption on a rail line shuts down the entire line, imposing high maintenance costs on an entire network to ensure reliable uptime. To add a destination to a network, an entire line must be graded and constructed from the existing network, and even then it will be direct to almost nowhere.
Contrast this with aircraft. There are 15,000 airports in the US. Any but the largest aircraft can fly to any of these airports. If I build another airport, I have added 15,000 potential connections to the network. If I build another rail terminal and branch line, at significantly greater cost than an airstrip, I have added only one additional connection to the network.
Roads and trucks are somewhere between rail and aircraft. The road network largely already exists everywhere, and there aren’t any strict gauge restrictions, mandatory union labor requirements, obscure signaling standards, or weird 19th century incompatible ownership structures. Damage or obstruction isn’t a showstopper, as trucks have two dimensions of freedom of movement, and can drive around an obstacle. In Los Angeles during the age of streetcars, a fire anywhere in the city would result in water hoses crossing the street from hydrant to firetruck, and then the network ground to a halt because steel wheels can’t cross a hose or surmount a temporary hump!
Building a metro system in an existing dense city is also great (if we can avoid cost disease) but for most of the cities in the US, the suburbs are already not walkable enough to enable non-vehicle transport to a neighborhood station. The suburbs of LA will never be able to depend on a Manhattan or Vienna-style underground railway.
To make this concrete in the context of the ill-fated California HSR project, the NYT article quotes the rail authority chair Tom Richards saying “The key to high-speed rail is to connect as many people as possible.” There are a couple of unstated assumptions here, but it also reveals a fundamental problem with California HSR as it was conceived, which is that in order to get enough political buy in it had to promise too many things to too many stakeholders.
If we want to reduce CO2-generating air traffic between San Francisco and Los Angeles (a worthy goal!) then the HSR route must be, above all, fast. The oft-stated goal time of 2 hours and 40 minutes is both unachievably rapid with finite money and current technology, and also too slow to compete with aircraft, but for insane TSA security delays that will probably also affect HSR. It prompted the Hyperloop experiment, which sidestepped some of the problems and generated others.
Routing HSR on the east side of the central valley via Bakersfield and Modesto means those cities can have a station, but frequent services means that most trains have to stop there, and each stop adds 20 minutes to the travel time just to slow down and speed back up. Alternatively, the stations and their railway corridors are extremely expensive city decorations that help no-one because the trains, dedicated to a high speed SF-LA shuttle, never stop. Because they are trains, we can’t have both. If it was aircraft, we could have smaller, more frequent commercial aircraft offering direct flights to dozens of destinations from both cities. But rail has relatively narrow limits in terms of train size and frequency meaning that any route will be both congested at peak times and under-utilized for much of the rest.
Serving peripheral population centers in California is a nice thing to do, but aircraft pollution from Modesto is not driving global warming. Car traffic from Modesto would hardly overwhelm the Interstate 5. HSR minimizes financial losses when it is serving large population centers with high speed direct services. By failing to make the political case serving the main mission, the CA HSR project adopted numerous unnecessary marginal requirements which added so much cost that the project is unlikely to succeed. Even if the money materializes and the project is completed, the train will be so slow that it will hardly impact aircraft demand, so expensive it will be unable to operate without substantial subsidies, and so limited in throughput that it will hardly even alleviate traffic from LA’s outer dormitory suburbs.
In other words, one can build a commuter rail network, an intercity network, or a point-to-point HSR line, but forcing all three usage modes into the same system cannot succeed.
The Earth is kinda bumpy
To understand the challenges of grading HSR, we need to first examine the nature of the bumpiness of the Earth.
To ancient humans who first walked the Earth, it appeared flat enough, at least at local scales. Go far enough or watch a Lunar eclipse and it becomes clear the Earth is, at large scale, round. To a decent approximation (about 0.3%) it is spherical.
Let’s examine corrections to this approximation. First, the equatorial bulge. The shape of the Earth is an equipotential, and centrifugal forces add to gravity, which makes the middle bulge out a bit – about 20 km depending on how one measures. There’s also some triaxiality, which is to say the equatorial bulge is marginally more bulgy through Africa/Hawaii than SE Asia/Americas. Next come the geoid corrections. Local variations in the density of the crust and upper mantle cause deviations to the equipotential surface of up to 100 m. This is rather small compared to the equatorial bulge, but still rather large. Once the geoid is added, we know everything there is to know about the Earth’s gravitational field, at least at scales of 100 km or so. Excepting deviations due to weather and tides of order 1 meter, the geoid gives the altitude of the ocean.
The final layer of detail is hills, mountains, valleys, and other hard rocky stuff that pokes up on the Earth’s land surface. For essentially the entire world, this has been mapped to a resolution of 90 m by the Shuttle Radar Topography Mission, while substantial swaths of the US and other countries have been mapped to 1 m resolution or better, using airborne lidar.
Okay, the Earth has bumps. What’s the big deal?
The bumps have a really big effect on how fast people can move close to the surface of the Earth. There are two ways to understand this. The first is intuitively, and the second is by looking at the von Karman-Gabrielli diagram.
The force experienced due to bumps: F = m v^2/r. For a given curve of radius r, such as a hump in the road, the force experienced increases quadratically with velocity v. This is a big deal! The big deal, even!
Human passengers don’t like to experience high forces, especially while walking around in a train, so in practice this fundamental physical relation limits the r that can be experienced at a given v. For v = 320 km/h, r = ~8 km. This applies for both lateral and vertical deviations! For context, my children’s Brio train set has a radius of curvature of about 30 cm or 12″. 8 km is roughly the distance to the horizon.
This makes sense intuitively, too. A twisty road that is comfortable to drive at 35 mph is edgy at 45 and dangerous at 55 – where the forces are 2.5x greater! Walking through a crowded mall presents no challenge but sprinting is asking for trouble. Speed + bumps = trouble.
To a good approximation, HSR lines have to be dead straight. In Kansas or California’s central valley, this is fine, up to a constant in Eminent Domain, which is the politically fraught process of the government taking your land by force. But both LA and SF are ringed by a series of extremely geologically active, steep, and tall mountain ranges. The Interstate 5 out of LA goes through the Grapevine, passing through a point where 5 (5!!) different active fault lines intersect in one place. Maintaining a useful speed through these mountains, not to mention densely populated areas nearby, requires nearly 100 miles (!) of tunnels. Current tunneling costs are orders of magnitude too high, but even then tunnels are typically only built in places with known and acceptable geology, and much of the geology under the San Gabriel mountains is simply not known.
This is not the place to go into depth, but those mountains have seen things, geologically speaking, which should not be possible. What is known is that the entire mountain range is a gigantic pile of broken, crushed rocks that have been rotated, turned upside down, drowned, volcanically erupted, eroded, subducted, and then sheared. Just one of the dozens of tunnels required could easily cost more than $113b, the current estimated cost for the project.
From a 2015 article: “No way,” said Leon Silver, a Caltech geologist and a leading expert on the San Gabriel Mountains. “The range is far more complex than anything those people know.”
The mountains surrounding the Bay are not quite as tall but, straddling the San Andreas fault, no less challenging. Remember, at 320 km/h, anything taller than a viaduct standoff counts as a mountain that needs a cut or a tunnel – perhaps 20 meters of wiggle room if we’re being extremely generous.
The recent NYT article lists a bunch of political reasons that the project is in deep trouble but even if various CA governors and US presidents had written a Chinese-style blank check and there were no land acquisition disputes, the mountains are still there.
The second way to understand the bumpy earth limitation is the von Karman-Gabrielli diagram. This diagram plots the speed and specific power of every mode of transport on a single chart. I love this kind of data presentation.
The zeroth order truth of the vK-G diagram is that there is a limit line of vehicular performance, which is essentially determined by momentum transfer limitations for vehicles that have to displace water or air to move along. It does not apply to spacecraft!
The first order truth is that, above 100 mph, the most efficient transport mechanism shifts from ground-based to air-based. For smaller creatures than humans, the transition speed is a lot lower – most insects fly instead of walk. For insects, this is because at their scale, the world is ridiculously bumpy and hard to navigate.
The galaxy-brain detail is that, between 100 mph and 300 mph, there is a gap in the frontier, where no technology gets close to the G-K limit. Many innovators have tried to slot hovercrafts or ground effect vehicles (such as the ekranoplan) into this gap, but all have failed. Hovercrafts have not caught on for the same reason as HSR – above 100 mph, the Earth is too rough to travel close to its surface.
This is also intuitively obvious to pilots, who understand that making a habit of flying planes within 20 m (or even 200 m) of the surface, particularly in mountains, is a career-limiting move. Indeed, even at slower approach speeds, commercial airliners take half the city to turn around to line up with the runway. Translate the motion of a 737 on downwind, for example, to the surface and even a fighter pilot would not be able to track the ground within the range that HSR must be built.
As a result, HSR grades cannot be built between nearly any city pair on Earth without moving a LOT of dirt and rock and pouring a LOT of (CO2-emitting) concrete, most of which only has an actual train on it for a few seconds per hour, and thus drives incredibly high cost of construction.
Of course roads also operate with public subsidies and require expensive grading, but road traffic is slower, more diverse, and more versatile, while road materials are far cheaper and car operating costs are borne by the user. The result is that the per mile and per passenger mile costs of roads are much lower than HSR. For example, the I-70 cost an average of about $2m/mile, despite routing through remote and mountainous parts of Utah. CA HSR is currently budgeted at more than $350m/mile.
Rail wear, or steel wheels in the real world
Finally, we come to the third major challenge of HSR and another major contributor to its cost. Steel wheels and rails are hard – they’re made of steel, but they wear over time. Wheels must be remachined and rails must be reground.
A typical Japanese maintenance schedule has each segment of rail reground, to exacting tolerances, every 6 months while total replacement is required every 5 years. These grinding and replacement operations, which must be carried out continuously, degrade system up time and require, on average, a fully salaried track worker per km of track. These numbers apply only to perfectly straight track – switches, curves, and steep grades wear out substantially faster.
How does wear occur? A typical HSR wheel bears a static load of 6 T across a contact patch the size of a postage stamp, with both rail and wheel deforming about 20 microns to enable contact. The center of this patch endures a pressure high enough to plastically deform the rail’s steel! The passage of the wheel places symmetric forces (first forward, then back) but the effect is to temper the surface, which accumulates stresses and can flake off. Additionally, torque on the wheel tends to lock the wheel statically to the track as the patch is loaded, but during the unload portion as the wheel passes the accumulated stresses are released, resulting in shear and friction, particularly on parts of the track where the train is accelerating, slowing down, climbing, or descending.
Despite this terrifying pressure, one wheel passing might deform the surface by only a single Angstrom – the width of a single atom. The Tokaido Shinkansen sees 150 services a day, each with a 16 car train and 4 wheels per track per car, so the track endures 1.5 million wheels between 6-monthly regrindings. Linear damage would imply 0.15 mm of wear, but damage isn’t linear.
Instead, once the rails deform more than a few nanometers, the “bumpiness of the world” comes back with a vengeance. Bumps induce acoustic oscillations in the wheels and track, which ring like a gong or very angry violin. Wheels being made of hard steel, these oscillations are poorly damped and cause local variations in the position and force of the contact surface. Some of these variations cancel out the bumps and smooth out the tracks, but some of them don’t, and over time the randomness of these acoustic perturbations roughen the tracks by much more than a single layer of atoms per wheel.
Rail wheels are much lighter than the cars they support, so their suspension system drives them into the track with a force of about 700 gs. At 320 km/h, the critical r, or bump height, is just 50 microns. Less than the width of a hair, and not that different to the 20 micron (mostly) elastic deformation of the contact patch. Once acoustically grown bumps get to 5 microns or so, they begin to induce oscillations in the suspension system. This is damped better than the wheel’s acoustic modes, but damping always lags the input and the effect is to begin to drive “washboard” shapes into the rail, rapidly increasing track deformations. Once deformations exceed 50 microns, the wheel actually breaks contact with the track, hammering it on its return with almost unimaginable force and rapidly grinding out holes.
Cumulatively, these effects are at first linear over time, then quadratic, and eventually exponential. The forces are proportional to the square of velocity, so faster HSR trains damage rails faster. The Tokaido line averages around 140 mph (somewhat less than its peak of ~185 mph), but increase that speed by just 40% and rail lifetime will (at least) halve, while track maintenance costs (at least) double. Maintenance costs that were already on the order of $200,000/km/year, in 2003 dollars. That’s $400m/year just for rail maintenance for the LA-SF route, once we correct for inflation and a higher design speed.
There’s got to be an easier way
As of 2022, the CA HSR project is supposed to cost $113b. The vast majority of this is unfunded, and yet the final project will almost certainly cost at least 10x this if it ever completed, and will still be unable to compete on the LA-SF route with aircraft.
Similar stories abound the world over. There are a handful of locations where land is flat enough and property ownership protections weak enough that HSR can be built with minimal fuss, and sometimes even between cities with strong latent transport demand that can be unlocked, but even then it is a niche solution that takes decades to develop and can’t pay for itself. If this weren’t the case, we’d see HSR developed everywhere, instead of something governments talk about for decades and, usually, never actually build.
By CA HSR’s own numbers, the completed system may carry 35 million passengers per year by 2040, or 100,000 per day. This capacity could also be served by a fleet of just 40 737s (less than current LAX-SFO traffic), of which Boeing makes more than 500 per year. Bought new, this fleet would cost $3.6b, and with a lead time of, at most, a few months. Upgrades to Modesto and Bakersfield airport terminals could service the 737 for mere $10s of millions. The fleet would cost about $2.9b to operate each year, which under current airline business models can be served by fares of about $60 each way. If we operate this airline for free (no tickets!) for 40 years, the total operating costs climb to $120b, which is equivalent to CA HSR’s currently wildly unrealistic estimated construction costs.
That is, a passenger jet that first flew in 1967 can continue to profitably serve the LA-SF transportation market for less money, over multiple decades, than the rather slow HSR could be constructed much less operated, in our wildest dreams.
Where HSR has to bore tunnels through >100 miles of incredibly unforgiving hard and flakey rock for decades just to get somewhere, planes fly serenely through the unobstructed atmosphere. Where trains must slow down and speed up to serve political expediency in smaller intermediate stations, planes route freely through the three dimensional sky direct to their destination, and at 3x the speed, and at lower overall energy usage per passenger-km.
Planes emit CO2 as they fly, but CO2 emissions on routes that could be served by HSR are a tiny fraction of aviation’s total, which itself is a small fraction of the totality of humanity’s output. It can be directly offset through carbon capture and sequestration for a modest increase in the ticket price, as plane ticket prices are mostly not fuel. Alternatively, synthetic aviation fuel is under development to make aircraft carbon neutral. Indeed, at Terraform Industries we think synthetic fuel will ultimately be even cheaper than current options, expanding access to the convenience, speed, and safety of air travel.
Trains are wonderful and I love the Shinkansen, but let’s stop flogging this dead horse. HSR is not a compelling option for generic high speed intercity transport.
Details of the Errol affair
On Errol's arrival in Nairobi:
...No one knew that Errol was joining the train at Athi station, just outside Nairobi, with his man, Waiweru, and his portable drinks cabinet...
A guide for the English who have decided to emigrate to Kenya
Geographic features and weather
Temperment, utility, work ethic of local tribes
Suitable crops and animals for farming
Wildlife
Pests, diseases, and medical advice
Sports and other distractions
Advice for women
Laws and etiquette
On the ending to a typical day: …Darkness falls all too quickly and at 6.30 a bath and change into pyjamas is the order of the day. At 7 dinner—soup, buck, partridge and pudding, not by any means forgetting whiskey and a glass of port. Then with a pipe comes the writing of a letter or two, the filling in of the day book and possibly the balancing of accounts. At nine o’clock, healthily tired, we call to our dog and turn in…
On hunting with Teddy Roosevelt: …Roosevelt’s bulk and conversational powers somewhat precluded him from tracking, since the utmost caution and lack of noise are essential…
On opportunities for meeting women: …The settler from the back blocks has very likely not met a white lady since his last race meeting and is ready to see beauty in the most meagre charms. She must be an unattractive damsel indeed who cannot, if so desirous, bring at least one eligible bachelor to her feet during the time at her disposal…
Golfing: … It is a game especially suitable to those who are working in the capital. The official or business man usually leaves his work about 4, and has just comfortable time for a cup of tea and a round of golf amid beautiful air and surroundings…
I ment to post this much earlier but have been fighting with neurological longhaul covid symptoms for most of the year.
TL;DR: I worked on biomedical literature search, discovery and recommender web applications for many months and concluded that extracting, structuring or synthesizing "insights" from academic publications (papers) or building knowledge bases from a domain corpus of literature has negligible value in industry.
Close to nothing of what makes science actually work is published as text on the web
UIs serving up “insights” from machine-reading millions of papers through specialized search engines, paper recommenders, citation analysis, correlation plots of mentions, hierarchical clusterings of topics, interaction networks (relation extraction), causation graphs [...]
Here’s the outline: (If you can't see a ToC, try reloading the page). You can comment on this post on Twitter or HN
Psychoanalysis of a Troubled Industry
Atop the published biomedical literature is an evolved industry around the extracting, semantic structuring and synthesizing of research papers into search, discovery and knowledge graph software applications (table of example companies). The usual sales pitch goes something like this:
Generate and Validate Drug Discovery Hypotheses Faster Using our Knowledge Graph
Keep up with the scientific literature, search for concepts and not papers, and make informed discovery decisions [tellic]
Find the relevant knowledge for your next breakthrough in the X million documents and Y million extracted causal interactions between genes, chemicals, drugs, cells… [biorelate - Galaxy]
Our insight prediction pipeline and dynamic knowledge map is like a digital scientist [OccamzRazor]
Try our sentence-level, context-aware, and linguistically informed extractive search system [AllenAI’s Spike]
Or from a grant application of yours truly:
a high-level semantic search engine and user interface for answering complex, quantitative research questions in biomedicine
You get the idea. All the projects above spring from similar premises:
The right piece of information is “out there”
If only research outputs were more machine interpretable, searchable and discoverable then “research” could be incrementally automated and progress would go through the roof
No mainstream public platform (eg. arxiv, pubmed, google scholar) provides features that leverage the last decade’s advances in natural language processing (NLP) and they do not improve on tasks mentioned in 2.
Two sentences semantically with multiple layers of semantic annotationAn example of a generic biomedical entity-relation schema that might power a knowledge base on top
It really does sound exciting and VCs and governments like to fund the promise of it. Companies get started, often get a few pivot customers, end up as software consultancies, or silently fail. In rare cases they get acquired for cheap or survive as a modest “request a demo” B2B SaaS with 70% of the company in sales with unit economics that lock them in to only sell to enterprises and never to individuals 0.
Meanwhile big players with all the means to provide a category leading platform have either stopped developing such an effort altogether (Google Scholar), shut it down and nobody cares (ie. Microsoft Academic Search) or are unable to commercialize any of it (AllenAI with SemanticScholar, Spike etc.).
Let’s take AllenAI as an example. AllenAI is a world-renowned AI research organization with a focus on NLP and automated reasoning. They sponsor a startup incubator to spin-off and commercialize their technologies. The AllenNLP platform and SemanticScholar is used by millions of engineers and researchers. And unlike most big AI research groups they also build domain-focused apps for biomedicine (like supp.ai and Spike).
And yet, despite all of that momentum, they can’t commercialize any of it:
We have been looking at opportunities to commercialize the technologies we developed in Semantic Scholar, Spike, bioNLP, etc. The challenge for us so far is in finding significant market demand. I'm curious if you have found an angle on this ? [...]
This was an email reply I received from a technical director at the AllenAI institute. At that point, I had already spent months building and trying to sell biomedical NLP applications (bioNLP) to biotechs which had made me cynical about the entire space. Sure, I might just be bad at sales but if the AllenAI with 10/10 engineering, distribution and industry credibility can’t sell their offerings, then literature review, search and knowledge discovery are just not that useful…
When I then looked at the SUM of all combined funding, valuations, exits of _NLP_ companies in the semantic search / biomedical literature search / biomedical relation extraction space I could find it was less than the valuation of any single bigger _bioinformatic_ drug discovery platform ( ie. Atomwise, Insitro, nFerence). Assuming the market is efficient and not lagging, it shows how little of a problem biomedical literature search and discovery actually is.
Just to clarify: This post is about the issues with semantic intelligence platforms that predominantly leverage the published academic literature. Bioinformatic knowledge apps that integrate biological omics data or clinical datasets are actually very valuable (ie. Data4cure), as is information extraction from Electronic Health Records [EHRs] (like nFerence, scienceIO) and patent databases.
My Quixotic Escapades in Building Literature Search and Discovery Tools
Back in March 2020 when the first covid lockdowns started I evacuated San Francisco and moved back to Austria. I decided to use the time to get a deeper understanding of molecular biology and began reading textbooks.
Before biology, I had worked on text mining, natural language processing and knowledge graphs and it got me thinking ... could you build something that can reason at the level of a mediocre biology graduate?
After a few weeks of reading papers in ontology learning, information extraction and reasoning systems and experimenting with toy programs, I had some idea of what's technologically possible. I figured that a domain-focused search engine would be much simpler to build than a reasoner/chatbot and that the basic building blocks are similar in any case 0.5.
I wrote up what I had in mind, applied to Emergent Ventures and was given a grant to continue working on the idea.
At that time I also moved to London as an Entrepreneur First (EF) fellow. I made good friends in the program and teamed up with Rico. We worked together for the entirety of the three month program. During that time we prototyped:
a search engine with entity, relation and quantitative options: A user could go detailed, expressive queries like:
< studies that used gene engineering [excluding selective evolution] in yeast [species A and B only] and have achieved at least [250%] in yield increase >
< studies with women over age 40, with comorbidity_A that had an upward change in biomarker_B >
They'd tell us in English and we'd translate it into our query syntax
This collapses a lot of searches and filtering into one query that isn't possible with known public search engines but we found out that it's not often in a biotechs lifecycle that questions like this need researching. To get a notion of ratios: one search like this could return enough ideas for weeks of lab work
a query builder (by demonstration): A user would upvote study abstracts or sentences that fit their requirements and it would iteratively build configuration. In a sense it was a no-code way for biologists to write interpretable labeling functions while also training a classifier (a constrained case of program synthesis)
We built this because almost no biologist could encode or tell us exactly which research they wanted to see but they "know it when I see it"
A paper recommender system (and later claim/sentence/statement level) that, in addition to academic literature, included tweets, company websites and other non-scholarly sources (lots of scientific discourse is happening on Twitter these days)
A claim explorer that takes a sentence/paragraph and lets you browse through similar claims
A claim verifier that showed sentences that confirm or contradict an entered claim sentence It worked ok-ish but the tech is not there to make fact checking work reliably, even in constraint domains
And a few more Wizard of Oz experiments to test different variants and combinations of the features above
At that point, lots of postdocs had told us that some of the apps would have saved them months during their PhD, but actual viable customers were only moderately excited. It became clear that this would have to turn into another B2B SaaS specialized tool with a lot of software consulting and ongoing efforts from a sales team ...
We definitely did not want to go down that route. We wanted to make a consumer product for consumers, startups, academics or independent researchers and had tested good-enough proxies for most ideas we thought of as useful w.r.t. the biomedical literature. We also knew that pivoting to Electronic Health Records (EHRs) or patents instead of research text was potentially a great business, but neither of us was excited to spend years working on that, even if successful.
So we were stuck. Rico, who didn't have exposure to biotech before we teamed up, understandably wanted to branch out and so we decided to build a discovery tool that we could use and evaluate the merits of ourselves.
And so we loosened up and spent three weeks playing around with knowledge search and discovery outside of biomed. We built:
a browser extension that finds similar paragraphs (in articles/essays from your browser history) to the text you're currently highlighting (when reading on the web) A third of the time the suggestions were insightful, but the download-retrain-upload loop for the embedding vectors every few days was tedious and we didn't like it enough to spend the time automating
similar: an app that pops up serendipitous connections between a corpus (previous writings, saved articles, bookmarks ...) and the active writing session or paragraph. The corpus, preferably your own, could be from folders, text files, blog archive, a Roam Research graph or a Notion/Evernote database. This was a surprisingly high signal to noise ratio from the get go and I still use it sometimes
an extension that after a few days of development was turning into Ampie and was shelved
an extension that after a few days of development was turning into Twemex and was shelved
Some of the above had promise for journalists, VCs, essayists or people that read, cite and tweet all day, but that group is too heterogeneous and fragmented and we couldn't trust our intuition building for them.
To us, these experimentsfelt mostly like gimmicks that, with more love and industry focus 1.5, could become useful but probably not essential.
Now it was almost Christmas and the EF program was over. Rico and I decided to split up as a team because we didn't have any clear next step in mind 1. I flew to the south of Portugal for a few weeks to escape London's food, bad weather and the upcoming covid surge.
There, I returned to biomed and tinkered with interactive, extractive search interfaces and no-code data programming UIs (users can design labeling functions to bootstrap datasets without coding expertise)
Ironically, I got covid in Portugal and developed scary neurological long-hauler symptoms after the acute infection. Except for a handful of ‘good days', spring and summer came and went without me being able to do meaningful cognitive work. Fortunately, this month symptoms have improved enough to finish this essay.
Fundamental Issues with Structuring Academic Literature as a Business
As I said in the beginning:
extracting, structuring or synthesizing "insights" from academic publications (papers) or building knowledge bases from a domain corpus of literature has negligible value in industry
To me the reasons feel elusive and trite simultaneously. All are blatantly obvious in hindsight.
Just a Paper, an Idea, an Insight Does Not Get You Innovation
Systems, teams and researchers matter much more than ideas. 2
The period from 1850-1900 could be described as the age of the inventor. During this period inventors were the main mechanism for creating innovations. These folks would usually sell these patents directly to large companies like the telegraph operators, railroads, or large chemical companies. The companies themselves did little R&D and primarily operated labs to test the patents that inventors brought them
But the complexity threshold kept rising and now we need to grow companies around inventions to actually make them happen:
DuPont bought the patent for creating viscose rayon (processed cellulose used for creating artificial silk and other fibers.) However, Dupont was unable to replicate the process successfully and eventually had to partner with the original inventors to get it to work
[...] making systems exponentially more valuable than single technologies
That’s why incumbents increasingly acqui-hire instead of just buying the IP and most successful companies that spin out of labs have someone who did the research as a cofounder.Technological utopians and ideologists like my former self underrate how important context and tacit knowledge is.
And even if you had that context and were perfectly set up to make use of new literature ... a significant share of actionable, relevant research findings aren’t published when they’re hot but after the authors milk them and the datasets for a sequence of derivative publications or patent them before publishing the accompanying paper (half the time behind paywalls) months later.
In the market of ideas information asymmetries turn into knowledge monopolies.
Scientists are incentivized to, and often do, withhold as much information as possible about their innovations in their publications to maintain a monopoly over future innovations. This slows the overall progress of science.
For example, a chemist who synthesizes a new molecule will publish that they have done so in order to be rewarded for their work with a publication. But in the publication they will describe their synthesis method in as minimal detail as they can while still making it through peer review. This forces other scientists to invest time and effort to reproduce their work, giving them a head-start in developing the next, better synthesis.
All that is to say: discovering relevant literature, compiling evidence, finding mechanisms turns out to be a tiny percentage of actual, real life R&D
Contextual, Tacit Knowledge is not Digital, not Encoded or just not Machine-Interpretable yet
Most knowledge necessary to make scientific progress is not online and not encoded. All tools on top of that bias the exploration towards encoded knowledge only (drunkard's search), which is vanishingly small compared to what scientists need to embody to make progress.
Expert and crowd-curated world knowledge graphs (eg. ConceptNet) can partially ground a machine learning model with some context and common sense but that is light years away from an expert’s ability to understand implicature and weigh studies and claims appropriately.
ML systems are fine for pattern matching, recommendations and generating variants but in the end great meta-research, including literature reviews, comes down to formulating an incisive research question, selecting the right studies and defining adequate evaluation criteria (metrics) 2.5.
Besides, accurately and programmatically transforming an entire piece of literature into a computer-interpretable, complete and actionable knowledge artifact remains a pipe dream.
Automatically generating complex schemas like micropubs (above) from text is many years away. Micropubs (2014), nanopubs, BEL (Biological Expression Language) and many more are all dead in my view (without admitting it)
Experts have well defined, internalized maps of their field
In industry, open literature reviews are a sporadic, non-standardized task. Professionals spend years building internal schemas of a domain and have social networks for sourcing (ie. discovery) and vetting relevant information if needed. That's why the most excited of our initial user cohorts were graduate and PhD students and life science VCs, not professionals who specialized and actively work in biotech or pharma companies.
The VC use case of doing due diligence was predominantly around legal and IP concerns rather than anything that a literature review might produce. The conceptual clearance or feedback on the idea itself was done by sending it to a referred expert in their network, not by searching the literature for contradictions or feasibility.
Scientific Publishing comes with Signaling, Status Games, Fraud and often Very Little Information
Many published papers have methodical or statistical errors, are derivative and don't add anything to the discourse, are misleading or obfuscated, sometimes even fraudulent or were just bad research to begin with. Papers are first and foremost career instruments.
A naive system will weigh the insights extracted from a useless publication equally to a seminal work. You can correct for that by normalizing on citations and other tricks but that will just mimic and propagate existing biases and issues with current scientific publishing.
For example, if I design a system that mimics the practices of an expert reader, it would result in biases towards institutions, countries, the spelling of names, clout of authors and so on. That’s either unfair, biased and unequal or it is efficient, resourceful and a reasonable response based on the reader's priors. In the end, there is no technological solution.
To push this point: if you believe in the great man theory of scientific progress, which has more merit than most want to admit, then why waste time making the other 99%+ of "unimportant" publications more searchable and discoverable? The "greats" (of your niche) will show up in your feed anyway, right? Shouldn't you just follow the best institutions and ~50 top individuals and be done with your research feed?
Why purchase access to a 3rd party AI reading engine or a knowledge graph when you can just hire hundreds of postdocs in Hyderabad to parse papers into JSON? (at a $6,000 yearly salary)
Would you invest in automation if you have billions of disposable income and access to cheap labor? After talking with employees of huge companies like GSK, AZ and Medscape the answer is a clear no.
Life science grads work for cheap, even at the post-grad level. In the UK a good bioinformatician can make 2-4 times of what non-technical lab technicians or early career biologists make. In the US the gap is even larger. The Oxford Chemistry and Biology postdocs I met during bus rides to the science park (from my time at Oxford Nanopore) earned £35k at AstraZeneca 3. That's half of what someone slightly competent earns after four months of youtubing Javascript tutorials 🤷♂️.
When we gave our pilot customers (biologists) spreadsheets with relations, mentions and paper recommendations that fit their exact requirements they were quite happy and said it saved them many hours per week. But it wasn’t a burning pain for them since their hourly wage is low either way.
I suspect the inconsequential costs of lab labor is a reason why computer aided biology (CAB) and lab automation, including cloud labs, are slow on the uptake …it can’t just be the clogging of liquid handling robots, right?
The Literature is Implicitly Reified in Public Structured Knowledge Bases
There are a ton of biomedical knowledge bases, databases, ontologies that are updated regularly 4. They are high signal because groups of experts curate and denoise them.
They're tremendously useful and will eventually make biomedical intelligent systems and reasoners easier to build. Bayer, AZ, GSK all integrate them into their production knowledge graphs and their existence makes any additional commercial attempts to extract relations from the literature less needed.
Advanced Interactives and Visualizations are Surprisingly Unsatisfying to Consume
As an artist this was the most painful lesson: Interactive graphs, trees, cluster visualizations, dendrograms, causal diagrams and what have you are much less satisfying than just lists, and most often lists will do.
On the other hand figures, plots and tables are at least as valuable as the actual text content of a paper but programs can't intelligibly process and extract their contents (too much implied context).
That alone cuts the value of tools based on extracting insights from text in half (even with perfect extraction).
I realized that when I sat next to a pro "reading" a paper. It goes like this:
Scan abstract. Read last sentence of introduction. Scan figures. Check results that discuss the figures....
which was entirely different from my non-expert approach to reading papers. It makes sense: If you don't have a strong domain model, you have no map that guides your scan, so you have to read it all top to bottom … like a computer.
Is this actually useful to experts?
This skip-and-scan selective processing also explains why agglomerative, auto-generated and compiled visualizations that incorporate a large corpus of papers are not that valuable: most of the sources would’ve been discarded up front.
Research contexts are so multiplicitous that every compiled report or visualization has huge amounts of noise and no user interface ever can be expressive enough to perfectly parameterize that context.
Unlike other Business Software, Domain Knowledge Bases of Research Companies are Maximally Idiosyncratic
Unlike other SaaS products that have standardized components (auth, storage, messaging, ...), research knowledge graphs are always designed around the domain and approach of a company and are tightly integrated with their infrastructure, proprietary datasets and IP.
Two enterprises can encode the same corpus and will produce starkly different knowledge base artifacts.
Pharmas all have their own custom knowledge graphs and have entire engineering teams working full time on keeping schemas consistent (AstraZeneca and their setup).
Incompatibilities are ontological, not technological. Knowledge representations are design decisions. That's why 3rd party knowledge graph vendors almost always have to do multi-month ongoing integration work and software consulting to make the sale. “Request a demo” often translates to “hire us for a bespoke data science project”.
To put knowledge base interoperability issues into perspective: even small database merges within the same company are still a huge problem in the software industry with no real, verifiable loss-free solution yet. Database incompatibilities have left many database administrators traumatized and knowledge bases have orders of magnitude higher schematic complexity than databases.
Divergent Tasks are Hard to Evaluate and Reason About
By “divergent” I mean loosely defined tasks where it's unclear when they're done. That includes "mapping a domain", "gathering evidence", "due diligence" and generally anything without a clear outcome like "book a hotel", "find a barber", "run this assay", "order new vials"...
It’s easy to reason about convergent tasks, like getting an answer to a factual question or checking a stock price. It’s hard to reason and quantify a divergent discovery process, like getting a map of a field or exploring a space of questions. You can't put a price tag on exploration and so it is set very low by default. Companies employ “researchers” not for reading literature, but for lab or coding work and the prestige of the “researcher” title pushes salaries down even further.
For example, take a product that offers corpus analytics:
What’s the value of getting a topic hierarchy, causal diagram or paper recommendations? It has some value if the user does not already have an operational model in his head but how often is that the case? Most biotech research jobs require a PhD level schema from the get go, so what’s the added value to a noisy AI-generated one?
When the components (claims, descriptions, papers) and tasks (evidence review, due diligence, mapping) are ambiguous it is tough to reason about returns on investment with clients. This continues inside the company: employees can defend hours in the lab much better than hours “researching”.
We’re also naturally good at diverging and auto-association which makes an in silico version of that feature less valuable. It often ends up being more effort to parse what those interfaces return than to actually, simply think.
Besides, most biotechs (before Series C) don’t have the infrastructure to try out new ideas faster than they can read them. A week in the lab can save you a day in the library6
Public Penance: My Mistakes, Biases and Self-Deceptions
Yes, I was raised catholic. How did you know?
My biggest mistake was that I didn't have experience as a biotech researcher or postdoc working in a lab. Sometimes being an outsider is an advantage, but in a field full of smart, creative people the majority of remaining inefficiencies are likely to come from incentives, politics and culture and not bad tooling.
I used to be friends with someone I consider exceptional who went on to found a biomedical cause-effect search engine (Yiannis the Co-founder of Causaly). It biased me towards thinking more about this category of products. Also, I had met employees at SciBite, a semantics company that was acquired by Elsevier for £65M, and was so unimpressed by the talent there that I was convinced that there’s lots left to do.
I wanted to make this my thing, my speciality. I had all the skills for making augmented interfaces for scientific text: machine learning, UI design and fine arts, frontend development, text mining and knowledge representation...
New intuitive query languages, no-code automation tools, new UIs for navigating huge corpi, automated reporting, cluster maps etc. where among the few applications that fit my aesthetics enough to withstand the disgusts of developing production software.
I had a good idea of the business model issues after a few weeks of talking to users, but I didn't want to stop without a first-principles explanation of why that is. Sunk-cost fallacy and a feeling of fiduciary responsibility to Emergent Ventures played a part too, but mainly because I wanted to know decisively why semantic structuring, academic search, discovery and publishing are such low-innovation zones. Looking back, I should’ve been OK with 80% certainty that it’s a lost cause and moved on.
I wanted to work on these tools because I could work with scientists and contribute something meaningful without needing to start from scratch teaching myself biology, bioinformatics and actually working in a lab. I started from nothing too many times in my life and had gotten impatient. I wanted to leverage what I already knew. But there aren’t real shortcuts and impatience made me climb a local optima.
Onwards
The initial not so modest proposal I sent to Emergent Ventures was along the lines of:
a next-generation scientific search and discovery web interface that can answer complex quantitative questions, built on extracted entities and relations from scientific text, such as causations, effects, biomarkers, quantities, methods and so on
that an hour of searching could be collapsed into a minute in many cases
All of that is still true but for the reasons I tried to share in this essay nothing of it matters.
I had to wrap my head around the fact that close to nothing of what makes science actually work is published as text on the web.
Research questions that can be answered logically through just reading papers and connecting the dots don't require a biotech corp to be formed around them. There's much less logic and deduction happening than you'd expect in a scientific discipline 5.
It's obvious in retrospect and I likely persisted for too long. I had idealistic notions of how scientific search and knowledge synthesis "should work".
I’ve been flirting with this entire cluster of ideas including open source web annotation, semantic search and semantic web, public knowledge graphs, nano-publications, knowledge maps, interoperable protocols and structured data, serendipitous discovery apps, knowledge organization, communal sense making and academic literature/publishing toolchains for a few years on and off ... nothing of it will go anywhere.
Don’t take that as a challenge. Take it as a red flag and run. Run towards better problems.
.png)
.png)
