(Bloomberg Markets) — It was a bleak moment for the oil industry. U.S. shale companies were failing by the dozen. Petrostates were on the brink of bankruptcy. Texas roughnecks and Kuwaiti princes alike had watched helplessly for months as the commodity that was their lifeblood tumbled to prices that had until recently seemed unthinkable. Below $50 a barrel, then below $40, then below $30.But inside the central London headquarters of one of the world’s largest oil companies, there was an air of calm. It was January 2016. Bob Dudley had been at the helm of BP Plc for six years. He ought to have had as much reason to panic as anyone in the rest of his industry. The unflashy American had been predicting lower prices for months. He was being proved right, though that was hardly a reason to celebrate.Unlike most of his peers, Dudley was no passive observer. At the heart of BP, far removed from the sprawling network of oil fields, refineries, and service stations that the company is known for, sits a vast trading unit, combining the logistical prowess of an air traffic control center with the master-of-the-universe swagger of a macro hedge fund. And, unknown to all but a few company insiders, BP’s traders had spotted, in the teeth of the oil price collapse, an opportunity.Over the course of 2015, Dudley had acquired a reputation as the oil industry’s Cassandra. Oil prices had been under pressure ever since Saudi Arabia launched a price war against U.S. shale producers a year earlier. When crude prices started falling, he confidently predicted they would remain “lower for longer.” A few months later, he went further. Oil prices, he said, were due to stay “lower for even longer.”On Jan. 20, 2016, the price of Brent crude oil plunged to $27.10 a barrel, the lowest in more than a decade. It was a nadir that would be reached again only in March 2020, when the Saudis launched another price war, this time targeting Russia, just as the coronavirus pandemic sapped global demand.When Dudley arrived in the Swiss ski resort of Davos for the World Economic Forum on Jan. 21, 2016, the industry was braced for more doom and gloom. Wearing a dark suit and blue tie, the BP chief executive officer made his way through the snowy streets. After one meeting, he was asked—as usual—for his oil forecast by a gaggle of journalists. “Prices will remain low for longer,” he said. This time, though, his by-then-well-known mantra came with a kicker: “But not forever.”Few understood the special significance of his comment. After months of slumping oil prices, BP’s traders had turned bullish. And, in complete secrecy, the company was putting money behind its conviction.Shortly before flying to Davos, Dudley had authorized a daring trade: BP would place a large bet on a rebound in oil prices. Although its stock is in the FTSE 100 index and owned by almost every British pension fund, this wager, worth hundreds of millions of dollars, has remained a closely guarded secret until now.BP was already heavily exposed to the price of oil. What the traders wanted to do was double down, to increase the exposure by buying futures contracts much as a hedge fund would. BP’s trading arm—staffed by about 3,000 people on its main trading floors in London, Chicago, Houston, and Singapore—argued that the price had fallen so far that it could only go up. And Dudley agreed.Quietly, BP bought Brent crude futures traded in London. It was a “management position”—a trade so large it couldn’t be the responsibility of any one trader and had to be overseen by the company’s most senior executives.The optimistic coda Dudley attached to his catchphrase in Davos proved prescient. By early February, oil was up by a third, trading above $35 a barrel. By the end of May, it was more than $50 a barrel.That’s when the company started to count the profits. The trade “made a lot of money,” says a former BP executive with direct knowledge of it. Another executive, who also was involved, put the payout at about $150 million to $200 million, declining to provide an exact figure. Publicly, however, BP —whose vast size means it’s not obligated to disclose even a windfall of that scale—said almost nothing.BP’s trades in the midst of the 2016 slump are a demonstration of one of Big Oil’s best-kept secrets. The company and its rivals Royal Dutch Shell Plc and Total SE aren’t just major oil producers; they’re also some of the world’s largest commodity traders. Shell, the most active of the three, is the world’s largest oil trader—ahead of independent houses such as Vitol Group and Glencore Plc.Massive trading floors that mirror those of Wall Street’s biggest banks are becoming increasingly important to the oil companies, which are driven by fears that global oil demand could start to drop in the next few years as climate change concerns reshape society’s—and investors’—attitudes toward fossil fuel producers. No longer looked down upon as handmaidens to the engineers who built Big Oil, the traders are increasingly being seen as their companies’ saviors. The brightest stars can make more than $10 million a year, outstripping their bosses.Like BP’s 2016 trade, much about the oil majors’ trading exploits has never been reported. Bloomberg Markets pieced together the story of these lucrative but secretive operations through interviews with more than two dozen current and former traders and executives, some of which were conducted for The World for Sale, our new book on the history of commodity trading.The oil majors trade in physical energy markets, buying tankers of crude, gasoline, and diesel. And they do the same in natural gas and power markets via pipelines and electricity grids. But they do more than that: They also speculate in financial markets, buying and selling futures, options, and other financial derivatives in energy markets and beyond—from corn to metals—and closing deals with hedge funds, private equity firms, and investment banks.As little known as their trading is to the outside world, BP, Shell, and Total see it as the heart of their business. In a conference call with industry analysts last year, Ben van Beurden, CEO of Shell, described the company’s trading in almost mystical terms: “It actually makes the magic.”And the wizardry pays off: In an average year, Shell makes as much as $4 billion in pretax profit from trading oil and gas; BP typically records from $2 billion to $3 billion annually; the French major Total not much less, according to people familiar with the three companies. In the case of BP, for instance, profits can equal roughly half of what the company’s upstream business of producing oil and gas makes in a normal year, such as 2019. In years of low prices, like 2016 or 2020, trading profits can far exceed those of the production business. Last year, both BP and Shell made about $1 billion above their typical profit target in oil and gas trading.One reason profits are so high is because the three companies can reduce their trading tax bill by routing their business through low-tax jurisdictions—a strategy not available to their oil pumping and refining businesses, which are rooted in physical infrastructure in particular countries. Shell, for example, concentrates all its trading of West African and Latin American crude via a subsidiary in the Bahamas. With just 36 traders in Nassau, Shell reported profits in the Bahamas of $847.5 million in 2019. Yet it didn’t pay a single dollar in taxes on those gains.Even better for the trio, trading profits tend to soar when markets are oversupplied, as was the case in 2015-16 and again in 2020, helping to cushion the blow of low prices on the traditional business of pumping and refining oil. Trading also gives them an edge over their U.S. rivals, Exxon Mobil Corp. and Chevron Corp., which for historical and cultural reasons have eschewed trading.For most shareholders, however, the trading business is a black box. “It is impossible to show exactly what we are doing, unless we want to completely open up our entire trading book, which is something we simply cannot do,” Shell’s van Beurden said last year when asked how much money the trading unit made. Total CEO Patrick Pouyanné, asked a similar question, replied more bluntly: “The oil trading is a secret.”What isn’t a secret is the size of the trades. Together the three companies trade almost 30 million barrels a day of oil and other petroleum products, equal to the daily production of the entire OPEC cartel. Shell alone trades about 12 million barrels a day. That’s physical trading. The paper volumes are much larger. Total, for example, trades 6.9 million barrels of physical oil a day, but the equivalent of 31 million barrels of oil derivatives such as futures and options.With trading comes risk. The business “suits people who have a real commercial bent, a real desire to make money for the company,” Andrew Smith, head of trading at Shell, says in a recruiting video. They must be fearless, too: “They also have to be comfortable with taking risk. There are very few risk-free trades. Some days we make money; some days you’d lose money,” he says.BP, Shell, and Total declined to comment for this article.The history of Big Oil and trading goes back to the industry’s origins. Shell started life in London in the 19th century as an oil trader—“Shell” Transport & Trading Co.—and only later got into oil production. Then, in the first half of the 20th century, oil trading simply ceased to exist as the biggest producers squeezed others out of the picture.A few large companies came to dominate the industry, underpinned by their agreements to divvy up the oil resources of the Middle East. These companies, BP and Shell among them, were known as the Seven Sisters. Outside their oligopoly, there was very little left to buy or sell.BP was emblematic of the era. The British group had grown out of the Anglo-Persian Oil Co., established after oil was first struck in Iran in 1908, and by the early 1970s it could rely on a gusher of oil from its Iranian assets that provided much of the total 5 million barrels a day that it was pumping around the world. BP didn’t need to trade. Instead the nerve center of its business was the dull-sounding “scheduling department,” charged with arranging for BP barrels to be transported in BP tankers into BP refineries and sold into BP fuel stations.Already early traders such as Marc Rich, who founded the company that is today Glencore, were finding ways to trade oil outside the control of the Seven Sisters on the nascent spot market. The big oil companies regarded trading as beneath them and looked down on the upstarts, but they would soon be forced to think differently.The Iranian revolution of 1979 at a stroke dispossessed BP of much of its oil production. The company was forced to turn to the spot market that it had long disdained to buy the oil its refineries needed.Soon BP was doing much more than just buying oil for its own refineries. Andy Hall, then a young graduate working in its scheduling department in New York, would go on to be one of the most successful oil traders in history after leaving BP. He recalls that he started buying any oil that looked cheap, whether BP needed it or not, figuring to resell it at a profit. “We basically started trading oil like crazy,” he says.The oil price slump of the late 1990s set the stage for what the three large trading businesses would become as a wave of consolidation swept through the oil industry.When Exxon merged with Mobil, which had had a successful trading business, the nontrading culture of Exxon prevailed. The same happened when Chevron took over Texaco. The Americans were pretty much out of the trading business.Meanwhile, BP bought Amoco, which had a large trading unit, expanding its reach. The merger of French companies Total and Elf—both large traders—further consolidated Total’s trading business. Shell, too, reorganized and centralized its trading unit.By the time the wave of consolidation was over in 2000, the European trio emerged as the kings of oil trading. Their timing was exquisite: Commodity trading was about to enjoy an enormous boom as skyrocketing Chinese demand spurred a decade-long supercycle in prices. Big Oil’s trading floors would be at home at JPMorgan Chase & Co. or UBS Group AG. Rows of desks sprouting vast arrays of flashing multicolored screens stretch out almost as far as the eye can see. The traders are arranged according to their market or region of focus, each desk representing a trading “book,” a little empire of supply contracts and derivatives deals.The floors don’t just look like Wall Street’s—they’re often located alongside them. BP’s London trading base isn’t at the company’s head office near Buckingham Palace, but in the banking hub of Canary Wharf. In Chicago its traders occupy the historic floor of the former Chicago Mercantile Exchange building.All in all, BP, Shell, and Total employ about 8,000 people in their trading divisions, a small fraction of their overall workforce of 250,000. The traders have more in common with the investment bankers across the road than they do with their colleagues sweating on oil rigs in Nigeria or mapping fields off the coast of Brazil. “Trading is a very uber-competitive environment,” Christine Sullivan, a 30-year veteran of Shell trading, says in one of the company’s recruiting videos. “Every day I can see the impact I’ve made to the bottom line. You see that moving up, hopefully, on a daily basis, and it just makes you want to do more.”Big Oil’s bosses like to say that speculation isn’t part of the business model of their trading units. That’s not really true. Within BP’s trading division, for example, there was for a number of years a pot of money traded, effectively, by a computer. The so-called Q Book was devised in the 1990s by two of BP’s in-house math whizzes—Chris Allen and Gordon Izatt—long before algorithmic trading became a dominant force in financial markets.The Q Book algorithm traded dozens of commodity futures including gold and corn, according to people with knowledge of it. And while BP shut down the Q Book a few years ago, it still has a unit that resembles an in-house hedge fund: The so-called Alpha One Book, run by Tim Hayes, aims to make money betting on financial commodity markets. At Shell and Total, there are similar groups.Even so, big speculative wagers on the direction of the price of oil, like the one BP took in 2016, are rare. The day-to-day job of the traders is a little like the role of the scheduling department of bygone eras, but with a healthy dose of entrepreneurial spirit thrown in.Their role gives them a huge position in the markets and opens up all kinds of opportunities to maximize profits. Last year, for example, Shell’s traders realized that the spreading coronavirus pandemic would have a catastrophic impact on international travel. They decided to bet that demand for jet fuel would collapse. It was a wager almost no other trader in the market could make on the scale that Shell did: Jet fuel is a niche market, dominated by refineries and airlines, and the market for jet fuel derivatives isn’t liquid enough for most traders to bet on easily.But Shell was well poised. It owns the Pernis refinery in Rotterdam—the largest in Europe, each day pumping out enough gasoline, diesel, and jet fuel to keep half of the cars, trucks, and planes in the Netherlands moving. It supplies jet fuel to Amsterdam’s Schiphol Airport.In early 2020, before air travel shrank, Shell’s traders tweaked Pernis’s production, cutting out jet fuel entirely while increasing output of other refined products. Shell still had contracts to supply jet fuel, however, so the company was left with a big short position: It would have to buy jet fuel in the market to deliver to its customers, whatever the price, if the company’s traders were wrong about the pandemic. If the price went up, Shell stood to lose millions.Of course, the traders weren’t wrong. Jet fuel demand soon plunged 90% in northwestern Europe. Across Europe, prices fell from $666 a ton at the beginning of the year to $125 a ton by late April. “We could buy jet fuel, make money on that particular trade, and then again reconstitute the products coming out of the refinery to make money elsewhere,” Shell’s van Beurden explained in an earnings call with investors in July. “That’s no ordinary trading. That is actually optimizing market positions that we know better than anybody how to take advantage of.”Shell didn’t disclose how much money it made on that single trade, but people familiar with the company said that in just the second quarter of 2020, the jet fuel traders made as much as they usually do in a whole year.“Inside Shell and BP, the traders are their Navy SEALs,” says former Shell oil analyst Florian Thaler, now head of OilX, an industry data analytics company. For their skills, traders are highly paid.For years their remuneration packages were a closely guarded secret. Then in 2006 a BP trader sued the company in the U.S. in a pay dispute. The legal fight that followed exposed the riches of Big Oil trading. The trader, Alison Myers, revealed that, on top of her regular annual salary of $150,000 for 2006, she was due a $5.5 million performance bonus—three times what BP’s then-CEO John Browne took home the same year.The legal battle revealed that others at BP did even better. The company said other traders took higher bonuses not only because their desks made more money, but also because speculative traders were generally better paid. “The market value of paper traders was higher than the value of physical traders,” BP said in a court filing.Since then, bonuses have only gone up. Nowadays many traders take home from $1 million to $10 million a year, and a handful even more. Every year at BP a list goes to the board for approval. It contains the names of the dozen or so traders whose bonuses are higher than those of the CEO, according to two people familiar with the process.At the top of the list typically sits the lead trader of the Cushing Book—the one responsible for buying and selling oil at the Oklahoma town that serves as the delivery point for the West Texas Intermediate benchmark. In a good year, this trader can make as much as $30 million, an amount that would outstrip the $23 million that David Solomon, the boss of Goldman Sachs Group Inc., took home in 2019.The immense scale of the oil companies’ trading units gives them outsize clout. Shell, as Bloomberg News has reported, has in the past made bold trades that, while not illegal, have violated the unspoken rules governing this lightly regulated market. On one occasion in 2016, for example, Shell bought roughly 70% of the cargoes of North Sea crude available for a particular month, triggering wild price gyrations while squeezing out other traders who privately complained to Shell.At times, Big Oil traders have broken the rules outright. In 2007, BP paid more than $300 million to settle charges that it manipulated U.S. propane markets, for example. At the time the fine was one of largest ever for alleged market manipulation in commodities. Earlier, U.S. regulators fined Shell $300,000 for manipulating U.S. oil futures markets in 2003 and 2004 and $30 million for manipulating natural gas markets in 2000 and 2002.Still, constrained by the sheer size and high public profiles of the companies they work for, BP, Shell, and Total traders are nowhere near as swashbuckling as their counterparts at independent houses, who, history has shown, have been more willing to make a foray into countries where corruption is rife and where buying oil sometimes involves suitcases full of cash.That means the oil giants have left many of the juiciest deals to the independents. Brian Gilvary, a former BP head of finance, puts it this way: “Is there value available to us that could be captured over and above what we capture today? Absolutely. Are we prepared to take the risk associated with that? Definitely no. I can give you a list of countries, but you know where they are.”In the last few years, Big Oil has muscled more and more into the realm previously dominated by big banks. When, after the 2008-09 financial crisis, the U.S. Congress attempted to tighten regulations around the vast and opaque market for swaps—a form of bespoke derivatives traded bilaterally—the process revealed for the first time the scale of the oil companies’ role in the financial markets.The 2010 Dodd-Frank Act on financial reforms required all major players in the swaps market to register themselves. There were the usual suspects: Bank of America, Goldman Sachs, JPMorgan, and other financial behemoths. And then there were three names that seemed out of place: Cargill, the world’s largest trader of agricultural commodities, BP, and Shell.As Wall Street banks scaled back their presence in commodities in the post-crisis world, Big Oil stepped in. Shell, for example, in 2016 became the first nonbank to move in on what commodity traders at Wall Street banks see as their largest annual deal: helping the Mexican government hedge its exposure to the price of oil.For its part, BP, in a brochure for its trading unit, says, “Our customers also include banks, hedge funds and private equity firms.” The document lists a range of financial strategies it can help customers implement—from “options (vanilla & tailored)” to “tiered volume restructure.”With investors of all kinds increasingly unimpressed by the traditional oil-pumping business, trading is becoming an ever more important part of the oil companies’ sales pitch. In a virtual meeting with investors in October 2020, Shell’s van Beurden described the company’s trading unit as “absolutely core to the success of our company.” Even Exxon, which long sneered at trading as an unnecessary distraction, has changed its stance, hiring experienced oil traders to start making bets with the company’s money.As BP shifts its investments from fossil fuels to renewable energy, its traders will help it juice the relatively low returns on those investments, Bernard Looney, who last year succeeded Dudley as CEO, said in a presentation to investors in 2020. Renewable energy projects typically generate returns of 5% to 6%, he said, but the company’s expert traders can add about 2 percentage points to that.As steeped as BP may seem to be in the rigs and offshore platforms and snaking pipelines of yesteryear, Looney painted an energy future that encompasses electric cars, hydrogen, and biofuels. “We love complexity like this,” he said. “It is why we have elevated our trading function to the leadership table.”Blas and Farchy cover energy out of London. Their book, The World for Sale: Money, Power, and the Traders Who Barter the Earth’s Resources, was published in the U.K. in February by Random House Business and in the U.S. in March by Oxford University Press. For more articles like this, please visit us at bloomberg.comSubscribe now to stay ahead with the most trusted business news source.©2021 Bloomberg L.P.