Last December, during a debate among the Republican candidates for the U.S. presidency, Senator Ted Cruz attacked the idea that the United States should pursue regime change in Syria. If Washington tries to topple Bashar al-Assad, Cruz warned, the jihadists of the self-proclaimed Islamic State (also known as ISIS) “will take over Syria, and it will worsen U.S. national security interests.” Cruz suggested a different plan: “Instead of being a Woodrow Wilson democracy promoter, we ought to hunt down our enemies and kill ISIS rather than creating opportunities for ISIS to take control of new countries.”

Americans are often faulted for their indifference to (or ignorance of) history. Yet a disparaging reference to Wilson, who served as U.S. president a century ago, can still score points during a political campaign.

Indeed, Wilson’s ideas are particularly relevant and contentious today. In the years before Wilson came to power, many American elites subscribed to the views of the influential naval theorist and strategist Alfred Thayer Mahan, who argued that the United States should focus narrowly on keeping its borders safe, its commercial interests unchallenged, and its navy dominant while avoiding ambitious overseas interventions. Wilson ultimately came to champion a stark alternative to that view. After the catastrophe of World War I, he pronounced interest-based approaches such as Mahan’s to be morally bankrupt and instead sought to remake the world in the democratic, capitalist image of the United States. Wilson’s foreign policy ended in disappointment with the failure of his brainchild, the League of Nations. Yet as the post-9/11 era has demonstrated—and as Cruz’s dismissal highlighted—Wilsonianism is still very much alive.

The legacy of such ideas is the subject of Worldmaking: The Art and Science of American Diplomacy, a fine new history of U.S. foreign policy and diplomacy from the late nineteenth century to the present by the British diplomatic historian David Milne. “U.S. foreign policy is often best understood as intellectual history,” writes Milne. To grasp the United States’ actions in the world, Milne argues, it is essential to understand the differing philosophies, competing academic disciplines, and varied life experiences that have informed the advice its policymakers have dispensed.

Milne’s book assesses the ideas and actions of nine thinkers, from Mahan to President Barack Obama and including figures such as the journalist Walter Lippmann and the former secretary of state Henry Kissinger. The book’s careful historical accounts are rich in insight and provide extraordinary contextual breadth. But its subtitle is misleading: Milne focuses far more on foreign policy than on diplomacy, and the two are not at all the same thing. Lippmann never delivered a démarche, nor did Mahan ever approve a visa. And although the ideas of elite policymakers in Washington have undoubtedly been important in the course of U.S. history, other streams of thought and action flow outside the nation’s capital.

This is especially true now. A new, decentralized diplomacy has emerged, driven by mass communications, economic change, and the expanding global reach of institutions as varied as businesses, charities, and crime cartels. Officials and policymakers no longer have a monopoly on information. Advocacy organizations, consulting firms, and philanthropic foundations perform many traditional diplomatic functions. In the future, the beliefs and preferences of a handful of advisers and leaders will not shape American diplomacy (or foreign policy, for that matter) quite as profoundly as in previous eras.

Diplomacy has broken out of its traditional restrictions and become more creative, flexible, and democratic. It has become easier for dissidents to undermine the grand schemes of elites. Official diplomats will have to adapt to this new world if they wish to be effective in years to come.

ARTISTS AND SCIENTISTS

Worldmaking begins in 1949 with the debate over whether the United States should develop the hydrogen bomb in response to the Soviet detonation of an atomic device. George Kennan, the director of policy planning at the State Department, argued against it, drawing on history, literature, and ethics to make his recommendation; his deputy, Paul Nitze, disagreed, basing his view on “science and strategic balance.” Nitze’s advice carried the day.

Milne uses this debate to distinguish between foreign policy “artists” and foreign policy “scientists.” The artists, who include Mahan, Lippmann, Kennan, Kissinger, and Obama, are drawn primarily to literature, philosophy, and history. They are cautious in the face of complexity, dismissive of abstract theorizing, and skeptical of the extent of their power. They confront the world as it actually is.

The scientists, who include Wilson, Nitze, and former Deputy Secretary of Defense Paul Wolfowitz, favor theory, like to analyze underlying patterns rather than surface phenomena, and are more inclined to try to remake the world.

This art-versus-science dichotomy runs throughout Milne’s account, but it’s a somewhat artificial organizing device. Numerous other binaries recur—including the long-running split between idealists, who often draw inspiration from Wilson’s robust and morally engaged internationalism, and realists, who share Mahan’s belief that states should seek to maximize their power in an anarchic world. Still, Milne readily admits that no category is absolute, and one of the strengths of his book is the way it demonstrates how historical context, human nature, and the propensity of individuals to change their minds frustrate all attempts to confine policymakers to one category or another.

For instance, Kennan, a statesman commonly identified as a realist and whom Milne dubs “the consummate foreign-policy artist,” called for collective action at the end of the Cold War to rid the world of nuclear weapons and to respond to the threat of environmental devastation. During testimony before the Senate in 1989, he stated, “I was long skeptical about Wilson’s vision. . . . But I begin today . . . to think that Wilson was way ahead of his time in his views about international organization.” This was a far cry from the realism of Kennan’s famous 1947 “X” article, in which he outlined the strategy of containment.

And Milne’s categories create unlikely alliances. He groups Wilson with Nitze, who was hardly a progressive internationalist, because both sought to understand the underlying causes of historical events and applied U.S. foreign policy to remake significant parts of the world. Wilson’s belief that the United States could make fundamental changes to the structure of world affairs owed much to a conception of foreign policy as a science; so, too, did Nitze’s idea of the “correlation of forces,” the study of the ratio between the United States’ military capabilities and those of its competitors. And although figures such as Kissinger believe that history is the discipline that sheds the most light on international events, for Nitze, “too much history leads to amateurism.”

Milne’s chapter on Wolfowitz illustrates how these typologies transcend ideological lines. In Milne’s view, Wolfowitz, who served as deputy secretary of defense in the George W. Bush administration, is a “scientist” like Wilson and Nitze (he has a Ph.D. in political science), although unlike Wilson, he is no lover of collective security. But he shares Wilson’s belief that democracy is the best guarantor of stability. Milne describes how, even after the 9/11 attacks, Wolfowitz steadfastly downplayed the threat of nonstate actors such as al Qaeda and pushed for a U.S. invasion of Iraq to overthrow Saddam Hussein and create a new U.S. ally in the Middle East. Just as the United States had built lasting democracies in Germany and Japan after World War II, Wolfowitz argued, so, too, could Washington create a democratic beacon in Iraq.

Obama has gone to great lengths to reject the ideas of Wolfowitz and other Bush-era neoconservatives and to outline a more restrained view of U.S. power. In 2006, Obama stated that he would follow “a strategy no longer driven by ideology and politics but one that is based on a realistic assessment of the sobering facts on the ground and our interests in the region.” Obama recognizes that there is evil in the world, citing the theologian Reinhold Niebuhr as an influence, but argues that “we should be humble and modest in our belief we can eliminate those things.” Here was the ultimate pragmatist: reason and principle would triumph over passion, ideology, and politics.

But this has opened Obama up to the criticism that he lacks the will to employ U.S. power. The accusation, fair or not, sets Obama apart from everyone else in Milne’s book: either Obama is the only judicious character, or he is the only one who doesn’t know how to flex American muscle. And yet even Wolfowitz cheered when Obama made the “gutsy” decision to take out Osama bin Laden. The qualities that best describe Obama—his comfort with complexity, his careful examination of problems, and his ability to balance contradictions—are necessary to understand the twenty-first-century world. But it is too soon to tell whether they will be sufficient for him to be anointed a foreign policy success.

DECENTRALIZED DIPLOMACY

Milne’s accounts of some of the most important U.S. foreign-policy makers of the last century are rich and insightful. But the book focuses almost exclusively on discussions of national security, rather than the actual practice of diplomacy, which involves on-the-ground assessments of events, implementation of policy, and honest reporting, both secret and public. Milne offers a few glimpses of high-level diplomatic activity—Wilson’s meetings with other leaders at Versailles in 1919, Kissinger’s outreach to China and the Soviet Union in the 1970s—but he mostly omits the nuts and bolts of statecraft.

Moreover, with few exceptions, Milne limits his focus to Washington insiders. Yet much of the world’s diplomatic activity happens outside of capitals and beyond the confidential meetings of high officials. In private exchanges, in boardrooms, and on far-flung frontiers, other people have an impact, too. When it comes to carrying out foreign policy and dealing with other countries, officials and policymakers are not the only ones who must confront the big questions, such as whether to embrace realism or idealism or whether to seek the roots of problems or focus on their manifestations.

This is more true than ever today. A new diplomacy is emerging, as global challenges become ever more complex and diverse. Government-to-government contacts, important as they are, cannot keep up with the penetration of information, capital, and independent players into areas traditionally reserved for statesmen and diplomats.

Contemporary diplomacy is shaped by the work of nongovernmental players such as equity investors, major private donors, and nonprofit advocacy and research groups. In the past, diplomats and intelligence analysts were the most important sources of inside information about business trends abroad or power struggles in foreign governments. But today, organizations such as the strategic consulting firm McKinsey, which boasts over 100 offices all over the world and some 11,000 consultants and researchers who together speak hundreds of languages, are powerhouses of information. McKinsey and other such consulting firms make contacts, build trust, and above all provide assessments and predictions that carry the kind of authority once enjoyed almost exclusively by government officials. The last two-plus decades, since the end of the Cold War, have seen a tremendous increase in the influence of private organizations and civil society groups on relations between and among states.

McKinsey is not alone. Organizations such as the Cohen Group hire former government officials to provide clients with contacts and advice on how to get things done overseas. Private intelligence firms such as Stratfor allow clients to keep track of Turkish troop movements near Mosul or the Australian navy’s deployments in the South China Sea. Then there are the advocates, such as Human Rights Watch and Amnesty International, which can mobilize public opinion and pressure governments. And of course, there are the increasingly powerful multinational businesses, often a force unto themselves.

An ambassador in the field today sees representatives of these groups every day, and he ignores them at his peril. A wise ambassador cultivates them not only to increase his own influence but also to learn from them. He does so knowing that he is surrendering some of the mystique of high officialdom, but the payoff—deeper knowledge and denser networks—is worth it.

Of course, back channels have long been part of diplomacy. People in power have used unofficial messengers to test ideas or discreetly make proposals. Consider Benjamin Franklin in London, as he tried to raise support for American independence, or the U.S. businessman Armand Hammer in Moscow, who served as a go-between for the White House and the Kremlin in the 1970s. But such unofficial, often self-appointed middlemen used to be rare. Not anymore: today, so-called track-two talks proliferate, thanks to modern communications and the ease of travel. Retired generals talk to journalists, retired basketball players talk to dictators, and former heads of state and businesspeople pass messages to each other in the shadows. It can be hard to know who represents whom, but these conversations are not mere background noise. There are simply some suggestions that are best avoided by those whose pronouncements are official—but which can ultimately play a significant role in solving problems around the world. The Iran nuclear deal, for instance, would not have come about without well-connected private citizens and former officials making inquiries, relaying messages, and finding common ground to prepare the way for governments to talk.

The French prime minister Georges Clemenceau is said to have claimed that “war is too important to be left to the generals”; diplomacy, in the modern era, seems to have become too important to be left to the diplomats. This need not be a bad thing. Modern challenges such as climate change and migration will take the concerted efforts not just of governments but also of whole societies, and so wider society should be more involved in the diplomatic process. But diplomats will have to adapt to this changing world if they are to succeed.

Milne’s nine figures were at the center of things when the center of things was U.S. state power. But it will take a different kind of book to help the United States respond to the world in the unsettled and complex days ahead.

To read this article in Foreign Affairs, click here.