July 19, 2019
July 19, 2019

Opinion

Opinion on the latest North Korea issues from analysts, academics and experts worldwide. Views expressed in Opinion articles are exclusively the authors’ own.

The perils and pitfalls of North Korean opinion polls
by nknews_hq on 2018-01-10 04:20:49
The perils and pitfalls of North Korean opinion polls
A recent survey of in-country views on the nuclear program must be taken with a pinch of salt
March 20th, 2018

Month in Review

Last month, the Center for Strategic and International Studies (CSIS) published a report that, among other things, tells us what North Koreans think about their nuclear weapons program. The report is based on a survey conducted inside the DPRK, with fifty regular people asked their opinions on the country’s nuke development.

Their responses, the report says, were overwhelmingly critical of the government: 43 out of 50 expressed “highly negative” views on the North Korean nuclear weapons program, and 70% said that they do not see nuclear weapons as a source of national pride. One of the respondents is even quoted as saying that “nuclear weapons are the devil’s weapons and will lead to our extinction.”

So, we can feel great relief, can’t we? As the survey suggests, North Koreans are closet peaceniks, and their country’s nuclear weapons arsenal is despised, even hated, by the people.

They will, then, not feel sorry about denuclearization, and will even applaud such a decision if it is made by their government. After all, this is now confirmed by a survey that surely relies on allegedly scientific methods: the moment there is a box to tick is the moment when the opinion of a small group becomes a fact, it seems.

But it is clearly not as simple as all that. The questions that social scientists ask matter, who they ask, how they ask, and how they find the people they ask all matter too. Let’s take a quick look at these issues.

IMPLICIT BIAS?

The survey was conducted in North Korea, and this alone raises manifold issues – many of which, to be fair, are acknowledged by CSIS in the “methodology” section of the report. The people who were found by the researchers – or, more likely, by their representatives – clearly knew that they were participating in a highly risky endeavor sponsored by a foreign organization.

This alone ensures massive “self-selection bias,” as it is known in survey methodology: people willing to participate in such an undertaking are also likely to have strong anti-regime tendencies, akin to going to UC Berkeley to survey students about Donald Trump and then claiming that the views are representative of the U.S. electorate.

The survey was conducted in North Korea, and this alone raises manifold issues

Additionally, most of the respondents were very likely somehow involved with cross-border trade with China or other commercial activities of various kinds – otherwise they would be far more difficult to locate and approach. This raises the problem of “non-selection bias” – the people who were not selected, and whose views are not reflected in the survey results.

The survey participants, given the circumstances, knew what the researchers wanted them to say, and they were likely to oblige to these perceived demands. This is called “reporting bias.” Subjects report what they know they are supposed to, what they believe to be right, and/or what they believe they have to.

One can say, presumably, that answers to the survey were self-administered, so the survey organizers could not possibly know which answers were given by a participant.

This is a standard precaution taken in surveys of a sensitive nature. However, North Korean survey subjects were almost certain to assume that their answers will be somehow be connected to their names by the researchers – making many more reluctant to answer and further narrowing the diversity of opinion available.

Having grown up in the Soviet Union, a far less repressive place than North Korea, I remember well how we never believed opinion polls could be anonymous, and how my parents instructed me to be careful with what I wrote in the ostensibly unsigned questionnaires, since “they will find it out anyway.”

soldiers photo

The survey uses too small a sample size to be truly accurate | Photo by nknews_hq

Let’s also not forget that the sample size is very small – so small is it described as a “micro-survey.” A non-randomly selected sample of 50 individuals, likely from a highly specific sub-population – in a country of almost 20 million adults tells us pretty much nothing about public opinion in that country.

It is what it is: the opinions of 50 people predisposed to be critical of the state. 

PR-FRIENDLY DATA

However, such a survey is going to attract much attention, for a simple reason: for nearly a century we have lived in a world where proper knowledge is expected to be quantitative and presentable in statistical form.

Figures win more trust than impressions which are bound to be treated as “anecdotal evidence” at best. It does not always matter that figures are remarkably unreliable, and are nothing but the nice packaging of what is guesswork at best and wishful thinking at worst.

Frankly, this author’s experiences with non-refugee North Koreans tell a story that is very different from what the survey says. A significant majority of North Koreans I have encountered in third countries were clearly supportive of the DPRK nuclear weapons.

Interestingly, among those nuclear enthusiasts, there were people who were willing to express critical, even highly critical, opinions of the North Korean government and system.

These dissident views did not prevent them from seeing nukes as a necessary, if expensive, way to safeguard their country’s security in an uncertain and dangerous world, populated by large predatory powers (not only the U.S., but also China).

North Koreans, like many other people in the world, would like their tribe to be strong

There is little doubt that the CSIS survey organizers wanted to produce ‘scientific’ knowledge about a difficult-to-access locality. However, their efforts, no matter how well intended, might produce dangerous results. 

When we are dealing with North Korea, we tend to talk to the people who are willing to talk to us, and who are often eager to produce answers that they believe we expect. Then we take these answers as a representative and authentic voice of the North Korean society. This approach makes us overestimate the popularity of, and support for, what can be described as “anti-system views,” and such overestimation might be dangerous.

We have seen how such wishful thinking led major Western powers in self-delusion about Iraq and Libya. In both countries, it was assumed that the local dictator was unpopular with the majority of the population, who supposedly just wanted to live in a liberal democracy. The results of the dictator’s downfall brought a rude awakening, however.

The common people might have despised Saddam or Gaddafi, but many of these regimes’ assumptions, fears, and ambitions were shared by the general population.

Regime collapse did not lead to the once-anticipated triumph of liberal democracy, but rather triggered civil wars which are likely to continue for years to come, and whose participants have values vastly different from values of liberal professors at East Coast universities.

Something similar might happen in the DPRK if claims about widespread dislike for nuclear weapons are going to be taken too seriously. North Koreans, like many other people in the world, would like their tribe to be strong.

Edited by Oliver Hotham

Featured Image: by nknews_hq on 2018-01-10 04:20:49

Get North Korea headlines delivered to your inbox daily

Subscribe to the NK News 'Daily Update' and get links to must-read stories each morning

Skip to toolbar