Measuring Cross-Border Perspectives… Challenges in Implementation, Ease of Discovering Mistakes, and the Sweetness of Results!
By Dr. Samir Abu Rumman
After more than a decade of experience in conducting and supervising global opinion research projects, and indices for various countries, such as the Arab Barometer project for Princeton University, the World Values Survey (WVS), the International Social Survey Programme (ISSP) for NORC at the University of Chicago, and others, I can say that implementing such complex multi-national projects comes with methodological, political, and scientific challenges despite the benefits in meticulous planning, attention to detail, and a deep understanding of different cultures and societies. I will try to elucidate why in the following paragraphs, distinguishing between these ongoing cross-border multinational projects and those conducted within a single country, perhaps once, what they call “a stand-alone project.”
One of the main differences between global surveys and those conducted in a single country is the lack of temporal data, regional analyses, and the inability to directly compare neighbouring countries, which poses a challenge in ensuring the quality and accuracy of the collected data. This challenge can be likened to opinion polls related to election predictions, where results quickly emerge and reveal accuracy. Thus, in cross-country opinion surveys, their execution is challenging, while discovering their mistakes is easy!
Although, as a pollster and producer of numbers, it is not necessary for me to explain my survey results or justify them, I do not forget my experience with the World Values Survey (WVS) in its sixth wave, where I had to study Kuwaiti responses with extreme care and compare them with the results of other Arab countries. Sometimes, I found it necessary to share some of the results to measure the opinions of local experts or even ordinary citizens. I remember well that I had to explain some differences in the questions, such as explaining why Kuwaitis had relatively less trust in others. I had to delve into some ideas related to historical events, experience, knowledge levels, and culture, which might make Kuwaitis less trusting of others than their counterparts elsewhere. In the same context, I once asked a senior official at the Pew Research Center in Washington about how to deal with global results and compare them with others. At the time, he mentioned another level of challenge and quality control issues when comparing their international results with those of Gallup, which share some questions, and responses in more than a hundred countries within their Gallup World Poll project.
Furthermore, the temporal and social diversity among countries adds another layer of complexity to international multi-country survey projects. Even among countries that share a common language, such as Arab countries, differences in the formulation and interpretation of survey questions may arise. To overcome these challenges, we use professional translation techniques, including back translation. Sometimes, some requesting entities for the survey stipulate the use of high-quality translation services for international surveys across multiple languages and countries.
These diverse ideas were part of a discussion with the team at D3 Systems in Virginia recently. They conduct hundreds of international opinion surveys worldwide, originating from the United States. I mentioned to them the struggles and challenges of implementing cross-border opinion survey projects. Still, at the same time, I reminded them of the pleasure of testing the comparative results and understanding the opinions of one nation with those of other nations worldwide, with their percentage representations. As a researcher pointed out, “Our understanding of opinions expands more than our haircuts or those of our relatives!”
Samir Abu-Rumman, PhD, is a visiting research scholar at Princeton University, USA, with extensive years of experience in research, education, and development in different countries. He is the supervisor of “World of Opinions” in Kuwait, Jordan, and the U.S., has led and supervised different regional and global research projects for organizations such as the World Values Survey and Arab Barometer for Princeton University. He is also one of the keynote panelists at Generation1.ca’s upcoming Virtual Insights Career Fair and Case Competition.