The question of China’s position and influence on the Russo-Ukrainian war has remained on the geopolitical agenda since the beginning of the war. There are official Chinese government statements that state that China encourages peace negotiations and promotes the application of international law (Ministry of Foreign Affairs of the People’s Republic of China, 2023). However, there are also geopolitical forecasts that predict that China will likely send military aid clandestinely to Russia within the next six months to prevent its defeat (Goldstein, 2023). For instance, Grey Dynamics is a British private intelligence firm that has published a six-month forecast in March, which analyses the likelihood of Chinese lethal aid to Russia. This forecast predicts that it is unlikely that China will support Russia openly with military aid similar to Western states supporting Ukraine. However, if military aid is being sent clandestinely, it will most likely be ammunition, drones, and infantry weapons. Additionally, the forecast predicts that non-lethal aid covertly sent to Russia will increase, which could lead to an extension of the war (Goldstein, 2023). There are numerous private intelligence companies that provide tailored geopolitical forecasts to clients such as individuals, businesses, or governments. In the past decade, extensive research has been conducted on how to increase the accuracy of geopolitical forecasts. This article focuses on falsifiability, quantifiability, psychological factors such as training, teaming, and tracking, and crowdsourcing.
Geopolitical forecasting is “the process of generating falsifiable predictions about future political events” (Perry World House, 2021, p. 10). For instance, the US government uses them to facilitate policymaking, as their predictions about future geopolitical developments indicate which policies should be prioritized and how resources should be allocated. These forecasts include probability and risk assessments. One important characteristic of any improved forecast is its falsifiability, which means that it can be refuted or confirmed. Falsifiable statements facilitate the evaluation process of geopolitical forecasts by keeping score of their accuracy. For instance, the statement that China is likely to send lethal aid to Russia within the next 6 months can be refuted or confirmed in 6 months. However, vague statements such as that China will be more aggressive in the next 6 months are more difficult to be proven true or false, as aggressive can be defined differently. This evaluation process can be facilitated by using quantified scorecards, as numerical performance averages of forecasters are more objective and easier to compare than qualitative feedback (Perry World House, 2021).
Quantifiability is also a key element for increasing the accuracy of geopolitical forecasts. However, a large number of foreign policy and intelligence analysts believe that the complexity of world politics and their subjective judgement make it impossible to create precise and quantified probability assessments. However, the informational value of geopolitical forecasts can be significantly improved by using “words of estimative probability” (WEPs) or numerical probabilities instead of estimative verbs and confidence levels, which are commonly used in intelligence reports. Estimative verbs include phrases such as “we estimate” or “we assess” that do not convey the level of certainty sufficiently. Confidence levels such as “low”, “moderate”, or “high” are used to convey how confident the analysts are in their judgements, but are often confused with likelihood. This ambiguity can lead to decision-makers interpreting geopolitical forecasts differently than it was intended. WEPs include spectrums of qualitative expressions that function as guidelines to convey certainty levels, figure 1 showing three examples (Friedman et al., 2015). While the WEPs present a method to improve the informational value of geopolitical forecasts using qualitative expressions, there is evidence that suggests that quantification of probability assessments would increase the predictive accuracy even further (Friedman et al., 2015). Probability assessments should be quantified by using numerical percentages or frequency representations (e.g. 1-in-10), as they become more precise (Friedman et al., 2015). Additionally, this facilitates the re-evaluation of forecasts after receiving new information. For instance, imagine that an analyst assigns a 79% chance to China sending ammunition to Russia within the next six months and receives information that China has agreed to provide Russia with lethal aid but disguised as civilian items (Hawkins, 2023). The usage of numerical percentages enables the analyst to re-evaluate their probability assessment and increase it to 82%. These incremental adjustments would not be possible when using WEPs but over time, they could turn into significant changes from the original assessment (Friedman et al., 2015).
Furthermore, the accuracy of geopolitical forecasting can be improved by three psychological factors: training, teaming, and tracking. Firstly, training includes methods to overcome cognitive biases to improve decision-making such as scenario training or probabilistic-reasoning training. Scenario training teaches geopolitical forecasters how to use decision trees and envision more possibilities. Probabilistic-reasoning training teaches forecasters when to extrapolate (extending a conclusion to an unknown situation justified by the assumption that existing trends will continue) and how to avoid overconfidence or confirmation bias (Mellers et al., 2014; Chang et al., 2016). Confirmation bias refers to the tendency to seek evidence that verifies own existing beliefs (Bell, 2021). Secondly, teaming refers to forecasters working in a group which yields better results than working independently, as groups benefit from information-sharing and encouragement. Thirdly, tracking refers to forecasters being sorted into groups of similar ability levels, which accelerates the learning process. Results of a two-year geopolitical forecasting tournament have shown that the team of the best forecasters from the first year considerably outperformed all other participating groups in the second year (Mellers et al., 2014). These individual differences in cognitive ability measured by cognitive reflection (tendency to re-evaluate an intuitive response and consider alternative solutions (Colman, 2008)) have a significant effect on forecasting accuracy (Chang et al., 2016). This proves that to a certain degree skill plays a role in geopolitical forecasting (Mellers et al., 2014).
Furthermore, the thinktank Perry World House has published a white paper that states that the geopolitical forecasting capacity of the US government can be increased with a crowdsourced geopolitical forecasting platform (Perry World House, 2021). Crowdsourcing refers to the inclusion of the general public in a project by asking them to contribute information and ideas (Chandler & Munday, 2016). According to empirical evidence that has been collected since 2010, the knowledge of the general public can be applied to national security problems. For instance, the Intelligence Advanced Research Projects Activity (IARPA) organizes tournaments, where the best forecasters performed 30% better while being limited to open-source intelligence than career intelligence analysts. Therefore, introducing an open forecasting platform as a complement to traditional forecasting methods would benefit the US government in increasing its geopolitical forecasting capacity (Perry World House, 2021).
Figure 1 – “Words of estimative probability”
From “The value of precision in geopolitical forecasting: Empirical foundations for intelligence analysis and foreign policy decision making”, by J. A. Friedman et al., 2015, NBER, Figure 1, p. 24. (https://conference.nber.org/confer/2015/SI2015/PENS/Friedman_Baker_Mellers_Tetlock_Zeckhauser.pdf)
Reference List
Bell, S. (2012). Confirmation bias. In A Dictionary of Forensic Science. Oxford University Press. Retrieved 25 Apr. 2023, from https://www.oxfordreference.com/view/10.1093/acref/9780199594009.001.0001/acref-9780199594009-e-0246
Chandler, D., & Munday, R. (2016). Crowdsourcing. In A Dictionary of Social Media. Oxford University Press. Retrieved 25 Apr. 2023, from https://www.oxfordreference.com/view/10.1093/acref/9780191803093.001.0001/acref-9780191803093-e-325
Chang, W., Chen, E., Mellers, B. A., & Tetlock, P. E. (2016). Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments. Judgment and Decision Making, 11(5), 509–526. https://doi.org/10.1017/s1930297500004599
Colman, A. (2008). Reflection-impulsivity. In A Dictionary of Psychology. Oxford University Press. Retrieved 25 Apr. 2023, from https://www.oxfordreference.com/view/10.1093/acref/9780199534067.001.0001/acref-9780199534067-e-7067
Friedman, J. A., Baker, J. D., Mellers., B. A., Tetlock, P. E., & Zeckhauser, R. (2015). The value of precision in geopolitical forecasting: Empirical foundations for intelligence analysis and foreign policy decision making [Working paper]. NBER. https://conference.nber.org/confer/2015/SI2015/PENS/Friedman_Baker_Mellers_Tetlock_Zeckhauser.pdf
Goldstein, M. (2023). Chinese lethal aid to Russia: A 6-month forecast. Grey Dynamics. https://greydynamics.com/chinese-lethal-aid-to-russia-a-6-month-forecast/
Hawkins, A. (2023, April 14). China agreed to secretly arm Russia, leaked Pentagon documents reveal. The Guardian. https://www.theguardian.com/us-news/2023/apr/14/china-agreed-secretly-arm-russia-leaked-pentagon-documents-reveal
Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E., & Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science, 25(5), 1106–1115. https://doi.org/10.1177/0956797614524255
Ministry of Foreign Affairs of the People’s Republic of China. (2023, February 24). China’s Position on the Political Settlement of the Ukraine Crisis. Government of the People’s Republic of China. https://www.fmprc.gov.cn/mfa_eng/zxxx_662805/202302/t20230224_11030713.html
Perry World House. (2021). Keeping score: A new approach to geopolitical forecasting [White paper]. University of Pennsylvania.
Home | Centuria Study Association 
Leave a comment