Why Probabilistic Population Projections Can Hardly Be Evaluated

Christina Bohk, University of Rostock
Roland Rau, University of Rostock

Population projections have a high societal impact. Valid evaluation tools are needed to analyze their methodology and outcome. Error measures have been proposed and successfully applied in the past. However, essential improvements in projection methodology-like capturing future's uncertainty in probabilistic approaches-motivate to ask whether error measures are still qualified for evaluation tasks. In this paper, we reconsider using error measures to evaluate probabilistic population projections. When evaluating a probabilistic projection, the projected result distribution should be compared with the real distribution of possible developments, though the real distribution is not observable. The unobservability of the real distribution of potential future developments leads to erroneous evaluation results when using conventional error measures. Hence, we propose to use ordinal similarity between projected and actually observed outcome as a criterion to evaluate probabilistic population projections. Ordinal similarity indicates whether fundamental development of actual fertility, mortality, and migration has been projected.