Investigating the Ukrainian Refugee Crisis¶

by: Luke Volpe¶

UKRrefugees.jpeg

Link to project site here

Project Goals¶

Human migration is one of the defining global trends of the 21st century, whether it is political, economic, or conflit-driven. The goal of this project is to investigate patterns in the movement of Ukrainian refugees as a result of the February 2022 Russian invasion. I will also be analyzing the relationships between refugee flows and host country indicators, with the final goal of identifying the political and economic indicators that are most predictive of having larger refugee populations.

The Ukrainian refugee crisis is the fastest growing and largest displacement crisis in Europe since WWII (UNHCR). Political institutional quality and economic strength are big determinants of integration success, and knowing which indicators are most related to refugee flows can help governments and aid agencies help refugees more effectively.

Collaboration Plan (N/A)¶

As a Master's student, I am working alone and will not need a specialized plan.

ETL: Project Dataset 1¶

The first dataset that I will use comes from the Refugee Data Finder at the United Nations High Commissioner for Refugees (UNHCR), the leading international institution on migration data. This database contains yearly population counts of refugee, asylum-seeker, and internally displaced persons, as well as their countries of origin and asylum. I am interested in this dataset because it is the most basic form of tracking refugee migration destinations. I will use it to answer questions like "Which countries have received the largest volumes of refugees?" or "Have refugee populations increased, decreased, or stabilized over the course of the war?"

In [39]:
%cd /content/drive/MyDrive/CMPS 6160 (Data Sci)/Proj Data
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
/content/drive/MyDrive/CMPS 6160 (Data Sci)/Proj Data
In [40]:
df_RefCounts_Orig = pd.read_csv('UKR Refugee Counts.csv')
df_RefCounts_Orig
Out[40]:
Year Country of Asylum Country of Origin Country of Asylum ISO Country of Origin ISO Refugees Asylum-seekers IDPs Other people in need of international protection
0 2022 Albania Ukraine ALB UKR 2542 5 0 0
1 2023 Albania Ukraine ALB UKR 4959 0 0 0
2 2024 Albania Ukraine ALB UKR 9250 0 0 0
3 2022 Egypt Ukraine EGY UKR 0 5 0 0
4 2023 Egypt Ukraine EGY UKR 0 9 0 0
... ... ... ... ... ... ... ... ... ...
236 2023 Uruguay Ukraine URY UKR 18 0 0 0
237 2024 Uruguay Ukraine URY UKR 18 5 0 0
238 2022 United States of America Ukraine USA UKR 3063 12369 0 0
239 2023 United States of America Ukraine USA UKR 3299 17773 0 0
240 2024 United States of America Ukraine USA UKR 3397 18658 0 0

241 rows × 9 columns

Because we are only studying refugee populations originating from Ukraine and going to other countries, we can drop the variables about country of origin and drop the three observations about Ukraine itself. I will also drop the "Other people in need of international protection" and "IDPs" columns because they contain no relevant data to my study and are filled only with zeros. I will also rename some of the asylum countries for convenience sake. Lastly, I will create a joint measure of refugees and asylum seekers. Since the distinction between the two classifications is primarily legal, if we want to know generally about all the people the war has displaced, we should use a combined count.

In [41]:
# Dropping variables and UKR observations:
df_RefCounts = df_RefCounts_Orig.drop(columns=['Country of Origin', 'Country of Origin ISO', 'Other people in need of international protection', 'IDPs'])
df_RefCounts = df_RefCounts[df_RefCounts['Country of Asylum ISO'] != 'UKR']

# Renaming countries:
df_RefCounts["Country of Asylum"] = df_RefCounts["Country of Asylum"].replace({
    'Bolivia (Plurinational State of)': 'Bolivia',
    'United Kingdom of Great Britain and Northern Ireland': 'United Kingdom',
    'United States of America': 'United States',
    'Rep. of Korea': 'South Korea',
    'Rep. of Moldova': 'Moldova',
    'Netherlands (Kingdom of the)': 'Netherlands',
    'Russian Federation': 'Russia',
    'Serbia and Kosovo: S/RES/1244 (1999)': 'Serbia',
    'United Rep. of Tanzania': 'Tanzania'
})
#Joint measure
df_RefCounts['Ref_Asylum'] = df_RefCounts['Refugees'] + df_RefCounts['Asylum-seekers']
df_RefCounts
Out[41]:
Year Country of Asylum Country of Asylum ISO Refugees Asylum-seekers Ref_Asylum
0 2022 Albania ALB 2542 5 2547
1 2023 Albania ALB 4959 0 4959
2 2024 Albania ALB 9250 0 9250
3 2022 Egypt EGY 0 5 5
4 2023 Egypt EGY 0 9 9
... ... ... ... ... ... ...
236 2023 Uruguay URY 18 0 18
237 2024 Uruguay URY 18 5 23
238 2022 United States USA 3063 12369 15432
239 2023 United States USA 3299 17773 21072
240 2024 United States USA 3397 18658 22055

238 rows × 6 columns

The data above is in a basic tidy format. While proper database management would create a system of relational tables, this will suffice for our purposes. Next, we must check the dtypes. We see that they are already correctly coded, with our categorical variables as objects and our quantitative variables as int64:

In [42]:
df_RefCounts.dtypes
Out[42]:
0
Year int64
Country of Asylum object
Country of Asylum ISO object
Refugees int64
Asylum-seekers int64
Ref_Asylum int64

ETL: Project Dataset 2¶

My next dataset comes from the World Bank's World Development Indicators (WDI). This database offers more than 1,500 different variables at the national level and covers broad topics like the environment, education, and inequality, to name just a few. I used their built-in database query interface to select variables focusing on my outcomes of interest, primarily those related to the economy, education, and demography. I also narrowed the database down to the three years of interest (2022-2024). Later on, I will conduct a left join on country and year using the df_RefCounts as df1, so it only keeps country/year combinations that have recorded Ukrainian refugee data.

With this data, I hope to build contextual profiles for each country of asylum. It will help me answer questions like, "What are the strongest correlates of refugee population in the context of this conflict?", and "How much of the refugee population is absorbed by countries with high vs. low levels of development?"

In [43]:
df_WDI_Orig = pd.read_csv('WDI 1.csv')
df_WDI_Orig
Out[43]:
Country Name Country Code Series Name Series Code 2022 [YR2022] 2023 [YR2023] 2024 [YR2024]
0 Afghanistan AFG Adjusted net enrollment rate, primary (% of pr... SE.PRM.TENR .. .. ..
1 Afghanistan AFG Adjusted net national income per capita (curre... NY.ADJ.NNTY.PC.CD .. .. ..
2 Afghanistan AFG Coverage of social safety net programs (% of p... per_sa_allsa.cov_pop_tot .. .. ..
3 Afghanistan AFG Educational attainment, at least Bachelor's or... SE.TER.CUAT.BA.ZS 3.98425006866455 .. ..
4 Afghanistan AFG Educational attainment, at least completed pri... SE.PRM.CUAT.ZS 21.0254802703857 .. ..
... ... ... ... ... ... ... ...
10416 NaN NaN NaN NaN NaN NaN NaN
10417 NaN NaN NaN NaN NaN NaN NaN
10418 NaN NaN NaN NaN NaN NaN NaN
10419 Data from database: World Development Indicators NaN NaN NaN NaN NaN NaN
10420 Last Updated: 10/07/2025 NaN NaN NaN NaN NaN NaN

10421 rows × 7 columns

Upon loading in this table, we see we must reshape it from wide to long format. First, let's melt the years data into one column, clean up the syntax of the Year column, and then pivot the table so each indicator is its own column.

In [44]:
# Melt
df_WDI_melt = df_WDI_Orig.melt(
    id_vars=['Country Name', 'Country Code', 'Series Name', 'Series Code'],
    var_name='Year',
    value_name='Value'
)
# Replace year values to clean syntax and convert to int64s
df_WDI_melt['Year'] = df_WDI_melt['Year'].replace({
    '2022 [YR2022]': '2022',
    '2023 [YR2023]': '2023',
    '2024 [YR2024]': '2024'
}).astype(int)
In [45]:
# Before pivoting must drop the footnote observations at the end.
display(df_WDI_melt.tail(6))
df_WDI_melt = df_WDI_melt.iloc[:31258]
Country Name Country Code Series Name Series Code Year Value
31257 Zimbabwe ZWE Unemployment, male (% of male labor force) (na... SL.UEM.TOTL.MA.NE.ZS 2024 ..
31258 NaN NaN NaN NaN 2024 NaN
31259 NaN NaN NaN NaN 2024 NaN
31260 NaN NaN NaN NaN 2024 NaN
31261 Data from database: World Development Indicators NaN NaN NaN 2024 NaN
31262 Last Updated: 10/07/2025 NaN NaN NaN 2024 NaN

I was still getting a Duplicate Index Entries error when pivoting, so let's search for and drop duplicates:

In [46]:
dupes = df_WDI_melt[df_WDI_melt.duplicated(
    subset=['Country Name', 'Country Code', 'Year', 'Series Name'],
    keep=False
)]
print(dupes)
df_WDI_melt = df_WDI_melt.drop(index=dupes.index)
      Country Name Country Code Series Name Series Code  Year Value
10416          NaN          NaN         NaN         NaN  2022   NaN
10417          NaN          NaN         NaN         NaN  2022   NaN
10418          NaN          NaN         NaN         NaN  2022   NaN
20837          NaN          NaN         NaN         NaN  2023   NaN
20838          NaN          NaN         NaN         NaN  2023   NaN
20839          NaN          NaN         NaN         NaN  2023   NaN
In [47]:
# Now pivot
df_WDI_pivot = df_WDI_melt.pivot(
    index=['Country Name', 'Country Code', 'Year'],
    columns='Series Name',
    values='Value'
).reset_index()
# Drop the column labeled NaN
df_WDI_pivot.drop(columns=[np.nan], inplace=True)
df_WDI_pivot
Out[47]:
Series Name Country Name Country Code Year Adjusted net enrollment rate, primary (% of primary school age children) Adjusted net national income per capita (current US$) Coverage of social safety net programs (% of population) Educational attainment, at least Bachelor's or equivalent, population 25+, total (%) (cumulative) Educational attainment, at least completed primary, population 25+ years, total (%) (cumulative) Educational attainment, at least completed upper secondary, population 25+, total (%) (cumulative) GDP growth (annual %) ... School enrollment, secondary (% net) School enrollment, tertiary (% gross) Unemployment with advanced education (% of total labor force with advanced education) Unemployment with basic education (% of total labor force with basic education) Unemployment, female (% of female labor force) (modeled ILO estimate) Unemployment, female (% of female labor force) (national estimate) Unemployment, male (% of male labor force) (modeled ILO estimate) Unemployment, male (% of male labor force) (national estimate) Unemployment, total (% of total labor force) (modeled ILO estimate) Unemployment, total (% of total labor force) (national estimate)
0 Afghanistan AFG 2022 .. .. .. 3.98425006866455 21.0254802703857 12.5973300933838 -6.24017199240269 ... .. .. .. .. 26.742 .. 13.168 .. 14.1 ..
1 Afghanistan AFG 2023 .. .. .. .. .. .. 2.26694373649188 ... .. .. .. .. 26.57 .. 13.066 .. 13.991 ..
2 Afghanistan AFG 2024 .. .. .. .. .. .. .. ... .. .. .. .. 25.636 .. 12.394 .. 13.295 ..
3 Albania ALB 2022 .. .. .. 19.8346436163563 97.8531397166699 53.2684854264147 4.82669611912765 ... .. 62.647921513098 10.228 9.437 10.405 11.235 9.917 10.398 10.137 10.785
4 Albania ALB 2023 .. .. .. 17.5100002288818 99.2799987792969 53.9700012207031 3.93661689192599 ... .. 64.7293520237257 8.947 9.694 10.376 10.958 9.888 10.421 10.108 10.669
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
650 Zambia ZMB 2023 .. .. .. 2.80119454477244 .. 33.5884544067257 5.36794285493059 ... .. .. 9.702 3.688 6.207 6.037 5.646 5.71 5.905 5.861
651 Zambia ZMB 2024 .. .. .. .. .. .. 4.0431482994443 ... .. .. .. .. 6.25 .. 5.713 .. 5.961 ..
652 Zimbabwe ZWE 2022 .. .. .. 0.913877996562985 .. .. 6.1391761744749 ... .. .. 5.945 8.878 10.562 10.461 9.615 9.554 10.087 10.001
653 Zimbabwe ZWE 2023 .. .. .. 1.13487348253901 .. .. 5.33677037339527 ... .. .. 5.933 8.493 8.824 9.46 8.694 9.126 8.759 9.29
654 Zimbabwe ZWE 2024 .. .. .. .. .. .. 2.02948414337123 ... .. 7.74912957845971 .. .. 8.587 .. 8.521 .. 8.554 ..

655 rows × 51 columns

Shape looks good and footnotes are gone. Now we can check our dtypes for the WDI dataset, but it doesn't look good. All of our variables besides Country Name and Country Code should be numeric:

In [48]:
df_WDI_pivot.dtypes
Out[48]:
0
Series Name
Country Name object
Country Code object
Year int64
Adjusted net enrollment rate, primary (% of primary school age children) object
Adjusted net national income per capita (current US$) object
Coverage of social safety net programs (% of population) object
Educational attainment, at least Bachelor's or equivalent, population 25+, total (%) (cumulative) object
Educational attainment, at least completed primary, population 25+ years, total (%) (cumulative) object
Educational attainment, at least completed upper secondary, population 25+, total (%) (cumulative) object
GDP growth (annual %) object
GDP per capita (current US$) object
GDP per capita growth (annual %) object
GNI per capita growth (annual %) object
GNI per capita, Atlas method (current US$) object
Gini index object
Government Effectiveness: Percentile Rank object
Labor force participation rate for ages 15-24, total (%) (modeled ILO estimate) object
Labor force participation rate for ages 15-24, total (%) (national estimate) object
Labor force participation rate, total (% of total population ages 15+) (modeled ILO estimate) object
Labor force participation rate, total (% of total population ages 15+) (national estimate) object
Labor force participation rate, total (% of total population ages 15-64) (modeled ILO estimate) object
Labor force with advanced education (% of total working-age population with advanced education) object
Labor force with basic education (% of total working-age population with basic education) object
Labor force with intermediate education (% of total working-age population with intermediate education) object
Labor force, female (% of total labor force) object
Net official flows from UN agencies, UNHCR (current US$) object
Political Stability and Absence of Violence/Terrorism: Percentile Rank object
Population ages 15-64 (% of total population) object
Population ages 15-64, female object
Population ages 15-64, male object
Population ages 15-64, total object
Population growth (annual %) object
Population, female (% of total population) object
Population, male (% of total population) object
Poverty headcount ratio at national poverty lines (% of population) object
Prevalence of moderate or severe food insecurity in the population (%) object
Primary completion rate, total (% of relevant age group) object
Rule of Law: Percentile Rank object
School enrollment, primary (% gross) object
School enrollment, primary (% net) object
School enrollment, secondary (% gross) object
School enrollment, secondary (% net) object
School enrollment, tertiary (% gross) object
Unemployment with advanced education (% of total labor force with advanced education) object
Unemployment with basic education (% of total labor force with basic education) object
Unemployment, female (% of female labor force) (modeled ILO estimate) object
Unemployment, female (% of female labor force) (national estimate) object
Unemployment, male (% of male labor force) (modeled ILO estimate) object
Unemployment, male (% of male labor force) (national estimate) object
Unemployment, total (% of total labor force) (modeled ILO estimate) object
Unemployment, total (% of total labor force) (national estimate) object

In [49]:
# Select numeric columns
numeric_cols = df_WDI_pivot.columns.difference(['Country Name', 'Country Code'])
# Force to numeric
df_WDI_pivot[numeric_cols] = df_WDI_pivot[numeric_cols].apply(pd.to_numeric, errors='coerce')
# Also, fill blank values with NaN instead of ".."
df_WDI_pivot.replace('..', np.nan, inplace=True)

display(df_WDI_pivot.dtypes)
#Much better!
0
Series Name
Country Name object
Country Code object
Year int64
Adjusted net enrollment rate, primary (% of primary school age children) float64
Adjusted net national income per capita (current US$) float64
Coverage of social safety net programs (% of population) float64
Educational attainment, at least Bachelor's or equivalent, population 25+, total (%) (cumulative) float64
Educational attainment, at least completed primary, population 25+ years, total (%) (cumulative) float64
Educational attainment, at least completed upper secondary, population 25+, total (%) (cumulative) float64
GDP growth (annual %) float64
GDP per capita (current US$) float64
GDP per capita growth (annual %) float64
GNI per capita growth (annual %) float64
GNI per capita, Atlas method (current US$) float64
Gini index float64
Government Effectiveness: Percentile Rank float64
Labor force participation rate for ages 15-24, total (%) (modeled ILO estimate) float64
Labor force participation rate for ages 15-24, total (%) (national estimate) float64
Labor force participation rate, total (% of total population ages 15+) (modeled ILO estimate) float64
Labor force participation rate, total (% of total population ages 15+) (national estimate) float64
Labor force participation rate, total (% of total population ages 15-64) (modeled ILO estimate) float64
Labor force with advanced education (% of total working-age population with advanced education) float64
Labor force with basic education (% of total working-age population with basic education) float64
Labor force with intermediate education (% of total working-age population with intermediate education) float64
Labor force, female (% of total labor force) float64
Net official flows from UN agencies, UNHCR (current US$) float64
Political Stability and Absence of Violence/Terrorism: Percentile Rank float64
Population ages 15-64 (% of total population) float64
Population ages 15-64, female float64
Population ages 15-64, male float64
Population ages 15-64, total float64
Population growth (annual %) float64
Population, female (% of total population) float64
Population, male (% of total population) float64
Poverty headcount ratio at national poverty lines (% of population) float64
Prevalence of moderate or severe food insecurity in the population (%) float64
Primary completion rate, total (% of relevant age group) float64
Rule of Law: Percentile Rank float64
School enrollment, primary (% gross) float64
School enrollment, primary (% net) float64
School enrollment, secondary (% gross) float64
School enrollment, secondary (% net) float64
School enrollment, tertiary (% gross) float64
Unemployment with advanced education (% of total labor force with advanced education) float64
Unemployment with basic education (% of total labor force with basic education) float64
Unemployment, female (% of female labor force) (modeled ILO estimate) float64
Unemployment, female (% of female labor force) (national estimate) float64
Unemployment, male (% of male labor force) (modeled ILO estimate) float64
Unemployment, male (% of male labor force) (national estimate) float64
Unemployment, total (% of total labor force) (modeled ILO estimate) float64
Unemployment, total (% of total labor force) (national estimate) float64

Due to variation in data availability and measurement, it is common for some World Development Indicators to contain no data. Here, we'll drop any variables that contain no information for any of our country-year observations. We will also recode the burdensome variable names exported from the WDIs.

In [50]:
#Drop columns that have NaN for all values
df_WDI_pivot.dropna(axis=1, how='all', inplace=True)

#Rename using dictionary
rename_dict = {
    'Country Name': 'Country Name',
    'Country Code': 'ISO',
    'Year': 'Year',
    'Coverage of social safety net programs (% of population)': 'SocialNet_Cov',
    "Educational attainment, at least Bachelor's or equivalent, population 25+, total (%) (cumulative)": 'Edu_Bach25p',
    'Educational attainment, at least completed primary, population 25+ years, total (%) (cumulative)': 'Edu_Prim25p',
    'Educational attainment, at least completed upper secondary, population 25+, total (%) (cumulative)': 'Edu_UpSec25p',
    'GDP growth (annual %)': 'GDP_Growth',
    'GDP per capita (current US$)': 'GDP_pc',
    'GDP per capita growth (annual %)': 'GDP_pc_Growth',
    'GNI per capita growth (annual %)': 'GNI_pc_Growth',
    'GNI per capita, Atlas method (current US$)': 'GNI_pc',
    'Gini index': 'Gini',
    'Government Effectiveness: Percentile Rank': 'GovEffect_rank',
    'Labor force participation rate for ages 15-24, total (%) (modeled ILO estimate)': 'LFPR_15_24_ILO',
    'Labor force participation rate for ages 15-24, total (%) (national estimate)': 'LFPR_15_24_Nat',
    'Labor force participation rate, total (% of total population ages 15+) (modeled ILO estimate)': 'LFPR_15p_ILO',
    'Labor force participation rate, total (% of total population ages 15+) (national estimate)': 'LFPR_15p_Nat',
    'Labor force participation rate, total (% of total population ages 15-64) (modeled ILO estimate)': 'LFPR_15_64_ILO',
    'Labor force with advanced education (% of total working-age population with advanced education)': 'LF_AdvEdu',
    'Labor force with basic education (% of total working-age population with basic education)': 'LF_BasicEdu',
    'Labor force with intermediate education (% of total working-age population with intermediate education)': 'LF_IntEdu',
    'Labor force, female (% of total labor force)': 'LF_Fem',
    'Net official flows from UN agencies, UNHCR (current US$)': 'Net_UNHCR',
    'Political Stability and Absence of Violence/Terrorism: Percentile Rank': 'Pol_Stab_rank',
    'Population ages 15-64 (% of total population)': 'Pop_15_64_pct',
    'Population ages 15-64, female': 'Pop_15_64_Fem',
    'Population ages 15-64, male': 'Pop_15_64_Male',
    'Population ages 15-64, total': 'Pop_15_64_Tot',
    'Population growth (annual %)': 'Pop_Growth',
    'Population, female (% of total population)': 'Pop_Fem_pct',
    'Population, male (% of total population)': 'Pop_Male_pct',
    'Poverty headcount ratio at national poverty lines (% of population)': 'Poverty_pct',
    'Prevalence of moderate or severe food insecurity in the population (%)': 'FoodInsec_pct',
    'Primary completion rate, total (% of relevant age group)': 'PrimComp_pct',
    'Rule of Law: Percentile Rank': 'RuleLaw_rank',
    'School enrollment, primary (% gross)': 'Enroll_Prim',
    'School enrollment, secondary (% gross)': 'Enroll_Sec',
    'School enrollment, tertiary (% gross)': 'Enroll_Tert',
    'Unemployment with advanced education (% of total labor force with advanced education)': 'Unemp_AdvEdu',
    'Unemployment with basic education (% of total labor force with basic education)': 'Unemp_BasicEdu',
    'Unemployment, female (% of female labor force) (modeled ILO estimate)': 'Unemp_Fem_ILO',
    'Unemployment, female (% of female labor force) (national estimate)': 'Unemp_Fem_Nat',
    'Unemployment, male (% of male labor force) (modeled ILO estimate)': 'Unemp_Male_ILO',
    'Unemployment, male (% of male labor force) (national estimate)': 'Unemp_Male_Nat',
    'Unemployment, total (% of total labor force) (modeled ILO estimate)': 'Unemp_Tot_ILO',
    'Unemployment, total (% of total labor force) (national estimate)': 'Unemp_Tot_Nat'
}
df_WDI_pivot = df_WDI_pivot.rename(columns=rename_dict)
df_WDI_pivot
Out[50]:
Series Name Country Name ISO Year SocialNet_Cov Edu_Bach25p Edu_Prim25p Edu_UpSec25p GDP_Growth GDP_pc GDP_pc_Growth ... Enroll_Sec Enroll_Tert Unemp_AdvEdu Unemp_BasicEdu Unemp_Fem_ILO Unemp_Fem_Nat Unemp_Male_ILO Unemp_Male_Nat Unemp_Tot_ILO Unemp_Tot_Nat
0 Afghanistan AFG 2022 NaN 3.984250 21.025480 12.597330 -6.240172 357.261153 -7.576669 ... NaN NaN NaN NaN 26.742 NaN 13.168 NaN 14.100 NaN
1 Afghanistan AFG 2023 NaN NaN NaN NaN 2.266944 413.757895 0.106093 ... NaN NaN NaN NaN 26.570 NaN 13.066 NaN 13.991 NaN
2 Afghanistan AFG 2024 NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN 25.636 NaN 12.394 NaN 13.295 NaN
3 Albania ALB 2022 NaN 19.834644 97.853140 53.268485 4.826696 6846.426694 6.108948 ... 97.322471 62.647922 10.228 9.437 10.405 11.235 9.917 10.398 10.137 10.785
4 Albania ALB 2023 NaN 17.510000 99.279999 53.970001 3.936617 8575.171346 5.137124 ... 96.136877 64.729352 8.947 9.694 10.376 10.958 9.888 10.421 10.108 10.669
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
650 Zambia ZMB 2023 NaN 2.801195 NaN 33.588454 5.367943 1330.727806 2.464640 ... NaN NaN 9.702 3.688 6.207 6.037 5.646 5.710 5.905 5.861
651 Zambia ZMB 2024 NaN NaN NaN NaN 4.043148 1235.084665 1.158387 ... 56.788929 NaN NaN NaN 6.250 NaN 5.713 NaN 5.961 NaN
652 Zimbabwe ZWE 2022 NaN 0.913878 NaN NaN 6.139176 2040.546587 4.343582 ... NaN NaN 5.945 8.878 10.562 10.461 9.615 9.554 10.087 10.001
653 Zimbabwe ZWE 2023 NaN 1.134873 NaN NaN 5.336770 2156.034093 3.584903 ... NaN NaN 5.933 8.493 8.824 9.460 8.694 9.126 8.759 9.290
654 Zimbabwe ZWE 2024 NaN NaN NaN NaN 2.029484 2656.409377 0.228944 ... NaN 7.749130 NaN NaN 8.587 NaN 8.521 NaN 8.554 NaN

655 rows × 47 columns

We also need the total population variable from the same database, which I had forgotten to export at first. I will quickly load and reshape it so that it can be merged into the joint dataset using similar steps as the other WDIs above.

In [51]:
df_pop = pd.read_csv('Pop22-24.csv')
df_pop = df_pop.melt(id_vars=['Country Name', 'Country Code', 'Series Name', 'Series Code'], var_name='Year', value_name='Value') # Reshape
df_pop['Year'] = df_pop['Year'].replace({'2022 [YR2022]': '2022', '2023 [YR2023]': '2023', '2024 [YR2024]': '2024'}).astype(int) # Fix year syntax
df_pop['Value'] = df_pop['Value'].apply(pd.to_numeric, errors='coerce') # Fix dtype of pop value
df_pop.drop(columns=['Series Name', 'Series Code'], inplace=True) # Drop series name and code
df_pop.rename(columns={'Value': 'Tot_Pop'}, inplace=True) # Rename population variable
df_pop.dtypes
Out[51]:
0
Country Name object
Country Code object
Year int64
Tot_Pop float64

Now our World Bank indicators are ready for merging and further analysis.

Exploratory Data Analysis (EDA)¶

2024 Refugee Counts Pie Chart¶

To begin our EDA, we will create a pie chart to answer the question, "What countries have the largest Ukrainian refugee populations today?" This is a simple yet helpful snapshot of how the refugees are distributed across host countries. Later analysis will tell us things like how these refugee flows have fluctuated over the course of the war.

In [52]:
# Isolate latest year
RefCounts_2024 = df_RefCounts[df_RefCounts['Year']== 2024].copy()
RefCounts_2024 = RefCounts_2024.sort_values('Ref_Asylum', ascending=False)

# Build df of top n countries
top_n = 9
top_2024 = RefCounts_2024.head(top_n)

# Sum the rest of the counts into a one-observation df for "Other"
others_sum = RefCounts_2024['Ref_Asylum'].iloc[top_n:].sum()
df_other = pd.DataFrame({'Country of Asylum':['Other'],'Ref_Asylum':[others_sum]})

# Concatenate the two
df_piechart = pd.concat([top_2024, df_other], ignore_index=True)

# Now we actually make the chart
plt.figure(figsize=(6,6))
plt.pie(
    df_piechart['Ref_Asylum'],
    labels = df_piechart['Country of Asylum'],
    autopct = '%1.1f%%',
    startangle = 140
    )
plt.title('2024 Refugee and Asylum-seekers by Country')
plt.show()
No description has been provided for this image

As we can see, the top countries are all in Europe, and about 50% of all Ukrainian refugees today are located in only 3 countries: Germany, Poland, and Czechia. In fact, the top 9 countries host 71.6% of all refugees. Knowing this can help us narrow down our expectations of what variables should predict higher refugee populations.

Histogram of Refugee Populations¶

Another interesting graphic we can make to show the distribution is a histogram of our 82 host countries by their 2024 Ukrainian refugee/asylum-seeker populations:

In [53]:
plt.figure(figsize=(7,4))
plt.hist(RefCounts_2024['Ref_Asylum'], bins=20)
plt.xlabel('Refugee and Asylum-seeker Population')
plt.ylabel('Number of Countries')
plt.title(f'Distribution of Displaced Ukrainians Across Countries in 2024')
plt.show()
No description has been provided for this image

We clearly see the data has a large right skew, representing how a few countries have large numbers of refugees. To correct for this, we can transform the data. Once done, this transformed data will serve as our dependent variable in OLS regression. Here, I've selected a log transformation because it will compress extreme values to a roughly symmetrical distribution (shown below) and still allow for easy coefficient interpretations in later regressions.

In [54]:
#Log-transform refugee counts into a more evenly distributed shape, creating a new column in the main RefCounts df.
x = df_RefCounts['Ref_Asylum']
df_RefCounts['ln_Ref_Asylum'] = np.log(x + 1) # Add 1 to account for zeros

#Now filter again to latest year
RefCounts_2024 = df_RefCounts[df_RefCounts['Year'] == 2024].copy()

#Calculate skew and check log-transformation is sufficiently symmetrical
from scipy.stats import skew
print("Skew (original):", skew(RefCounts_2024['Ref_Asylum']))
print("Skew (log-transformed):", skew(RefCounts_2024['ln_Ref_Asylum'])) #Should be near zero

plt.figure(figsize=(7,4))
plt.hist(RefCounts_2024['ln_Ref_Asylum'], bins=20)
plt.xlabel('ln(Refugee Population)')
plt.ylabel('Number of Countries')
plt.title(f'Refugee/Asylum-seeker Populations Across Countries in 2024')
plt.show()
Skew (original): 5.077644846318671
Skew (log-transformed): -0.15662866559786284
No description has been provided for this image

Merging for further EDA¶

Now we will merge our datasets together on their ISO codes, dropping the redundant key columns.

In [55]:
# Merge refugee counts with WDIs
df_RefWDI = pd.merge(df_RefCounts, df_WDI_pivot, how='left', left_on=['Country of Asylum ISO', 'Year'], right_on=['ISO', 'Year'])
# Merge in total population variable too
df_RefWDI = pd.merge(df_RefWDI, df_pop, how='left', left_on=['Country of Asylum ISO', 'Year'], right_on=['Country Code', 'Year'])
# Drop redundant key columns
df_RefWDI.drop(columns=['Country Name_x', 'Country Name_y', 'ISO', 'Country Code'], inplace=True)

df_RefWDI
Out[55]:
Year Country of Asylum Country of Asylum ISO Refugees Asylum-seekers Ref_Asylum ln_Ref_Asylum SocialNet_Cov Edu_Bach25p Edu_Prim25p ... Enroll_Tert Unemp_AdvEdu Unemp_BasicEdu Unemp_Fem_ILO Unemp_Fem_Nat Unemp_Male_ILO Unemp_Male_Nat Unemp_Tot_ILO Unemp_Tot_Nat Tot_Pop
0 2022 Albania ALB 2542 5 2547 7.843064 NaN 19.834644 97.853140 ... 62.647922 10.228 9.437 10.405 11.235 9.917 10.398 10.137 10.785 2777689.0
1 2023 Albania ALB 4959 0 4959 8.509161 NaN 17.510000 99.279999 ... 64.729352 8.947 9.694 10.376 10.958 9.888 10.421 10.108 10.669 2745972.0
2 2024 Albania ALB 9250 0 9250 9.132487 NaN NaN NaN ... 81.998627 NaN NaN 10.527 NaN 10.023 NaN 10.250 NaN 2714617.0
3 2022 Egypt EGY 0 5 5 1.791759 NaN NaN 66.608430 ... 36.462231 15.183 4.705 18.283 18.516 4.987 5.051 7.344 7.344 112618250.0
4 2023 Egypt EGY 0 9 9 2.302585 NaN 18.052291 69.221964 ... 37.900558 14.175 4.313 18.234 15.329 4.972 4.617 7.334 6.753 114535772.0
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
233 2023 Uruguay URY 18 0 18 2.944439 NaN NaN 92.800000 ... 79.953690 2.034 9.926 9.435 9.462 7.408 7.429 8.355 8.355 3388081.0
234 2024 Uruguay URY 18 5 23 3.178054 NaN 12.929680 93.832077 ... NaN 1.909 9.789 9.757 9.416 7.209 7.201 8.401 8.209 3386588.0
235 2022 United States USA 3063 12369 15432 9.644263 NaN 37.660000 99.029999 ... 79.361931 2.321 5.507 3.613 3.614 3.680 3.681 3.650 3.650 334017321.0
236 2023 United States USA 3299 17773 21072 9.955748 NaN 38.601131 99.075211 ... NaN 2.373 5.591 3.450 3.454 3.795 3.800 3.638 3.638 336806231.0
237 2024 United States USA 3397 18658 22055 10.001340 NaN 39.029478 99.682239 ... NaN 2.608 6.501 3.892 3.917 4.285 4.115 4.106 4.022 340110988.0

238 rows × 52 columns

Normalized Refugee Counts¶

To be able to better compare countries of different sizes in our future model, we should also create a new refugee count variable normalized by the host population. While the logged values from earlier are useful for analyzing total flows, a measurement like refugees per-100 people can help us better understand the relative burden of migrant flows on host countries.

When comparing the top host countries in total count then normalized count below, we can see that larger-population countries like Germany, Russia, and the U.K. drop out of the top 10. This also emphasizes the impact of the refugee crisis on countries like Poland, Czechia, and Slovakia, which have large refugee populations in both level and ratio terms.

In [56]:
df_RefWDI['Refs_per_100'] = df_RefWDI['Ref_Asylum'] / df_RefWDI['Tot_Pop'] * 100

# Compare top rankings with level variable
top_refs = df_RefWDI.groupby('Country of Asylum')[['Ref_Asylum', 'Tot_Pop']].mean().reset_index().sort_values('Ref_Asylum', ascending=False)
display(top_refs.head(10))
top_normed = df_RefWDI.groupby('Country of Asylum')[['Refs_per_100', 'Tot_Pop']].mean().reset_index().sort_values('Refs_per_100', ascending=False)
display(top_normed.head(10))
Country of Asylum Ref_Asylum Tot_Pop
28 Germany 1.103695e+06 8.373695e+07
65 Poland 9.700287e+05 3.668794e+07
68 Russia 8.369263e+05 1.438656e+08
19 Czechia 3.988330e+05 1.080611e+07
80 United Kingdom 2.217697e+05 6.844067e+07
74 Spain 1.922450e+05 4.831372e+07
38 Italy 1.624677e+05 5.899772e+07
70 Slovakia 1.206673e+05 5.426854e+06
67 Romania 1.198760e+05 1.905911e+07
52 Moldova 1.195533e+05 2.458571e+06
Country of Asylum Refs_per_100 Tot_Pop
53 Montenegro 6.176500 6.236503e+05
52 Moldova 4.878297 2.458571e+06
19 Czechia 3.693071 1.080611e+07
24 Estonia 3.040658 1.363704e+06
65 Poland 2.644142 3.668794e+07
44 Latvia 2.310928 1.873090e+06
48 Lithuania 2.249647 2.863760e+06
70 Slovakia 2.223784 5.426854e+06
36 Ireland 1.764716 5.300231e+06
11 Bulgaria 1.506933 6.452020e+06

Statistics for Time-Series Analysis¶

Another important aspect of my project is its temporal dimension; the refugee data I have collected is a 3-year long panel of different countries. While the number of periods in the panel is low, there are still important statistics we can generate that will help our later modeling. For example, lags of refugee counts are essential in autoregressive modeling if I want to account for the serial correlation between observations. I can also compute discrete growth rates and log-differences of a country's refugee population from year to year, revealing how volatile or stable the flows are over time.

In [57]:
# Lags for the variable Ref_Asylum
df_RefWDI['Ref_Asylum_t-1'] = df_RefWDI.groupby('Country of Asylum')['Ref_Asylum'].shift(1)
df_RefWDI['Ref_Asylum_t-2'] = df_RefWDI.groupby('Country of Asylum')['Ref_Asylum'].shift(2)

# Replace NaNs in lagged refugee counts with zeros because pre-war, they had zero refugees.
df_RefWDI.fillna({'Ref_Asylum_t-1': 0}, inplace = True)
df_RefWDI.fillna({'Ref_Asylum_t-2': 0}, inplace = True)

# Now log transform these lags as well to make them compatible with our dependent variable.
df_RefWDI['ln_Ref_Asylum_t-1'] = np.log(df_RefWDI['Ref_Asylum_t-1'] + 1)
df_RefWDI['ln_Ref_Asylum_t-2'] = np.log(df_RefWDI['Ref_Asylum_t-2'] + 1)

# Also compute log-differences for Ref_Asylum
df_RefWDI['Ref_Growth_LD'] = df_RefWDI['ln_Ref_Asylum'] - df_RefWDI['ln_Ref_Asylum_t-1']
# Discrete growth rates
df_RefWDI['Ref_Growth'] = (df_RefWDI['Ref_Asylum'] - df_RefWDI['Ref_Asylum_t-1']) / df_RefWDI['Ref_Asylum_t-1'] * 100
In [58]:
display(df_RefWDI.sort_values(by='Ref_Growth_LD', ascending=False).head(5))
display(df_RefWDI[df_RefWDI['Ref_Growth'] != np.inf].sort_values(by='Ref_Growth', ascending=False).head(5))
Year Country of Asylum Country of Asylum ISO Refugees Asylum-seekers Ref_Asylum ln_Ref_Asylum SocialNet_Cov Edu_Bach25p Edu_Prim25p ... Unemp_Tot_ILO Unemp_Tot_Nat Tot_Pop Refs_per_100 Ref_Asylum_t-1 Ref_Asylum_t-2 ln_Ref_Asylum_t-1 ln_Ref_Asylum_t-2 Ref_Growth_LD Ref_Growth
204 2022 Russia RUS 1275315 610 1275925 14.059183 NaN NaN 100.000000 ... 3.867 3.867 144236933.0 0.884604 0.0 0.0 0.0 0.0 14.059183 inf
84 2022 Germany DEU 1002999 1383 1004382 13.819884 NaN 29.575744 100.000000 ... 3.120 3.137 83797985.0 1.198575 0.0 0.0 0.0 0.0 13.819884 inf
194 2022 Poland POL 958147 372 958519 13.773146 NaN NaN 99.512287 ... 2.811 2.811 36821749.0 2.603133 0.0 0.0 0.0 0.0 13.773146 inf
57 2022 Czechia CZE 433501 137 433638 12.979968 NaN 22.287664 99.882802 ... 2.224 2.224 10672118.0 4.063280 0.0 0.0 0.0 0.0 12.979968 inf
78 2022 United Kingdom GBR 182423 976 183399 12.119425 NaN NaN NaN ... 3.730 3.768 67604000.0 0.271284 0.0 0.0 0.0 0.0 12.119425 inf

5 rows × 59 columns

Year Country of Asylum Country of Asylum ISO Refugees Asylum-seekers Ref_Asylum ln_Ref_Asylum SocialNet_Cov Edu_Bach25p Edu_Prim25p ... Unemp_Tot_ILO Unemp_Tot_Nat Tot_Pop Refs_per_100 Ref_Asylum_t-1 Ref_Asylum_t-2 ln_Ref_Asylum_t-1 ln_Ref_Asylum_t-2 Ref_Growth_LD Ref_Growth
31 2023 Brazil BRA 1129 30 1159 7.056175 NaN NaN 85.200000 ... 7.947 7.947 211140729.0 0.000549 108.0 0.0 4.691348 0.0 2.364827 973.148148
13 2023 Australia AUS 7472 809 8281 9.021840 NaN 35.764835 99.491927 ... 3.668 3.668 26652777.0 0.031070 2029.0 0.0 7.615791 0.0 1.406049 308.132085
208 2023 Serbia SRB 4063 0 4063 8.309923 NaN 18.669201 98.741946 ... 8.268 8.268 6623183.0 0.061345 1108.0 0.0 7.011214 0.0 1.298709 266.696751
28 2023 Bolivia BOL 7 10 17 2.890372 NaN 20.118389 71.232811 ... 3.024 2.948 12244159.0 0.000139 5.0 0.0 1.791759 0.0 1.098612 240.000000
189 2023 Peru PER 30 90 120 4.795791 NaN NaN 75.800000 ... 4.899 4.899 33845617.0 0.000355 36.0 0.0 3.610918 0.0 1.184873 233.333333

5 rows × 59 columns

As we can see from the above tables, in the log-differences version, the single largest growth episodes were in the first year of the war to countries like Russia, Germany, and Poland, and at a massive scale; Russia's log-difference of 14.1 indicates growth in refugee population by a factor of about 1.2 million. However, this is expected because log-difference growth starting from zero would be very large compared to growth between years with non-zero refugee counts. Furthermore, in such an explosive conflict, it makes sense contextually that the first year would see the largest outflow of Ukrainian refugees.

Looking at discrete growth, since the 2022 rates start from zero, those observed rates are mathematically infinite. Excluding those occurrences to look at growth after the first year of arrivals in a given country, we can see that Brazil had the next largest relative increase in refugee population, up 973% from 108 to 1159 in 2023. In fact, we can see that all of the top 5 single largest increases (outside 2022) occurred in 2023, so we know that the rate of refugee flows appears to have started high and slowly decreased from year to year in this data set.

Handling Missing Data¶

The World Development Indicators are often full of missing data, often due to varying data collection capacities between countries. We must be able to handle the missing observations as part of our analysis. We can create a missing data heatmap to see at a glance which variables have many missing variables.

In [59]:
plt.figure(figsize=(12,6))
sns.heatmap(df_RefWDI.isnull(), cbar=False)
plt.title("Missing Data Heatmap")
plt.show()
No description has been provided for this image

We can also generate missing value counts and percentages for a more explicit breakdown by variable.

In [60]:
missing_count = df_RefWDI.isna().sum()
missing_pct = (missing_count/len(df_RefWDI))*100

missing_summary = pd.DataFrame({
    'missing_count': missing_count,
    'missing_pct': missing_pct
}).sort_values(by='missing_pct', ascending=False)
missing_summary_full = missing_summary.copy() # Save a copy for later
missing_summary
Out[60]:
missing_count missing_pct
SocialNet_Cov 225 94.537815
Net_UNHCR 194 81.512605
Poverty_pct 166 69.747899
Gini 134 56.302521
Edu_Bach25p 114 47.899160
PrimComp_pct 103 43.277311
FoodInsec_pct 100 42.016807
Edu_UpSec25p 91 38.235294
Enroll_Tert 86 36.134454
Enroll_Sec 85 35.714286
Edu_Prim25p 85 35.714286
Pol_Stab_rank 81 34.033613
RuleLaw_rank 81 34.033613
GovEffect_rank 81 34.033613
Enroll_Prim 75 31.512605
LF_IntEdu 52 21.848739
Unemp_AdvEdu 51 21.428571
Unemp_BasicEdu 51 21.428571
GNI_pc_Growth 50 21.008403
LF_BasicEdu 50 21.008403
LF_AdvEdu 50 21.008403
Unemp_Fem_Nat 38 15.966387
Unemp_Male_Nat 38 15.966387
Unemp_Tot_Nat 38 15.966387
LFPR_15_24_Nat 38 15.966387
LFPR_15p_Nat 36 15.126050
GDP_Growth 5 2.100840
GNI_pc 5 2.100840
GDP_pc_Growth 5 2.100840
LFPR_15p_ILO 4 1.680672
LFPR_15_64_ILO 4 1.680672
Unemp_Tot_ILO 4 1.680672
LF_Fem 4 1.680672
Unemp_Fem_ILO 4 1.680672
Unemp_Male_ILO 4 1.680672
LFPR_15_24_ILO 4 1.680672
GDP_pc 3 1.260504
Year 0 0.000000
Country of Asylum ISO 0 0.000000
Asylum-seekers 0 0.000000
Ref_Asylum 0 0.000000
Refugees 0 0.000000
Country of Asylum 0 0.000000
ln_Ref_Asylum 0 0.000000
Pop_Male_pct 0 0.000000
Pop_15_64_pct 0 0.000000
Pop_15_64_Tot 0 0.000000
Pop_15_64_Male 0 0.000000
Pop_Fem_pct 0 0.000000
Pop_Growth 0 0.000000
Pop_15_64_Fem 0 0.000000
Tot_Pop 0 0.000000
Refs_per_100 0 0.000000
Ref_Asylum_t-1 0 0.000000
Ref_Asylum_t-2 0 0.000000
ln_Ref_Asylum_t-1 0 0.000000
ln_Ref_Asylum_t-2 0 0.000000
Ref_Growth_LD 0 0.000000
Ref_Growth 0 0.000000

As we can see, there are many variables missing large amounts of data. To start, we will drop columns that are missing 50% or more of their observations.

In [61]:
# Turn columns index into list of columns
to_drop = missing_summary[missing_summary['missing_pct'] > 50].index.tolist()
# Drop columns/rows from missing summary and the main dataframe
missing_summary.drop(to_drop, inplace=True)
df_RefWDI.drop(columns=to_drop, inplace=True)

I also made a modified version of our dataframe with only the remaining variables that are missing 20% or more of their observations to see if there are any visible patterns in the missing data

In [62]:
# Vars missing over 20% of observations
manymiss_vars = missing_summary[missing_summary['missing_pct'] > 20].index
to_drop = ['ln_Ref_Asylum_t-2', 'Ref_Growth', 'ln_Ref_Asylum_t-1', 'Ref_Asylum_t-1']
manymiss_vars_clean = manymiss_vars.difference(to_drop)

# See main df with only these variables to look for patterns
to_keep = ['Country of Asylum', 'Year'] + list(manymiss_vars_clean)
df_manymiss = df_RefWDI[to_keep]
df_manymiss
Out[62]:
Country of Asylum Year Edu_Bach25p Edu_Prim25p Edu_UpSec25p Enroll_Prim Enroll_Sec Enroll_Tert FoodInsec_pct GNI_pc_Growth GovEffect_rank LF_AdvEdu LF_BasicEdu LF_IntEdu Pol_Stab_rank PrimComp_pct RuleLaw_rank Unemp_AdvEdu Unemp_BasicEdu
0 Albania 2022 19.834644 97.853140 53.268485 96.371230 97.322471 62.647922 32.2 7.856992 56.132076 77.472 54.670 65.703 49.528301 101.176280 47.169811 10.228 9.437
1 Albania 2023 17.510000 99.279999 53.970001 93.713812 96.136877 64.729352 33.0 7.289707 60.377357 82.056 54.455 68.530 51.658768 95.135152 45.754719 8.947 9.694
2 Albania 2024 NaN NaN NaN 111.511650 108.355392 81.998627 NaN NaN NaN NaN NaN NaN NaN 107.345322 NaN NaN NaN
3 Egypt 2022 NaN 66.608430 54.313658 89.510087 NaN 36.462231 29.8 4.736284 34.433964 64.216 27.450 47.904 14.622642 NaN 42.924530 15.183 4.705
4 Egypt 2023 18.052291 69.221964 55.692370 95.921707 84.474862 37.900558 30.8 0.896229 41.981133 64.715 29.346 49.534 16.587677 93.841399 44.339622 14.175 4.313
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
233 Uruguay 2023 NaN 92.800000 34.000000 104.532466 119.105006 79.953690 16.2 2.573633 79.245285 81.355 61.637 72.827 81.990524 101.169008 75.943398 2.034 9.926
234 Uruguay 2024 12.929680 93.832077 35.009258 NaN NaN NaN NaN 2.610834 NaN 81.471 62.368 74.021 NaN NaN NaN 1.909 9.789
235 United States 2022 37.660000 99.029999 91.190002 96.971733 97.473488 79.361931 9.3 1.852199 87.264153 71.780 42.572 56.076 46.226414 95.743523 88.679245 2.321 5.507
236 United States 2023 38.601131 99.075211 91.461288 NaN NaN NaN 10.3 1.821216 87.735847 71.866 44.160 56.415 47.393364 NaN 88.679245 2.373 5.591
237 United States 2024 39.029478 99.682239 95.447222 NaN NaN NaN NaN 1.633877 NaN 71.592 44.352 56.369 NaN NaN NaN 2.608 6.501

238 rows × 19 columns

In this table above, we can see that a lot of the missing observations come in the year 2024, indicating that it's likely due to a lag in reporting of the most recent data. This trend persists across many different countries, so it is safe to use mean imputation on this data if we assume the 2024 indicators have no reason to be significantly different from their 2022 and 2023 values within a given country.

In [63]:
# Make list of just the numeric columns, except year.
numeric_cols = df_RefWDI.select_dtypes(include=np.number).columns.drop('Year', errors='ignore')

# Apply mean imputation, grouping by country, saving to a new df
df_RefWDI_meanimpute = df_RefWDI.copy()
df_RefWDI_meanimpute[numeric_cols] = df_RefWDI.groupby('Country of Asylum')[numeric_cols].transform(lambda x: x.fillna(x.mean()))

df_RefWDI_meanimpute
Out[63]:
Year Country of Asylum Country of Asylum ISO Refugees Asylum-seekers Ref_Asylum ln_Ref_Asylum Edu_Bach25p Edu_Prim25p Edu_UpSec25p ... Unemp_Tot_ILO Unemp_Tot_Nat Tot_Pop Refs_per_100 Ref_Asylum_t-1 Ref_Asylum_t-2 ln_Ref_Asylum_t-1 ln_Ref_Asylum_t-2 Ref_Growth_LD Ref_Growth
0 2022 Albania ALB 2542 5 2547 7.843064 19.834644 97.853140 53.268485 ... 10.137 10.785 2777689.0 0.091695 0.0 0.0 0.000000 0.000000 7.843064 inf
1 2023 Albania ALB 4959 0 4959 8.509161 17.510000 99.279999 53.970001 ... 10.108 10.669 2745972.0 0.180592 2547.0 0.0 7.843064 0.000000 0.666097 94.699647
2 2024 Albania ALB 9250 0 9250 9.132487 18.672322 98.566569 53.619243 ... 10.250 10.727 2714617.0 0.340748 4959.0 2547.0 8.509161 7.843064 0.623326 86.529542
3 2022 Egypt EGY 0 5 5 1.791759 18.052291 66.608430 54.313658 ... 7.344 7.344 112618250.0 0.000004 0.0 0.0 0.000000 0.000000 1.791759 inf
4 2023 Egypt EGY 0 9 9 2.302585 18.052291 69.221964 55.692370 ... 7.334 6.753 114535772.0 0.000008 5.0 0.0 1.791759 0.000000 0.510826 80.000000
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
233 2023 Uruguay URY 18 0 18 2.944439 12.929680 92.800000 34.000000 ... 8.355 8.355 3388081.0 0.000531 18.0 0.0 2.944439 0.000000 0.000000 0.000000
234 2024 Uruguay URY 18 5 23 3.178054 12.929680 93.832077 35.009258 ... 8.401 8.209 3386588.0 0.000679 18.0 18.0 2.944439 2.944439 0.233615 27.777778
235 2022 United States USA 3063 12369 15432 9.644263 37.660000 99.029999 91.190002 ... 3.650 3.650 334017321.0 0.004620 0.0 0.0 0.000000 0.000000 9.644263 inf
236 2023 United States USA 3299 17773 21072 9.955748 38.601131 99.075211 91.461288 ... 3.638 3.638 336806231.0 0.006256 15432.0 0.0 9.644263 0.000000 0.311485 36.547434
237 2024 United States USA 3397 18658 22055 10.001340 39.029478 99.682239 95.447222 ... 4.106 4.022 340110988.0 0.006485 21072.0 15432.0 9.955748 9.644263 0.045592 4.664958

238 rows × 55 columns

In [64]:
# Now regenerate the missing summary to see how it went
missing_count = df_RefWDI_meanimpute.isna().sum()
missing_pct = (missing_count/len(df_RefWDI_meanimpute))*100

missing_summary = pd.DataFrame({
    'missing_count': missing_count,
    'missing_pct': missing_pct
}).sort_values(by='missing_pct', ascending=False)
missing_summary
Out[64]:
missing_count missing_pct
Edu_Bach25p 63 26.470588
PrimComp_pct 48 20.168067
Edu_UpSec25p 45 18.907563
LF_IntEdu 40 16.806723
Unemp_BasicEdu 37 15.546218
LF_AdvEdu 37 15.546218
LF_BasicEdu 37 15.546218
Unemp_AdvEdu 37 15.546218
Edu_Prim25p 36 15.126050
FoodInsec_pct 27 11.344538
LFPR_15_24_Nat 25 10.504202
Enroll_Tert 23 9.663866
Unemp_Male_Nat 22 9.243697
Unemp_Fem_Nat 22 9.243697
Unemp_Tot_Nat 22 9.243697
GNI_pc_Growth 20 8.403361
LFPR_15p_Nat 19 7.983193
Enroll_Sec 19 7.983193
Enroll_Prim 9 3.781513
RuleLaw_rank 4 1.680672
GovEffect_rank 4 1.680672
Pol_Stab_rank 4 1.680672
GDP_pc_Growth 3 1.260504
Unemp_Tot_ILO 3 1.260504
LFPR_15_64_ILO 3 1.260504
LFPR_15_24_ILO 3 1.260504
LFPR_15p_ILO 3 1.260504
GNI_pc 3 1.260504
LF_Fem 3 1.260504
Unemp_Fem_ILO 3 1.260504
Unemp_Male_ILO 3 1.260504
GDP_Growth 3 1.260504
Year 0 0.000000
Country of Asylum 0 0.000000
Country of Asylum ISO 0 0.000000
Refugees 0 0.000000
Asylum-seekers 0 0.000000
Ref_Asylum 0 0.000000
ln_Ref_Asylum 0 0.000000
GDP_pc 0 0.000000
Pop_Male_pct 0 0.000000
Pop_15_64_Tot 0 0.000000
Pop_Fem_pct 0 0.000000
Pop_15_64_Fem 0 0.000000
Pop_15_64_Male 0 0.000000
Pop_Growth 0 0.000000
Pop_15_64_pct 0 0.000000
Tot_Pop 0 0.000000
Refs_per_100 0 0.000000
Ref_Asylum_t-1 0 0.000000
Ref_Asylum_t-2 0 0.000000
ln_Ref_Asylum_t-1 0 0.000000
ln_Ref_Asylum_t-2 0 0.000000
Ref_Growth_LD 0 0.000000
Ref_Growth 0 0.000000

After mean imputation, we still have a decent number of NaNs in some of our variables. This is because some of the countries have missing values for all three years, so there is no data to average in the first imputation method. There are also some countries that only appear in the dataset once (i.e. they only recorded Ukrainian refugees in one year) and are missing an indicator value, so they cannot be averaged either. To fill in these values, we can use SciKit Learn to run a K-nearest neighbors imputation on the remaining NaNs.

In [65]:
from sklearn.impute import KNNImputer
from sklearn.preprocessing import StandardScaler
imputer = KNNImputer(n_neighbors=5)
scaler = StandardScaler()
# Make copy
df_knnimpute = df_RefWDI_meanimpute.copy()
# Select numeric columns except year and engineered variables
numeric_cols = df_knnimpute.select_dtypes(include=np.number).columns.drop(['Year', 'Ref_Asylum_t-1', 'Ref_Asylum_t-2', 'ln_Ref_Asylum_t-1', 'ln_Ref_Asylum_t-2', 'Ref_Growth_LD', 'Ref_Growth', 'Refs_per_100'], errors='ignore')
# Scale
df_scaled = scaler.fit_transform(df_knnimpute[numeric_cols])
# Impute
df_sca_imp = imputer.fit_transform(df_scaled)
# Reverse the scaling so the values are interpretable again
df_imputed = scaler.inverse_transform(df_sca_imp)
# Put values back in original dataframe
df_knnimpute[numeric_cols] = df_imputed

df_final = df_knnimpute
df_final
Out[65]:
Year Country of Asylum Country of Asylum ISO Refugees Asylum-seekers Ref_Asylum ln_Ref_Asylum Edu_Bach25p Edu_Prim25p Edu_UpSec25p ... Unemp_Tot_ILO Unemp_Tot_Nat Tot_Pop Refs_per_100 Ref_Asylum_t-1 Ref_Asylum_t-2 ln_Ref_Asylum_t-1 ln_Ref_Asylum_t-2 Ref_Growth_LD Ref_Growth
0 2022 Albania ALB 2542.0 5.0 2547.0 7.843064 19.834644 97.853140 53.268485 ... 10.137 10.785 2777689.0 0.091695 0.0 0.0 0.000000 0.000000 7.843064 inf
1 2023 Albania ALB 4959.0 0.0 4959.0 8.509161 17.510000 99.279999 53.970001 ... 10.108 10.669 2745972.0 0.180592 2547.0 0.0 7.843064 0.000000 0.666097 94.699647
2 2024 Albania ALB 9250.0 0.0 9250.0 9.132487 18.672322 98.566569 53.619243 ... 10.250 10.727 2714617.0 0.340748 4959.0 2547.0 8.509161 7.843064 0.623326 86.529542
3 2022 Egypt EGY 0.0 5.0 5.0 1.791759 18.052291 66.608430 54.313658 ... 7.344 7.344 112618250.0 0.000004 0.0 0.0 0.000000 0.000000 1.791759 inf
4 2023 Egypt EGY 0.0 9.0 9.0 2.302585 18.052291 69.221964 55.692370 ... 7.334 6.753 114535772.0 0.000008 5.0 0.0 1.791759 0.000000 0.510826 80.000000
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
233 2023 Uruguay URY 18.0 0.0 18.0 2.944439 12.929680 92.800000 34.000000 ... 8.355 8.355 3388081.0 0.000531 18.0 0.0 2.944439 0.000000 0.000000 0.000000
234 2024 Uruguay URY 18.0 5.0 23.0 3.178054 12.929680 93.832077 35.009258 ... 8.401 8.209 3386588.0 0.000679 18.0 18.0 2.944439 2.944439 0.233615 27.777778
235 2022 United States USA 3063.0 12369.0 15432.0 9.644263 37.660000 99.029999 91.190002 ... 3.650 3.650 334017321.0 0.004620 0.0 0.0 0.000000 0.000000 9.644263 inf
236 2023 United States USA 3299.0 17773.0 21072.0 9.955748 38.601131 99.075211 91.461288 ... 3.638 3.638 336806231.0 0.006256 15432.0 0.0 9.644263 0.000000 0.311485 36.547434
237 2024 United States USA 3397.0 18658.0 22055.0 10.001340 39.029478 99.682239 95.447222 ... 4.106 4.022 340110988.0 0.006485 21072.0 15432.0 9.955748 9.644263 0.045592 4.664958

238 rows × 55 columns

In [66]:
# Check entire dataframe for NaNs
df_final.isna().sum().sum().item()
Out[66]:
0

Now there is no more missing data!

Regressor Selection: Correlation Matrix¶

Here, we will look at the linear correlations between a number of macroeconomic variables and our prospective dependent variables. This is an important step because when it comes time to select OLS regressors, these correlations help us understand which variables may be most telling.

In [67]:
corr_vars = ['ln_Ref_Asylum', 'ln_Ref_Asylum_t-1', 'GDP_pc', 'Unemp_AdvEdu','Pol_Stab_rank', 'RuleLaw_rank', 'GovEffect_rank', 'Unemp_Tot_ILO', 'PrimComp_pct', 'FoodInsec_pct', 'LFPR_15_64_ILO', 'Edu_Bach25p']

fig, ax = plt.subplots(figsize=(10,10))
sns.heatmap(df_RefWDI[corr_vars].corr(), annot=True, cmap='coolwarm', ax=ax)
Out[67]:
<Axes: >
No description has been provided for this image

Already, we can see that ln_Ref_Asylum has moderately high correlations with host factors like its first lag, bachelor's degree attainment, food insecurity rate, and percentile ranks in rule of law and government effectiveness. There are also moderate correlations with political stability rank, labor force participation rate, and advanced-education unemployment rate. This information can help inform our regressor selection.

We can also loop through all of our indicators to find the variables with the strongest correlations and sort them in order, merging with the original missing summary so we don't lose sight of how the precision we imposed via imputation may impact each variable's correlation:

In [68]:
all_numeric_cols = df_final.select_dtypes(include=np.number).columns.drop(['Year', 'Ref_Asylum_t-1', 'Ref_Asylum_t-2', 'Ref_Growth', 'Refs_per_100'], errors='ignore')

corr_list = []
for col in all_numeric_cols:
    corr_val = df_final[col].corr(df_final['ln_Ref_Asylum'])
    corr_list.append((col, corr_val, abs(corr_val)))

corr_df = pd.DataFrame(corr_list, columns=['Variable', 'Correlation', 'AbsCorrelation']).sort_values('AbsCorrelation', ascending=False)

# Reset index for missing_summary_full
missing_summary_full.reset_index(inplace=True)
missing_summary_full.rename(columns={'index': 'Variable'}, inplace=True)

# Merge with corr_df on variable name
corr_df = pd.merge(corr_df, missing_summary_full, on='Variable', how='left')
corr_df = corr_df.drop(columns='missing_count')
corr_df
Out[68]:
Variable Correlation AbsCorrelation missing_pct
0 ln_Ref_Asylum 1.000000 1.000000 0.000000
1 Edu_Prim25p 0.764332 0.764332 35.714286
2 Edu_UpSec25p 0.725495 0.725495 38.235294
3 FoodInsec_pct -0.693349 0.693349 42.016807
4 Edu_Bach25p 0.586804 0.586804 47.899160
5 ln_Ref_Asylum_t-1 0.552996 0.552996 0.000000
6 RuleLaw_rank 0.545991 0.545991 34.033613
7 GovEffect_rank 0.539856 0.539856 34.033613
8 LF_Fem 0.527798 0.527798 1.680672
9 LF_BasicEdu -0.521405 0.521405 21.008403
10 Enroll_Sec 0.516126 0.516126 35.714286
11 Unemp_AdvEdu -0.512115 0.512115 21.428571
12 LFPR_15_64_ILO 0.497893 0.497893 1.680672
13 Ref_Asylum 0.489153 0.489153 0.000000
14 Refugees 0.487681 0.487681 0.000000
15 GNI_pc 0.485333 0.485333 2.100840
16 Enroll_Tert 0.485075 0.485075 36.134454
17 Pol_Stab_rank 0.438945 0.438945 34.033613
18 Pop_Fem_pct 0.383133 0.383133 0.000000
19 Pop_Male_pct -0.383133 0.383133 0.000000
20 GDP_pc 0.382734 0.382734 1.260504
21 Unemp_Fem_Nat -0.355100 0.355100 15.966387
22 Unemp_Fem_ILO -0.348388 0.348388 1.680672
23 Unemp_BasicEdu 0.313410 0.313410 21.428571
24 ln_Ref_Asylum_t-2 0.311366 0.311366 0.000000
25 PrimComp_pct 0.310162 0.310162 43.277311
26 Pop_Growth -0.303291 0.303291 0.000000
27 Ref_Growth_LD 0.278001 0.278001 0.000000
28 GDP_Growth -0.239881 0.239881 2.100840
29 Unemp_Tot_Nat -0.210714 0.210714 15.966387
30 Unemp_Tot_ILO -0.203330 0.203330 1.680672
31 Asylum-seekers 0.198408 0.198408 0.000000
32 LF_AdvEdu 0.186202 0.186202 21.008403
33 Tot_Pop -0.183711 0.183711 0.000000
34 Pop_15_64_Fem -0.181670 0.181670 0.000000
35 Pop_15_64_Tot -0.181136 0.181136 0.000000
36 Pop_15_64_Male -0.180623 0.180623 0.000000
37 GDP_pc_Growth -0.119371 0.119371 2.100840
38 Unemp_Male_Nat -0.101633 0.101633 15.966387
39 Unemp_Male_ILO -0.098559 0.098559 1.680672
40 Enroll_Prim 0.078320 0.078320 31.512605
41 LFPR_15p_Nat 0.045324 0.045324 15.126050
42 LFPR_15_24_Nat 0.042950 0.042950 15.966387
43 GNI_pc_Growth -0.030822 0.030822 21.008403
44 Pop_15_64_pct -0.026782 0.026782 0.000000
45 LFPR_15_24_ILO 0.025132 0.025132 1.680672
46 LFPR_15p_ILO 0.016677 0.016677 1.680672
47 LF_IntEdu 0.012487 0.012487 21.848739

Now this is a great place to start with our model design in the next phase! As done below, we can pick and choose regressors, being careful to select correlated regressors without overselecting those that had lots of missing data originally.

Modeling¶

Now we are ready for the final phase of the project: modeling. This OLS regression model will be help tell us which political and economic indicators in a country may make them more likely to take more refugees in times of crisis.

OLS Regression¶

One important objective in OLS regressions is avoiding multicollinearity between regressors. If two or more variables are too highly correlated (meaning they would essentially measure the same relationship with the outcome variable) or have direct linear dependency, the model will struggle to estimate the unique effect of each variable. This would also lead to unstable coefficients and inflated standard errors, making proper statistical inference impossible.

I choose to modify two existing variables to avoid this. Below, we can see the coefficients between the measured government rankings are quite high, so I combine these into a composite rank index with equal weighting. Like with my dependent variable, I also log transform GNI per capita because it follows a similarly right-skewed distribution, and transforming it will enable me to find a more precise pattern from this variable.

In [69]:
# Calculate correlation between 'RuleLaw_rank', 'GovEffect_rank', and 'Pol_Stab_rank'
display(df_final[['RuleLaw_rank', 'GovEffect_rank', 'Pol_Stab_rank']].corr())
# Build composite index out of governance ranks due to correlation
df_final['Governance_index'] = (df_final['RuleLaw_rank'] + df_final['GovEffect_rank'] + df_final['Pol_Stab_rank']) / 3

# Log transform GNI_pc to fix skew; rich-country income differences are small relative to global range
df_final['ln_GNI_pc'] = np.log(df_final['GNI_pc'])
RuleLaw_rank GovEffect_rank Pol_Stab_rank
RuleLaw_rank 1.000000 0.946841 0.830163
GovEffect_rank 0.946841 1.000000 0.786557
Pol_Stab_rank 0.830163 0.786557 1.000000

Here, we run our final model. When I first ran this model with each regressor in level terms (except ln_GNI_pc and the lags, which are already transformed by definition), because the variables were in such vastly different unit spaces, the condition number was too high for proper inference. However, once I z-standardized the regressors, these estimates stabilized.

The only regressors that were not scaled are the two lags of refugee count and ln_GNI_pc to preserve comparability with the DV, as well as clean interpretability as elasticities.

In [70]:
import statsmodels.api as sm

regressors = ['ln_Ref_Asylum_t-1', 'ln_Ref_Asylum_t-2', 'Governance_index', 'LF_Fem', 'LFPR_15_64_ILO', 'Edu_UpSec25p', 'Unemp_Tot_ILO',
              'GDP_Growth', 'ln_GNI_pc', 'Pop_Growth', 'Pop_15_64_pct', 'FoodInsec_pct']
target = 'ln_Ref_Asylum'

X = df_final[regressors]
y = df_final[target].loc[X.index]

noscale_vars = ['ln_Ref_Asylum_t-1', 'ln_Ref_Asylum_t-2', 'ln_GNI_pc']
vars_to_scale = [col for col in regressors if col not in noscale_vars]

# Scale non-lag regressors to avoid high condition number caused by unit imbalance
X_scaled = X.copy()
X_scaled[vars_to_scale] = scaler.fit_transform(X[vars_to_scale])
# Add constant
X_scaled = sm.add_constant(X_scaled)
# Model with SEs clustered by country to correct for panel autocorrelation
model = sm.OLS(y, X_scaled).fit(cov_type='cluster', cov_kwds={'groups': df_final['Country of Asylum ISO']})
print(model.summary())
                            OLS Regression Results                            
==============================================================================
Dep. Variable:          ln_Ref_Asylum   R-squared:                       0.728
Model:                            OLS   Adj. R-squared:                  0.714
Method:                 Least Squares   F-statistic:                     85.42
Date:                Mon, 31 Aug 2026   Prob (F-statistic):           4.25e-41
Time:                        18:39:26   Log-Likelihood:                -497.77
No. Observations:                 238   AIC:                             1022.
Df Residuals:                     225   BIC:                             1067.
Df Model:                          12                                         
Covariance Type:              cluster                                         
=====================================================================================
                        coef    std err          z      P>|z|      [0.025      0.975]
-------------------------------------------------------------------------------------
const                -2.8515      4.308     -0.662      0.508     -11.295       5.592
ln_Ref_Asylum_t-1     0.2056      0.036      5.748      0.000       0.136       0.276
ln_Ref_Asylum_t-2    -0.0331      0.015     -2.175      0.030      -0.063      -0.003
Governance_index     -0.1069      0.441     -0.242      0.809      -0.972       0.758
LF_Fem                0.4259      0.372      1.144      0.253      -0.304       1.156
LFPR_15_64_ILO       -0.1538      0.456     -0.337      0.736      -1.047       0.740
Edu_UpSec25p          0.9690      0.372      2.607      0.009       0.241       1.697
Unemp_Tot_ILO         0.0009      0.232      0.004      0.997      -0.454       0.455
GDP_Growth            0.1970      0.160      1.231      0.218      -0.117       0.511
ln_GNI_pc             0.9772      0.446      2.190      0.029       0.103       1.852
Pop_Growth           -0.8084      0.242     -3.334      0.001      -1.284      -0.333
Pop_15_64_pct        -0.4313      0.249     -1.733      0.083      -0.919       0.056
FoodInsec_pct        -0.7850      0.278     -2.820      0.005      -1.331      -0.239
==============================================================================
Omnibus:                        3.396   Durbin-Watson:                   0.972
Prob(Omnibus):                  0.183   Jarque-Bera (JB):                3.091
Skew:                          -0.210   Prob(JB):                        0.213
Kurtosis:                       3.368   Cond. No.                         259.
==============================================================================

Notes:
[1] Standard Errors are robust to cluster correlation (cluster)

Results and Interpretation¶

In the regression table, we should only look at the statistically significant coefficients, meaning they have p-values less than or equal to 0.05. From our results, besides the two lag values that are obviously highly explanatory, the most significant predictive indicators are Edu_UpSec25p, ln_GNI_pc, Pop_Growth, and FoodInsec_pct. Significant at the 10% threshold is also Pop_15_64_pct.

For the scaled regressors, coefficients must be exponentiated and are interpreted in terms of standard deviations. For ln_GNI_pc, this result is interpreted as an elasticity because both it and the outcome variable are log-transformed.

For example, a one-SD increase in the share of the host population with educational attainment of at least upper secondary is associated with a 0.969 increase in log refugee popoulation, on average.

  • e^0.969 - 1 = 163% increase in refugees

Second example: A one-SD increase in the share of the population suffering from food insecurity is associated with a -0.785 decrease in logged refugees, on average.

  • e^-0.785 - 1 = 54.4% decrease in refugee population

Additionally, on average, a 1% increase in GNI per capita is associated with a 0.98% increase in refugee population: a near proportionate elasticity. This can be scaled up linearly to predict flows based on however much a host country may have increased their GNI per capita by in a given year, holding other factors constant.

Conclusion¶

Based on the positivity and negativity of certain coefficients in our regressions, we can identify various "push" and "pull" factors in the context of refugee movement. Positive coefficients on Edu_UpSec25p and ln_GNI_pc indicate that refugees cluster in wealthier, higher-educated, and institutionally capable socities. Negative coefficients on Pop_Growth and FoodInsec_pct indicate that refugees tend to not go to societies with existing internal socioeconomic stress. The hope is that insights like these can help governments and organizations be better prepared for future migrant crises.

Thanks for following along with this project!

Mount and Extract:¶

In [71]:
from google.colab import drive
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
In [72]:
!jupyter nbconvert --to html "/content/drive/MyDrive/CMPS 6160 (Data Sci)/Milestone 3"
[NbConvertApp] Converting notebook /content/drive/MyDrive/CMPS 6160 (Data Sci)/Milestone 3 to html
[NbConvertApp] Writing 523006 bytes to /content/drive/MyDrive/CMPS 6160 (Data Sci)/Milestone .html