Home Projects Topics Search Newsletter About

Running a regression on cleaned EOIR data#

Do certain judges or cities report refugees more often? This dataset is problematic and we're hardly experts, but let's take a look.

Read online Download notebook Interactive version 

import pandas as pd

pd.set_option("display.max_columns", 100)
pd.set_option("display.max_rows", 500)
pd.set_option("display.float_format", "{:,.5f}".format)

TODO: While this notebook is in theory complete, it needs to be revised to add a lot lot lot more words.

Read in data#

The data is split across two files - one for cases, and one for proceedings. We previously filtered them down to about 2 million cases from around 8-10 million, so we're working with the filtered version here.

You'll also need to download the original dataset from here for information on judges and nationalities.

cases = pd.read_csv("data/cases-filtered.csv")
cases.head()
IDNCASE NAT CUSTODY CASE_TYPE UPDATE_SITE DATE_OF_ENTRY
0 2048319 MX N RMV MIA 1954-08-06 00:00:00.000
1 2048321 MX N RMV HOU NaN
2 2048337 MX N RMV HOU 1992-01-01 00:00:00.000
3 2048340 CH N RMV LOS 2000-04-01 00:00:00.000
4 2047883 TH N RMV KAN 2002-05-10 00:00:00.000 
proceedings = pd.read_csv("data/proceedings-filtered.csv")
proceedings.head()
IDNCASE OSC_DATE IJ_CODE DEC_TYPE DEC_CODE COMP_DATE ABSENTIA CASE_TYPE
0 3320158 2000-02-16 00:00:00.000 THS O X 2002-08-23 00:00:00.000 N RMV
1 3320170 1998-12-23 00:00:00.000 HSD O T 1999-06-08 00:00:00.000 N RMV
2 3320191 1998-04-30 00:00:00.000 PLM NaN NaN 2000-05-09 00:00:00.000 N RMV
3 3320210 1997-06-03 00:00:00.000 BAN O V 2000-04-17 00:00:00.000 N RMV
4 3320218 1997-09-10 00:00:00.000 EAL O V 1999-04-14 00:00:00.000 N RMV

Merge the datasets#

We can now combine the two datasets based on the case IDs.

merged = cases.merge(proceedings, on='IDNCASE')
merged.shape

(3504464, 13)
merged.head(2)
IDNCASE NAT CUSTODY CASE_TYPE_x UPDATE_SITE DATE_OF_ENTRY OSC_DATE IJ_CODE DEC_TYPE DEC_CODE COMP_DATE ABSENTIA CASE_TYPE_y
0 2048319 MX N RMV MIA 1954-08-06 00:00:00.000 2001-02-13 00:00:00.000 PAM O T 2002-03-18 00:00:00.000 N RMV
1 2048321 MX N RMV HOU NaN 1999-06-10 00:00:00.000 CMR O T 2000-02-02 00:00:00.000 N RMV

Remove missing data#

We don't want anyone who is missing any of these fields. No date of entry? Out! No decision code? Out! No nationality? Out!

Let's take a quick look to see how much missing data there is.

merged.isna().mean()

IDNCASE         0.00000
NAT             0.00093
CUSTODY         0.00000
CASE_TYPE_x     0.00000
UPDATE_SITE     0.00001
DATE_OF_ENTRY   0.26022
OSC_DATE        0.00294
IJ_CODE         0.01560
DEC_TYPE        0.51286
DEC_CODE        0.62581
COMP_DATE       0.26548
ABSENTIA        0.26861
CASE_TYPE_y     0.00000
dtype: float64

It looks like around 63% don't have a decision, which of course we can't analyze. Looks like we're going to be losing a lot lot lot of rows, though.

print("Before dropping missing data:", merged.shape)
merged = merged.dropna()
print("After dropping missing data:", merged.shape)

Before dropping missing data: (3504464, 13)
After dropping missing data: (1002980, 13)

That hurts, but maybe it's for the best!

Analysis#

Decision codes#

What are the different decision codes? Let's use the lookup table to see.

codes = pd.read_csv("data/tblDecCode.csv", sep='\t')
codes
idnDecCode strCode strDescription datCreatedOn datModifiedOn blnActive
0 1 A LEGALLY ADMITTED 2003-08-10 11:22:47.617 NaN 1
1 2 C CONDITIONAL GRANT 2003-08-10 11:22:47.617 NaN 1
2 3 D DEPORTED 2003-08-10 11:22:47.617 NaN 1
3 4 E EXCLUDED 2003-08-10 11:22:47.617 NaN 1
4 5 G GRANTED 2003-08-10 11:22:47.617 2003-11-03 00:00:00.000 1
5 6 O OTHER 2003-08-10 11:22:47.617 NaN 1
6 7 R RELIEF/RESCINDED 2003-08-10 11:22:47.617 NaN 1
7 8 S ALIEN MAINTAINS LEGAL STATUS 2003-08-10 11:22:47.617 NaN 1
8 9 T CASE TERMINATED BY IJ 2003-08-10 11:22:47.617 NaN 1
9 10 V VOLUNTARY DEPARTURE 2003-08-10 11:22:47.617 NaN 1
10 11 W WITHDRAWN 2003-08-10 11:22:47.617 NaN 1
11 12 X REMOVED 2003-08-10 11:22:47.617 NaN 1

We're going to be interested in:

  • A - LEGALLY ADMITTED as successful
  • C - CONDITIONAL GRANT as successful
  • D - DEPORTED as unsuccessful
  • G - GRANTED as successful
  • R - RELIEF/RESCINDED as successful
  • X - REMOVED as unsuccessful

The others we'll remove, as we can't make a good judgment about what they really mean in the accepted/rejected spectrum.

Before we remove them, let's again see how common each code is.

merged.DEC_CODE.value_counts()

R    322356
X    257959
V    208949
T    192742
O      7625
D      4365
G      3426
Q      2913
I      1627
W       668
C       191
A       157
J         2
Name: DEC_CODE, dtype: int64

And now we'll filter for the decision codes we're interested in.

merged = merged[merged.DEC_CODE.isin(['A', 'C', 'D', 'G', 'R', 'X'])]
merged.head()
IDNCASE NAT CUSTODY CASE_TYPE_x UPDATE_SITE DATE_OF_ENTRY OSC_DATE IJ_CODE DEC_TYPE DEC_CODE COMP_DATE ABSENTIA CASE_TYPE_y
3 2048340 CH N RMV LOS 2000-04-01 00:00:00.000 2000-08-02 00:00:00.000 WJM O R 2000-12-04 00:00:00.000 N RMV
5 2047890 NN N RMV PHI 2049-08-31 00:00:00.000 1997-06-09 00:00:00.000 DVF O X 1998-04-23 00:00:00.000 N RMV
8 2047893 MX N RMV SFR 1954-06-25 00:00:00.000 2003-08-18 00:00:00.000 LLR O R 2004-11-29 00:00:00.000 N RMV
11 2047908 CA N RMV DET 1952-06-02 00:00:00.000 1997-06-10 00:00:00.000 RDN O R 2005-12-16 00:00:00.000 N RMV
12 2047908 CA N RMV DET 1952-06-02 00:00:00.000 1997-06-10 00:00:00.000 JFW O X 1998-12-11 00:00:00.000 N RMV 
merged.shape

(588454, 13)

Down to around 600,000 cases! Didn't we start with like ten million in the last notebook?

Feature engineering#

We're now going to create a feature as to whether asylum was granted or not. We'll count these:

  • A - LEGALLY ADMITTED
  • C - CONDITIONAL GRANT
  • G - GRANTED
  • R - RELIEF/RESCINDED

And the others - D - DEPORTED and X - REMOVED - we'll count as unsuccessful. We'll use the "turning true/false values into numbers" trick here.

merged['granted'] = merged.DEC_CODE.isin(['A', 'C', 'G', 'R']).astype(int)
merged.granted.value_counts()

1    326130
0    262324
Name: granted, dtype: int64
merged.DEC_TYPE.value_counts()

O    505468
W     79523
7      3460
6         2
C         1
Name: DEC_TYPE, dtype: int64
merged.head()
IDNCASE NAT CUSTODY CASE_TYPE_x UPDATE_SITE DATE_OF_ENTRY OSC_DATE IJ_CODE DEC_TYPE DEC_CODE COMP_DATE ABSENTIA CASE_TYPE_y granted
3 2048340 CH N RMV LOS 2000-04-01 00:00:00.000 2000-08-02 00:00:00.000 WJM O R 2000-12-04 00:00:00.000 N RMV 1
5 2047890 NN N RMV PHI 2049-08-31 00:00:00.000 1997-06-09 00:00:00.000 DVF O X 1998-04-23 00:00:00.000 N RMV 0
8 2047893 MX N RMV SFR 1954-06-25 00:00:00.000 2003-08-18 00:00:00.000 LLR O R 2004-11-29 00:00:00.000 N RMV 1
11 2047908 CA N RMV DET 1952-06-02 00:00:00.000 1997-06-10 00:00:00.000 RDN O R 2005-12-16 00:00:00.000 N RMV 1
12 2047908 CA N RMV DET 1952-06-02 00:00:00.000 1997-06-10 00:00:00.000 JFW O X 1998-12-11 00:00:00.000 N RMV 0

More filters#

We now want to filter our cases a bit more:

  • Only judges who have had a certain number of cases
  • Only sites that have had a certain number of cases
  • Only nationalities that have had a certain number of cases.

If they haven't had many occurrences, we can't reasonably pass judgment on them. We'll start by making a copy of our dataset so we can filter it again/differently later.

Filter judges#

Let's only look at judges with 300 or more cases.

common_judges = list(merged.IJ_CODE.value_counts()[merged.IJ_CODE.value_counts() >= 300].index)
has_common_judge = merged.IJ_CODE.isin(common_judges)

And sites that have shown up 300 or more times

common_sites = list(merged.UPDATE_SITE.value_counts()[merged.UPDATE_SITE.value_counts() >= 300].index)
has_common_site = merged.UPDATE_SITE.isin(common_sites)
has_common_site.value_counts()

True     586894
False      1560
Name: UPDATE_SITE, dtype: int64

And nationalities that have shown up... 300 times?

common_nats = list(merged.NAT.value_counts()[merged.NAT.value_counts() >= 300].index)
has_common_nationality = merged.NAT.isin(common_nats)
has_common_nationality.value_counts()

True     579619
False      8835
Name: NAT, dtype: int64
filtered = merged[has_common_judge & has_common_site & has_common_nationality]
filtered.shape

(548433, 14)

Inspecting our filtered results#

filtered.IJ_CODE.value_counts().head()

ROS    5817
PLM    5423
TAB    5320
BLF    5286
LTB    5219
Name: IJ_CODE, dtype: int64
filtered.UPDATE_SITE.value_counts().head()

NYC    117866
MIA     81559
LOS     58525
SFR     43356
ORL     21764
Name: UPDATE_SITE, dtype: int64
filtered.NAT.value_counts().head()

CH    79263
MX    71713
ES    43123
GT    36133
HA    30280
Name: NAT, dtype: int64

Perform our regression#

Now that we've successfully filtered our data, we can run our regression. We'll control for judges, cities, and nationalities. We'll be using the statsmodels formula method, with C() for categorical variables. We'll use ROS, NYC as reference since they're the most popular judge and site. We'll use CH (China) as the reference for nationality because the results are more readable than if we used Mexico.

No matter what you use as your reference the calculations are the same - it's just the point of comparison that is adjusted

%%time

import statsmodels.formula.api as smf

model = smf.logit("""
    granted ~ C(IJ_CODE, Treatment('ROS')) + C(UPDATE_SITE, Treatment('NYC')) + C(NAT, Treatment('CH'))
""", data=filtered)

CPU times: user 1min 13s, sys: 8.1 s, total: 1min 21s
Wall time: 1min 3s

It's a big dataset, so the actual fitting of the model takes a solid 7 minutes on my computer.

%%time

result = model.fit(method='bfgs', maxiter=1000)
result.summary()

Optimization terminated successfully.
         Current function value: 0.553741
         Iterations: 442
         Function evaluations: 443
         Gradient evaluations: 443
CPU times: user 10min 40s, sys: 19.8 s, total: 11min
Wall time: 5min 48s
Logit Regression Results
Dep. Variable: granted No. Observations: 548433
Model: Logit Df Residuals: 547919
Method: MLE Df Model: 513
Date: Sun, 19 Jan 2020 Pseudo R-squ.: 0.1915
Time: 13:36:03 Log-Likelihood: -3.0369e+05
converged: True LL-Null: -3.7561e+05
Covariance Type: nonrobust LLR p-value: 0.000
coef std err z P>|z| [0.025 0.975]
Intercept 0.0616 0.052 1.192 0.233 -0.040 0.163
C(IJ_CODE, Treatment('ROS'))[T.AA] 0.2650 0.116 2.276 0.023 0.037 0.493
C(IJ_CODE, Treatment('ROS'))[T.AAK] 0.8781 0.100 8.816 0.000 0.683 1.073
C(IJ_CODE, Treatment('ROS'))[T.AAS] 1.0328 0.158 6.519 0.000 0.722 1.343
C(IJ_CODE, Treatment('ROS'))[T.AAT] -0.3299 0.076 -4.322 0.000 -0.480 -0.180
C(IJ_CODE, Treatment('ROS'))[T.AAV] -0.6595 0.073 -9.012 0.000 -0.803 -0.516
C(IJ_CODE, Treatment('ROS'))[T.ABM] 0.0499 0.138 0.362 0.718 -0.221 0.321
C(IJ_CODE, Treatment('ROS'))[T.ACH] 0.9506 0.133 7.143 0.000 0.690 1.211
C(IJ_CODE, Treatment('ROS'))[T.ADM] -0.2655 0.144 -1.839 0.066 -0.548 0.017
C(IJ_CODE, Treatment('ROS'))[T.AED] 0.7910 0.095 8.336 0.000 0.605 0.977
C(IJ_CODE, Treatment('ROS'))[T.AEG] 1.0850 0.102 10.601 0.000 0.884 1.286
C(IJ_CODE, Treatment('ROS'))[T.AEM] -1.0406 0.157 -6.617 0.000 -1.349 -0.732
C(IJ_CODE, Treatment('ROS'))[T.AJR] 1.2478 0.088 14.208 0.000 1.076 1.420
C(IJ_CODE, Treatment('ROS'))[T.ALP] -0.2225 0.121 -1.833 0.067 -0.460 0.015
C(IJ_CODE, Treatment('ROS'))[T.AND] 0.4919 0.136 3.615 0.000 0.225 0.758
C(IJ_CODE, Treatment('ROS'))[T.AO] -0.6558 0.067 -9.757 0.000 -0.788 -0.524
C(IJ_CODE, Treatment('ROS'))[T.ARD] -0.0690 0.107 -0.646 0.518 -0.279 0.140
C(IJ_CODE, Treatment('ROS'))[T.ASE] 1.0163 0.070 14.525 0.000 0.879 1.153
C(IJ_CODE, Treatment('ROS'))[T.ASG] 0.6007 0.085 7.057 0.000 0.434 0.768
C(IJ_CODE, Treatment('ROS'))[T.ASL] 1.1510 0.082 13.968 0.000 0.990 1.313
C(IJ_CODE, Treatment('ROS'))[T.ASM] -0.6735 0.096 -7.047 0.000 -0.861 -0.486
C(IJ_CODE, Treatment('ROS'))[T.ATG] 0.1473 0.081 1.824 0.068 -0.011 0.306
C(IJ_CODE, Treatment('ROS'))[T.AVP] 0.3564 0.069 5.195 0.000 0.222 0.491
C(IJ_CODE, Treatment('ROS'))[T.BAN] -0.3424 0.061 -5.632 0.000 -0.461 -0.223
C(IJ_CODE, Treatment('ROS'))[T.BAR] 0.5409 0.092 5.854 0.000 0.360 0.722
C(IJ_CODE, Treatment('ROS'))[T.BAZ] -0.4705 0.097 -4.865 0.000 -0.660 -0.281
C(IJ_CODE, Treatment('ROS'))[T.BHA] -1.7065 0.099 -17.308 0.000 -1.900 -1.513
C(IJ_CODE, Treatment('ROS'))[T.BHS] -0.5059 0.093 -5.457 0.000 -0.688 -0.324
C(IJ_CODE, Treatment('ROS'))[T.BJE] 0.2841 0.090 3.163 0.002 0.108 0.460
C(IJ_CODE, Treatment('ROS'))[T.BJP] -0.1138 0.113 -1.011 0.312 -0.334 0.107
C(IJ_CODE, Treatment('ROS'))[T.BKS] -0.0131 0.077 -0.170 0.865 -0.164 0.137
C(IJ_CODE, Treatment('ROS'))[T.BLF] 1.1604 0.061 18.900 0.000 1.040 1.281
C(IJ_CODE, Treatment('ROS'))[T.BMB] -0.3922 0.090 -4.381 0.000 -0.568 -0.217
C(IJ_CODE, Treatment('ROS'))[T.BMO] -0.0044 0.126 -0.035 0.972 -0.252 0.243
C(IJ_CODE, Treatment('ROS'))[T.BMP] 0.2731 0.084 3.260 0.001 0.109 0.437
C(IJ_CODE, Treatment('ROS'))[T.BP1] -0.5799 0.073 -7.905 0.000 -0.724 -0.436
C(IJ_CODE, Treatment('ROS'))[T.BWS] -0.2720 0.066 -4.150 0.000 -0.400 -0.144
C(IJ_CODE, Treatment('ROS'))[T.CAB] -0.1570 0.076 -2.060 0.039 -0.306 -0.008
C(IJ_CODE, Treatment('ROS'))[T.CAD] -0.3937 0.095 -4.139 0.000 -0.580 -0.207
C(IJ_CODE, Treatment('ROS'))[T.CAK] 0.7315 0.081 9.070 0.000 0.573 0.890
C(IJ_CODE, Treatment('ROS'))[T.CBA] -0.6749 0.089 -7.619 0.000 -0.849 -0.501
C(IJ_CODE, Treatment('ROS'))[T.CC] 0.0011 0.089 0.012 0.991 -0.173 0.175
C(IJ_CODE, Treatment('ROS'))[T.CC1] -0.2162 0.118 -1.829 0.067 -0.448 0.016
C(IJ_CODE, Treatment('ROS'))[T.CDB] 0.1482 0.115 1.291 0.197 -0.077 0.373
C(IJ_CODE, Treatment('ROS'))[T.CEP] 0.2739 0.083 3.314 0.001 0.112 0.436
C(IJ_CODE, Treatment('ROS'))[T.CES] 0.3058 0.071 4.314 0.000 0.167 0.445
C(IJ_CODE, Treatment('ROS'))[T.CHC] 0.0475 0.093 0.511 0.609 -0.135 0.230
C(IJ_CODE, Treatment('ROS'))[T.CJL] 0.7796 0.100 7.835 0.000 0.585 0.975
C(IJ_CODE, Treatment('ROS'))[T.CJS] -0.5317 0.063 -8.388 0.000 -0.656 -0.407
C(IJ_CODE, Treatment('ROS'))[T.CM1] -0.8210 0.190 -4.329 0.000 -1.193 -0.449
C(IJ_CODE, Treatment('ROS'))[T.CMG] -0.8440 0.169 -4.998 0.000 -1.175 -0.513
C(IJ_CODE, Treatment('ROS'))[T.CMH] 0.4059 0.079 5.120 0.000 0.251 0.561
C(IJ_CODE, Treatment('ROS'))[T.CMR] -0.3523 0.081 -4.326 0.000 -0.512 -0.193
C(IJ_CODE, Treatment('ROS'))[T.CMW] 1.3187 0.194 6.795 0.000 0.938 1.699
C(IJ_CODE, Treatment('ROS'))[T.CMZ] -0.4522 0.089 -5.104 0.000 -0.626 -0.279
C(IJ_CODE, Treatment('ROS'))[T.CRH] -0.5533 0.090 -6.156 0.000 -0.729 -0.377
C(IJ_CODE, Treatment('ROS'))[T.CSG] 0.8985 0.115 7.834 0.000 0.674 1.123
C(IJ_CODE, Treatment('ROS'))[T.CYG] 0.0753 0.125 0.604 0.546 -0.169 0.320
C(IJ_CODE, Treatment('ROS'))[T.DA] 0.2814 0.095 2.964 0.003 0.095 0.468
C(IJ_CODE, Treatment('ROS'))[T.DAL] 0.5449 0.162 3.356 0.001 0.227 0.863
C(IJ_CODE, Treatment('ROS'))[T.DAM] 0.0052 0.107 0.048 0.962 -0.204 0.215
C(IJ_CODE, Treatment('ROS'))[T.DAR] -0.0532 0.099 -0.540 0.589 -0.246 0.140
C(IJ_CODE, Treatment('ROS'))[T.DB] -0.7022 0.116 -6.067 0.000 -0.929 -0.475
C(IJ_CODE, Treatment('ROS'))[T.DBS] 1.0389 0.061 17.008 0.000 0.919 1.159
C(IJ_CODE, Treatment('ROS'))[T.DCA] -0.0816 0.079 -1.030 0.303 -0.237 0.074
C(IJ_CODE, Treatment('ROS'))[T.DDB] 0.0133 0.074 0.180 0.857 -0.131 0.158
C(IJ_CODE, Treatment('ROS'))[T.DDS] -0.4187 0.084 -4.959 0.000 -0.584 -0.253
C(IJ_CODE, Treatment('ROS'))[T.DHP] -0.4297 0.138 -3.124 0.002 -0.699 -0.160
C(IJ_CODE, Treatment('ROS'))[T.DHS] -1.2996 0.096 -13.589 0.000 -1.487 -1.112
C(IJ_CODE, Treatment('ROS'))[T.DJC] 0.1903 0.112 1.694 0.090 -0.030 0.410
C(IJ_CODE, Treatment('ROS'))[T.DK1] -0.1912 0.122 -1.564 0.118 -0.431 0.048
C(IJ_CODE, Treatment('ROS'))[T.DKG] -1.3977 0.156 -8.957 0.000 -1.704 -1.092
C(IJ_CODE, Treatment('ROS'))[T.DL] -0.4975 0.064 -7.724 0.000 -0.624 -0.371
C(IJ_CODE, Treatment('ROS'))[T.DLL] 0.2095 0.106 1.977 0.048 0.002 0.417
C(IJ_CODE, Treatment('ROS'))[T.DLM] 0.8285 0.078 10.565 0.000 0.675 0.982
C(IJ_CODE, Treatment('ROS'))[T.DMB] 0.4779 0.166 2.873 0.004 0.152 0.804
C(IJ_CODE, Treatment('ROS'))[T.DML] -0.2921 0.065 -4.516 0.000 -0.419 -0.165
C(IJ_CODE, Treatment('ROS'))[T.DNS] 1.2023 0.068 17.792 0.000 1.070 1.335
C(IJ_CODE, Treatment('ROS'))[T.DRH] -0.3367 0.136 -2.482 0.013 -0.603 -0.071
C(IJ_CODE, Treatment('ROS'))[T.DRJ] 0.1973 0.163 1.209 0.227 -0.123 0.517
C(IJ_CODE, Treatment('ROS'))[T.DRS] 0.1822 0.119 1.533 0.125 -0.051 0.415
C(IJ_CODE, Treatment('ROS'))[T.DVF] -1.0206 0.094 -10.872 0.000 -1.205 -0.837
C(IJ_CODE, Treatment('ROS'))[T.DWC] -0.0845 0.074 -1.144 0.253 -0.229 0.060
C(IJ_CODE, Treatment('ROS'))[T.DWE] -0.7168 0.122 -5.854 0.000 -0.957 -0.477
C(IJ_CODE, Treatment('ROS'))[T.DWT] 0.8469 0.098 8.673 0.000 0.655 1.038
C(IJ_CODE, Treatment('ROS'))[T.EAH] -0.9363 0.093 -10.073 0.000 -1.118 -0.754
C(IJ_CODE, Treatment('ROS'))[T.EAK] 0.2622 0.103 2.550 0.011 0.061 0.464
C(IJ_CODE, Treatment('ROS'))[T.EAL] 1.2234 0.064 19.035 0.000 1.097 1.349
C(IJ_CODE, Treatment('ROS'))[T.EBW] -0.9960 0.063 -15.886 0.000 -1.119 -0.873
C(IJ_CODE, Treatment('ROS'))[T.ECC] 0.3453 0.138 2.498 0.013 0.074 0.616
C(IJ_CODE, Treatment('ROS'))[T.ECK] 0.4608 0.089 5.170 0.000 0.286 0.635
C(IJ_CODE, Treatment('ROS'))[T.EDS] 0.9592 0.145 6.625 0.000 0.675 1.243
C(IJ_CODE, Treatment('ROS'))[T.EGL] -0.0877 0.134 -0.656 0.512 -0.350 0.175
C(IJ_CODE, Treatment('ROS'))[T.ELY] 0.4238 0.124 3.424 0.001 0.181 0.666
C(IJ_CODE, Treatment('ROS'))[T.EM2] -0.2368 0.182 -1.303 0.192 -0.593 0.119
C(IJ_CODE, Treatment('ROS'))[T.EMS] -0.1844 0.068 -2.723 0.006 -0.317 -0.052
C(IJ_CODE, Treatment('ROS'))[T.EP] -0.1018 0.095 -1.070 0.285 -0.288 0.085
C(IJ_CODE, Treatment('ROS'))[T.EPD] 1.1608 0.103 11.265 0.000 0.959 1.363
C(IJ_CODE, Treatment('ROS'))[T.ERH] 0.2855 0.097 2.934 0.003 0.095 0.476
C(IJ_CODE, Treatment('ROS'))[T.ERK] 0.0694 0.071 0.974 0.330 -0.070 0.209
C(IJ_CODE, Treatment('ROS'))[T.ERT] 0.4745 0.146 3.244 0.001 0.188 0.761
C(IJ_CODE, Treatment('ROS'))[T.ET] 0.0136 0.115 0.118 0.906 -0.212 0.240
C(IJ_CODE, Treatment('ROS'))[T.FGL] 1.1772 0.082 14.316 0.000 1.016 1.338
C(IJ_CODE, Treatment('ROS'))[T.FLC] 0.1412 0.107 1.323 0.186 -0.068 0.350
C(IJ_CODE, Treatment('ROS'))[T.FMT] -0.1419 0.085 -1.662 0.096 -0.309 0.025
C(IJ_CODE, Treatment('ROS'))[T.GA] 0.6913 0.077 8.934 0.000 0.540 0.843
C(IJ_CODE, Treatment('ROS'))[T.GAS] -1.0004 0.171 -5.845 0.000 -1.336 -0.665
C(IJ_CODE, Treatment('ROS'))[T.GBP] 0.2344 0.105 2.227 0.026 0.028 0.441
C(IJ_CODE, Treatment('ROS'))[T.GCV] -0.1292 0.063 -2.056 0.040 -0.252 -0.006
C(IJ_CODE, Treatment('ROS'))[T.GDB] -2.1635 0.111 -19.466 0.000 -2.381 -1.946
C(IJ_CODE, Treatment('ROS'))[T.GDM] -0.4260 0.110 -3.857 0.000 -0.642 -0.210
C(IJ_CODE, Treatment('ROS'))[T.GEE] -1.7060 0.098 -17.440 0.000 -1.898 -1.514
C(IJ_CODE, Treatment('ROS'))[T.GLB] 0.8280 0.147 5.651 0.000 0.541 1.115
C(IJ_CODE, Treatment('ROS'))[T.GMP] -2.1079 0.148 -14.288 0.000 -2.397 -1.819
C(IJ_CODE, Treatment('ROS'))[T.GMR] -0.4729 0.124 -3.803 0.000 -0.717 -0.229
C(IJ_CODE, Treatment('ROS'))[T.GNB] 0.3013 0.130 2.315 0.021 0.046 0.556
C(IJ_CODE, Treatment('ROS'))[T.GPM] -0.5093 0.104 -4.877 0.000 -0.714 -0.305
C(IJ_CODE, Treatment('ROS'))[T.GRB] -0.2006 0.145 -1.387 0.166 -0.484 0.083
C(IJ_CODE, Treatment('ROS'))[T.GTC] 1.1741 0.063 18.774 0.000 1.052 1.297
C(IJ_CODE, Treatment('ROS'))[T.GTG] 0.4122 0.080 5.152 0.000 0.255 0.569
C(IJ_CODE, Treatment('ROS'))[T.GWP] 0.7454 0.101 7.392 0.000 0.548 0.943
C(IJ_CODE, Treatment('ROS'))[T.GWR] -0.9676 0.180 -5.379 0.000 -1.320 -0.615
C(IJ_CODE, Treatment('ROS'))[T.HAR] -2.6411 0.140 -18.833 0.000 -2.916 -2.366
C(IJ_CODE, Treatment('ROS'))[T.HEA] -1.3087 0.148 -8.863 0.000 -1.598 -1.019
C(IJ_CODE, Treatment('ROS'))[T.HJS] 0.9937 0.063 15.846 0.000 0.871 1.117
C(IJ_CODE, Treatment('ROS'))[T.HKM] 0.3544 0.106 3.330 0.001 0.146 0.563
C(IJ_CODE, Treatment('ROS'))[T.HLG] -0.0170 0.109 -0.156 0.876 -0.231 0.197
C(IJ_CODE, Treatment('ROS'))[T.HLK] -0.2637 0.093 -2.829 0.005 -0.446 -0.081
C(IJ_CODE, Treatment('ROS'))[T.HO] -0.3731 0.073 -5.146 0.000 -0.515 -0.231
C(IJ_CODE, Treatment('ROS'))[T.HPI] -0.1497 0.076 -1.968 0.049 -0.299 -0.001
C(IJ_CODE, Treatment('ROS'))[T.HSD] 0.4090 0.088 4.622 0.000 0.236 0.582
C(IJ_CODE, Treatment('ROS'))[T.HVW] -1.0893 0.158 -6.883 0.000 -1.399 -0.779
C(IJ_CODE, Treatment('ROS'))[T.IBB] -0.3933 0.076 -5.183 0.000 -0.542 -0.245
C(IJ_CODE, Treatment('ROS'))[T.ICD] 2.0892 0.187 11.194 0.000 1.723 2.455
C(IJ_CODE, Treatment('ROS'))[T.IEB] 0.0304 0.071 0.431 0.666 -0.108 0.169
C(IJ_CODE, Treatment('ROS'))[T.IKH] 0.2705 0.096 2.830 0.005 0.083 0.458
C(IJ_CODE, Treatment('ROS'))[T.ILD] -0.2433 0.198 -1.230 0.219 -0.631 0.144
C(IJ_CODE, Treatment('ROS'))[T.IPF] 0.4286 0.090 4.737 0.000 0.251 0.606
C(IJ_CODE, Treatment('ROS'))[T.IS] -0.4861 0.115 -4.244 0.000 -0.711 -0.262
C(IJ_CODE, Treatment('ROS'))[T.IW] 0.9564 0.112 8.537 0.000 0.737 1.176
C(IJ_CODE, Treatment('ROS'))[T.JAD] 0.2361 0.124 1.910 0.056 -0.006 0.478
C(IJ_CODE, Treatment('ROS'))[T.JB] 0.6628 0.064 10.395 0.000 0.538 0.788
C(IJ_CODE, Treatment('ROS'))[T.JBC] 0.2248 0.068 3.301 0.001 0.091 0.358
C(IJ_CODE, Treatment('ROS'))[T.JBL] -0.5429 0.155 -3.504 0.000 -0.847 -0.239
C(IJ_CODE, Treatment('ROS'))[T.JBV] 0.2937 0.102 2.891 0.004 0.095 0.493
C(IJ_CODE, Treatment('ROS'))[T.JCA] -0.8251 0.142 -5.813 0.000 -1.103 -0.547
C(IJ_CODE, Treatment('ROS'))[T.JCL] -1.5033 0.186 -8.078 0.000 -1.868 -1.139
C(IJ_CODE, Treatment('ROS'))[T.JCS] 0.6225 0.096 6.492 0.000 0.435 0.810
C(IJ_CODE, Treatment('ROS'))[T.JCW] 1.1855 0.096 12.293 0.000 0.997 1.375
C(IJ_CODE, Treatment('ROS'))[T.JD] 0.3345 0.123 2.714 0.007 0.093 0.576
C(IJ_CODE, Treatment('ROS'))[T.JDC] -0.8744 0.105 -8.360 0.000 -1.079 -0.669
C(IJ_CODE, Treatment('ROS'))[T.JDD] -0.4166 0.063 -6.585 0.000 -0.541 -0.293
C(IJ_CODE, Treatment('ROS'))[T.JDL] 0.0320 0.073 0.438 0.661 -0.111 0.175
C(IJ_CODE, Treatment('ROS'))[T.JDP] -1.1107 0.132 -8.403 0.000 -1.370 -0.852
C(IJ_CODE, Treatment('ROS'))[T.JDT] -0.2847 0.094 -3.024 0.002 -0.469 -0.100
C(IJ_CODE, Treatment('ROS'))[T.JEB] 0.5289 0.065 8.156 0.000 0.402 0.656
C(IJ_CODE, Treatment('ROS'))[T.JFG] 0.1563 0.084 1.851 0.064 -0.009 0.322
C(IJ_CODE, Treatment('ROS'))[T.JFW] -0.0901 0.069 -1.310 0.190 -0.225 0.045
C(IJ_CODE, Treatment('ROS'))[T.JG] -0.4782 0.065 -7.349 0.000 -0.606 -0.351
C(IJ_CODE, Treatment('ROS'))[T.JHD] -0.2026 0.086 -2.365 0.018 -0.371 -0.035
C(IJ_CODE, Treatment('ROS'))[T.JIP] 1.2224 0.178 6.878 0.000 0.874 1.571
C(IJ_CODE, Treatment('ROS'))[T.JLA] -0.2109 0.123 -1.716 0.086 -0.452 0.030
C(IJ_CODE, Treatment('ROS'))[T.JLB] -0.1917 0.114 -1.680 0.093 -0.415 0.032
C(IJ_CODE, Treatment('ROS'))[T.JLG] 0.4223 0.093 4.557 0.000 0.241 0.604
C(IJ_CODE, Treatment('ROS'))[T.JLP] 1.0091 0.084 12.080 0.000 0.845 1.173
C(IJ_CODE, Treatment('ROS'))[T.JLR] -1.0412 0.082 -12.680 0.000 -1.202 -0.880
C(IJ_CODE, Treatment('ROS'))[T.JMB] 0.1278 0.082 1.560 0.119 -0.033 0.288
C(IJ_CODE, Treatment('ROS'))[T.JMG] -0.3954 0.124 -3.182 0.001 -0.639 -0.152
C(IJ_CODE, Treatment('ROS'))[T.JMM] -0.0868 0.096 -0.904 0.366 -0.275 0.101
C(IJ_CODE, Treatment('ROS'))[T.JN] -0.2700 0.130 -2.082 0.037 -0.524 -0.016
C(IJ_CODE, Treatment('ROS'))[T.JO] -0.7014 0.067 -10.396 0.000 -0.834 -0.569
C(IJ_CODE, Treatment('ROS'))[T.JPL] -1.2726 0.163 -7.792 0.000 -1.593 -0.952
C(IJ_CODE, Treatment('ROS'))[T.JPV] -0.2340 0.124 -1.891 0.059 -0.477 0.008
C(IJ_CODE, Treatment('ROS'))[T.JRF] -0.1143 0.092 -1.248 0.212 -0.294 0.065
C(IJ_CODE, Treatment('ROS'))[T.JRO] 1.4401 0.120 11.967 0.000 1.204 1.676
C(IJ_CODE, Treatment('ROS'))[T.JS] -0.4560 0.097 -4.693 0.000 -0.646 -0.266
C(IJ_CODE, Treatment('ROS'))[T.JSC] -0.2029 0.077 -2.628 0.009 -0.354 -0.052
C(IJ_CODE, Treatment('ROS'))[T.JSL] -1.0638 0.188 -5.646 0.000 -1.433 -0.694
C(IJ_CODE, Treatment('ROS'))[T.JTH] -0.7453 0.121 -6.167 0.000 -0.982 -0.508
C(IJ_CODE, Treatment('ROS'))[T.JVC] -0.4352 0.092 -4.705 0.000 -0.617 -0.254
C(IJ_CODE, Treatment('ROS'))[T.JVJ] 0.9444 0.145 6.517 0.000 0.660 1.228
C(IJ_CODE, Treatment('ROS'))[T.JWH] -0.2836 0.130 -2.181 0.029 -0.538 -0.029
C(IJ_CODE, Treatment('ROS'))[T.JWR] 1.1191 0.124 9.007 0.000 0.876 1.363
C(IJ_CODE, Treatment('ROS'))[T.JWS] -0.0964 0.162 -0.594 0.552 -0.414 0.222
C(IJ_CODE, Treatment('ROS'))[T.KAB] -0.1215 0.096 -1.263 0.207 -0.310 0.067
C(IJ_CODE, Treatment('ROS'))[T.KC] -0.7144 0.056 -12.667 0.000 -0.825 -0.604
C(IJ_CODE, Treatment('ROS'))[T.KCW] -1.1074 0.091 -12.120 0.000 -1.287 -0.928
C(IJ_CODE, Treatment('ROS'))[T.KGB] -0.4454 0.071 -6.279 0.000 -0.584 -0.306
C(IJ_CODE, Treatment('ROS'))[T.KJ] 0.7758 0.072 10.797 0.000 0.635 0.917
C(IJ_CODE, Treatment('ROS'))[T.KW] -0.2815 0.126 -2.234 0.025 -0.529 -0.035
C(IJ_CODE, Treatment('ROS'))[T.KWO] 0.0889 0.123 0.725 0.468 -0.151 0.329
C(IJ_CODE, Treatment('ROS'))[T.LB7] -0.0339 0.088 -0.387 0.698 -0.206 0.138
C(IJ_CODE, Treatment('ROS'))[T.LD] -0.0051 0.082 -0.062 0.951 -0.166 0.155
C(IJ_CODE, Treatment('ROS'))[T.LIS] 0.5184 0.090 5.760 0.000 0.342 0.695
C(IJ_CODE, Treatment('ROS'))[T.LJM] -0.4339 0.078 -5.549 0.000 -0.587 -0.281
C(IJ_CODE, Treatment('ROS'))[T.LLR] 0.9469 0.078 12.075 0.000 0.793 1.101
C(IJ_CODE, Treatment('ROS'))[T.LME] -0.2705 0.083 -3.253 0.001 -0.433 -0.108
C(IJ_CODE, Treatment('ROS'))[T.LND] 0.4301 0.084 5.092 0.000 0.265 0.596
C(IJ_CODE, Treatment('ROS'))[T.LOB] 0.4692 0.070 6.656 0.000 0.331 0.607
C(IJ_CODE, Treatment('ROS'))[T.LOC] -1.1217 0.094 -11.914 0.000 -1.306 -0.937
C(IJ_CODE, Treatment('ROS'))[T.LRJ] 0.7615 0.101 7.572 0.000 0.564 0.959
C(IJ_CODE, Treatment('ROS'))[T.LSF] 0.2115 0.122 1.730 0.084 -0.028 0.451
C(IJ_CODE, Treatment('ROS'))[T.LSG] 0.5040 0.078 6.441 0.000 0.351 0.657
C(IJ_CODE, Treatment('ROS'))[T.LTB] -0.2609 0.062 -4.217 0.000 -0.382 -0.140
C(IJ_CODE, Treatment('ROS'))[T.MAT] 0.2352 0.108 2.172 0.030 0.023 0.447
C(IJ_CODE, Treatment('ROS'))[T.MB] 0.3074 0.108 2.834 0.005 0.095 0.520
C(IJ_CODE, Treatment('ROS'))[T.MCG] -0.0396 0.100 -0.395 0.693 -0.236 0.157
C(IJ_CODE, Treatment('ROS'))[T.MCH] 0.1048 0.067 1.555 0.120 -0.027 0.237
C(IJ_CODE, Treatment('ROS'))[T.MEN] 1.4050 0.125 11.280 0.000 1.161 1.649
C(IJ_CODE, Treatment('ROS'))[T.MFH] -1.3296 0.072 -18.507 0.000 -1.470 -1.189
C(IJ_CODE, Treatment('ROS'))[T.MG1] -2.2438 0.182 -12.296 0.000 -2.601 -1.886
C(IJ_CODE, Treatment('ROS'))[T.MGK] 1.1290 0.110 10.260 0.000 0.913 1.345
C(IJ_CODE, Treatment('ROS'))[T.MGM] -1.0052 0.122 -8.226 0.000 -1.245 -0.766
C(IJ_CODE, Treatment('ROS'))[T.MH] 1.0570 0.082 12.844 0.000 0.896 1.218
C(IJ_CODE, Treatment('ROS'))[T.MHB] 0.9792 0.110 8.898 0.000 0.764 1.195
C(IJ_CODE, Treatment('ROS'))[T.MHM] -0.3551 0.109 -3.267 0.001 -0.568 -0.142
C(IJ_CODE, Treatment('ROS'))[T.MJD] 0.3688 0.092 4.000 0.000 0.188 0.550
C(IJ_CODE, Treatment('ROS'))[T.MJT] 0.2330 0.079 2.951 0.003 0.078 0.388
C(IJ_CODE, Treatment('ROS'))[T.MJY] -0.2097 0.109 -1.925 0.054 -0.423 0.004
C(IJ_CODE, Treatment('ROS'))[T.MKM] -0.1176 0.075 -1.574 0.115 -0.264 0.029
C(IJ_CODE, Treatment('ROS'))[T.MKN] -0.9104 0.100 -9.081 0.000 -1.107 -0.714
C(IJ_CODE, Treatment('ROS'))[T.MLE] -0.2901 0.098 -2.947 0.003 -0.483 -0.097
C(IJ_CODE, Treatment('ROS'))[T.MLY] 1.0938 0.112 9.753 0.000 0.874 1.314
C(IJ_CODE, Treatment('ROS'))[T.MMC] 0.6014 0.085 7.096 0.000 0.435 0.768
C(IJ_CODE, Treatment('ROS'))[T.MMF] -0.0048 0.129 -0.037 0.970 -0.258 0.249
C(IJ_CODE, Treatment('ROS'))[T.MML] -0.1826 0.102 -1.792 0.073 -0.382 0.017
C(IJ_CODE, Treatment('ROS'))[T.MMT] 0.5446 0.094 5.775 0.000 0.360 0.729
C(IJ_CODE, Treatment('ROS'))[T.MOS] 0.8935 0.094 9.465 0.000 0.708 1.079
C(IJ_CODE, Treatment('ROS'))[T.MPB] -1.3318 0.111 -11.977 0.000 -1.550 -1.114
C(IJ_CODE, Treatment('ROS'))[T.MR] -1.4740 0.204 -7.220 0.000 -1.874 -1.074
C(IJ_CODE, Treatment('ROS'))[T.MRR] 0.0397 0.093 0.427 0.670 -0.143 0.222
C(IJ_CODE, Treatment('ROS'))[T.MS1] -2.6524 0.220 -12.071 0.000 -3.083 -2.222
C(IJ_CODE, Treatment('ROS'))[T.MSY] -0.6600 0.081 -8.159 0.000 -0.819 -0.501
C(IJ_CODE, Treatment('ROS'))[T.MT] -0.3396 0.129 -2.625 0.009 -0.593 -0.086
C(IJ_CODE, Treatment('ROS'))[T.MWK] -0.8217 0.119 -6.915 0.000 -1.055 -0.589
C(IJ_CODE, Treatment('ROS'))[T.MWS] -0.0266 0.098 -0.272 0.786 -0.218 0.165
C(IJ_CODE, Treatment('ROS'))[T.NAF] 0.5887 0.077 7.610 0.000 0.437 0.740
C(IJ_CODE, Treatment('ROS'))[T.NB] 1.7849 0.068 26.124 0.000 1.651 1.919
C(IJ_CODE, Treatment('ROS'))[T.NK] 0.2112 0.143 1.479 0.139 -0.069 0.491
C(IJ_CODE, Treatment('ROS'))[T.NRM] -0.1072 0.068 -1.581 0.114 -0.240 0.026
C(IJ_CODE, Treatment('ROS'))[T.ODS] -1.2603 0.170 -7.420 0.000 -1.593 -0.927
C(IJ_CODE, Treatment('ROS'))[T.OJB] -0.7417 0.089 -8.305 0.000 -0.917 -0.567
C(IJ_CODE, Treatment('ROS'))[T.OLC] 1.6656 0.112 14.924 0.000 1.447 1.884
C(IJ_CODE, Treatment('ROS'))[T.OMB] 0.6420 0.098 6.550 0.000 0.450 0.834
C(IJ_CODE, Treatment('ROS'))[T.PAM] 0.1901 0.066 2.862 0.004 0.060 0.320
C(IJ_CODE, Treatment('ROS'))[T.PAR] 1.4008 0.066 21.326 0.000 1.272 1.530
C(IJ_CODE, Treatment('ROS'))[T.PAW] 0.3149 0.075 4.219 0.000 0.169 0.461
C(IJ_CODE, Treatment('ROS'))[T.PD1] 1.5684 0.138 11.362 0.000 1.298 1.839
C(IJ_CODE, Treatment('ROS'))[T.PDF] -0.1243 0.063 -1.970 0.049 -0.248 -0.001
C(IJ_CODE, Treatment('ROS'))[T.PDM] 0.5282 0.129 4.108 0.000 0.276 0.780
C(IJ_CODE, Treatment('ROS'))[T.PDO] 1.0556 0.135 7.803 0.000 0.790 1.321
C(IJ_CODE, Treatment('ROS'))[T.PG] 0.5919 0.116 5.087 0.000 0.364 0.820
C(IJ_CODE, Treatment('ROS'))[T.PJ] -0.0719 0.138 -0.521 0.602 -0.342 0.198
C(IJ_CODE, Treatment('ROS'))[T.PJC] 0.0912 0.102 0.896 0.370 -0.108 0.291
C(IJ_CODE, Treatment('ROS'))[T.PJM] -1.8175 0.184 -9.875 0.000 -2.178 -1.457
C(IJ_CODE, Treatment('ROS'))[T.PKM] -2.3267 0.165 -14.100 0.000 -2.650 -2.003
C(IJ_CODE, Treatment('ROS'))[T.PLJ] -1.3421 0.145 -9.280 0.000 -1.626 -1.059
C(IJ_CODE, Treatment('ROS'))[T.PLM] 1.2123 0.061 19.727 0.000 1.092 1.333
C(IJ_CODE, Treatment('ROS'))[T.PMG] 0.7396 0.091 8.123 0.000 0.561 0.918
C(IJ_CODE, Treatment('ROS'))[T.PMM] 2.1254 0.072 29.338 0.000 1.983 2.267
C(IJ_CODE, Treatment('ROS'))[T.PMS] -1.2417 0.148 -8.409 0.000 -1.531 -0.952
C(IJ_CODE, Treatment('ROS'))[T.PMT] -1.4770 0.199 -7.411 0.000 -1.868 -1.086
C(IJ_CODE, Treatment('ROS'))[T.PRM] 0.8959 0.136 6.594 0.000 0.630 1.162
C(IJ_CODE, Treatment('ROS'))[T.PS] -0.0319 0.096 -0.331 0.741 -0.221 0.157
C(IJ_CODE, Treatment('ROS'))[T.PSL] -0.3446 0.078 -4.446 0.000 -0.497 -0.193
C(IJ_CODE, Treatment('ROS'))[T.PSS] 1.4075 0.139 10.147 0.000 1.136 1.679
C(IJ_CODE, Treatment('ROS'))[T.PTM] 0.1766 0.117 1.505 0.132 -0.053 0.407
C(IJ_CODE, Treatment('ROS'))[T.PTW] -0.0249 0.084 -0.296 0.767 -0.190 0.140
C(IJ_CODE, Treatment('ROS'))[T.PV] 0.8428 0.110 7.638 0.000 0.627 1.059
C(IJ_CODE, Treatment('ROS'))[T.PWS] 0.5016 0.092 5.458 0.000 0.321 0.682
C(IJ_CODE, Treatment('ROS'))[T.QVB] -0.0978 0.091 -1.073 0.283 -0.277 0.081
C(IJ_CODE, Treatment('ROS'))[T.RAC] 1.5378 0.116 13.307 0.000 1.311 1.764
C(IJ_CODE, Treatment('ROS'))[T.RAJ] 0.2575 0.070 3.669 0.000 0.120 0.395
C(IJ_CODE, Treatment('ROS'))[T.RB] -0.4009 0.102 -3.932 0.000 -0.601 -0.201
C(IJ_CODE, Treatment('ROS'))[T.RBJ] 1.4325 0.161 8.885 0.000 1.116 1.748
C(IJ_CODE, Treatment('ROS'))[T.RCH] -0.5166 0.066 -7.822 0.000 -0.646 -0.387
C(IJ_CODE, Treatment('ROS'))[T.RCP] -0.4023 0.092 -4.387 0.000 -0.582 -0.223
C(IJ_CODE, Treatment('ROS'))[T.RDM] 0.4258 0.111 3.853 0.000 0.209 0.642
C(IJ_CODE, Treatment('ROS'))[T.RDN] -0.8202 0.086 -9.514 0.000 -0.989 -0.651
C(IJ_CODE, Treatment('ROS'))[T.RDV] -0.3323 0.083 -3.981 0.000 -0.496 -0.169
C(IJ_CODE, Treatment('ROS'))[T.RDW] 0.3377 0.062 5.440 0.000 0.216 0.459
C(IJ_CODE, Treatment('ROS'))[T.REC] -0.4335 0.121 -3.589 0.000 -0.670 -0.197
C(IJ_CODE, Treatment('ROS'))[T.REF] 0.8525 0.092 9.281 0.000 0.672 1.033
C(IJ_CODE, Treatment('ROS'))[T.RFB] -0.1708 0.130 -1.313 0.189 -0.426 0.084
C(IJ_CODE, Treatment('ROS'))[T.RGS] -0.4602 0.076 -6.033 0.000 -0.610 -0.311
C(IJ_CODE, Treatment('ROS'))[T.RHC] 0.1984 0.083 2.376 0.017 0.035 0.362
C(IJ_CODE, Treatment('ROS'))[T.RHK] 0.9381 0.152 6.175 0.000 0.640 1.236
C(IJ_CODE, Treatment('ROS'))[T.RJ1] -1.6959 0.157 -10.791 0.000 -2.004 -1.388
C(IJ_CODE, Treatment('ROS'))[T.RJA] -0.9516 0.119 -8.015 0.000 -1.184 -0.719
C(IJ_CODE, Treatment('ROS'))[T.RJB] 0.7770 0.101 7.696 0.000 0.579 0.975
C(IJ_CODE, Treatment('ROS'))[T.RJF] -0.9414 0.067 -13.966 0.000 -1.074 -0.809
C(IJ_CODE, Treatment('ROS'))[T.RKM] -1.3698 0.111 -12.394 0.000 -1.586 -1.153
C(IJ_CODE, Treatment('ROS'))[T.RKP] 0.7454 0.123 6.045 0.000 0.504 0.987
C(IJ_CODE, Treatment('ROS'))[T.RLH] -0.3131 0.090 -3.473 0.001 -0.490 -0.136
C(IJ_CODE, Treatment('ROS'))[T.RLM] -0.1065 0.114 -0.931 0.352 -0.331 0.118
C(IJ_CODE, Treatment('ROS'))[T.RLR] -0.1213 0.081 -1.502 0.133 -0.280 0.037
C(IJ_CODE, Treatment('ROS'))[T.RM] -0.0165 0.068 -0.244 0.807 -0.149 0.116
C(IJ_CODE, Treatment('ROS'))[T.RMH] -2.4511 0.277 -8.839 0.000 -2.995 -1.908
C(IJ_CODE, Treatment('ROS'))[T.RPM] 0.1774 0.079 2.232 0.026 0.022 0.333
C(IJ_CODE, Treatment('ROS'))[T.RPO] 0.5434 0.088 6.149 0.000 0.370 0.717
C(IJ_CODE, Treatment('ROS'))[T.RR1] 0.4289 0.116 3.693 0.000 0.201 0.656
C(IJ_CODE, Treatment('ROS'))[T.RRO] -1.0971 0.096 -11.441 0.000 -1.285 -0.909
C(IJ_CODE, Treatment('ROS'))[T.RS] 0.0454 0.066 0.686 0.493 -0.084 0.175
C(IJ_CODE, Treatment('ROS'))[T.RSH] -0.8227 0.122 -6.738 0.000 -1.062 -0.583
C(IJ_CODE, Treatment('ROS'))[T.RW] -0.4384 0.070 -6.277 0.000 -0.575 -0.302
C(IJ_CODE, Treatment('ROS'))[T.RWK] -1.4959 0.114 -13.113 0.000 -1.719 -1.272
C(IJ_CODE, Treatment('ROS'))[T.RY] -0.1920 0.080 -2.398 0.016 -0.349 -0.035
C(IJ_CODE, Treatment('ROS'))[T.RZA] 1.1597 0.085 13.627 0.000 0.993 1.326
C(IJ_CODE, Treatment('ROS'))[T.SAM] 0.3465 0.107 3.238 0.001 0.137 0.556
C(IJ_CODE, Treatment('ROS'))[T.SAS] -1.8626 0.211 -8.847 0.000 -2.275 -1.450
C(IJ_CODE, Treatment('ROS'))[T.SD] -1.0931 0.138 -7.934 0.000 -1.363 -0.823
C(IJ_CODE, Treatment('ROS'))[T.SEC] -0.2917 0.102 -2.852 0.004 -0.492 -0.091
C(IJ_CODE, Treatment('ROS'))[T.SEM] 0.1552 0.063 2.467 0.014 0.032 0.279
C(IJ_CODE, Treatment('ROS'))[T.SFK] -0.3700 0.073 -5.095 0.000 -0.512 -0.228
C(IJ_CODE, Treatment('ROS'))[T.SG1] -1.6933 0.147 -11.549 0.000 -1.981 -1.406
C(IJ_CODE, Treatment('ROS'))[T.SGA] -0.1510 0.063 -2.387 0.017 -0.275 -0.027
C(IJ_CODE, Treatment('ROS'))[T.SH] -0.7943 0.062 -12.812 0.000 -0.916 -0.673
C(IJ_CODE, Treatment('ROS'))[T.SHK] 0.4113 0.161 2.560 0.010 0.096 0.726
C(IJ_CODE, Treatment('ROS'))[T.SLS] 0.7100 0.081 8.734 0.000 0.551 0.869
C(IJ_CODE, Treatment('ROS'))[T.SMB] 0.7851 0.068 11.614 0.000 0.653 0.918
C(IJ_CODE, Treatment('ROS'))[T.SRA] -0.0618 0.080 -0.774 0.439 -0.218 0.095
C(IJ_CODE, Treatment('ROS'))[T.SRK] 0.0275 0.077 0.356 0.722 -0.124 0.179
C(IJ_CODE, Treatment('ROS'))[T.SSC] 1.6033 0.077 20.880 0.000 1.453 1.754
C(IJ_CODE, Treatment('ROS'))[T.SSG] 0.5414 0.094 5.775 0.000 0.358 0.725
C(IJ_CODE, Treatment('ROS'))[T.TAB] 2.1420 0.069 30.833 0.000 2.006 2.278
C(IJ_CODE, Treatment('ROS'))[T.TC] -0.4707 0.065 -7.250 0.000 -0.598 -0.343
C(IJ_CODE, Treatment('ROS'))[T.TCR] -2.5239 0.165 -15.304 0.000 -2.847 -2.201
C(IJ_CODE, Treatment('ROS'))[T.TF] -0.4885 0.084 -5.823 0.000 -0.653 -0.324
C(IJ_CODE, Treatment('ROS'))[T.TGC] -1.2230 0.156 -7.842 0.000 -1.529 -0.917
C(IJ_CODE, Treatment('ROS'))[T.TGS] 0.1269 0.092 1.379 0.168 -0.053 0.307
C(IJ_CODE, Treatment('ROS'))[T.THS] 0.9092 0.067 13.497 0.000 0.777 1.041
C(IJ_CODE, Treatment('ROS'))[T.TJM] 1.1969 0.077 15.566 0.000 1.046 1.348
C(IJ_CODE, Treatment('ROS'))[T.TMO] -0.4276 0.142 -3.009 0.003 -0.706 -0.149
C(IJ_CODE, Treatment('ROS'))[T.TMR] -0.2391 0.127 -1.888 0.059 -0.487 0.009
C(IJ_CODE, Treatment('ROS'))[T.TNN] -1.6895 0.091 -18.538 0.000 -1.868 -1.511
C(IJ_CODE, Treatment('ROS'))[T.TPQ] 0.1780 0.081 2.209 0.027 0.020 0.336
C(IJ_CODE, Treatment('ROS'))[T.TWJ] -0.2940 0.145 -2.026 0.043 -0.578 -0.010
C(IJ_CODE, Treatment('ROS'))[T.VAW] 0.9784 0.082 11.875 0.000 0.817 1.140
C(IJ_CODE, Treatment('ROS'))[T.VBM] -2.4115 0.182 -13.231 0.000 -2.769 -2.054
C(IJ_CODE, Treatment('ROS'))[T.VG] 0.1725 0.042 4.100 0.000 0.090 0.255
C(IJ_CODE, Treatment('ROS'))[T.VGU] 1.2307 0.065 18.847 0.000 1.103 1.359
C(IJ_CODE, Treatment('ROS'))[T.VPG] -0.1052 0.112 -0.941 0.347 -0.324 0.114
C(IJ_CODE, Treatment('ROS'))[T.VSC] -0.3907 0.120 -3.268 0.001 -0.625 -0.156
C(IJ_CODE, Treatment('ROS'))[T.WAC] -1.0164 0.121 -8.383 0.000 -1.254 -0.779
C(IJ_CODE, Treatment('ROS'))[T.WAH] -0.1025 0.106 -0.964 0.335 -0.311 0.106
C(IJ_CODE, Treatment('ROS'))[T.WCP] -0.7176 0.084 -8.555 0.000 -0.882 -0.553
C(IJ_CODE, Treatment('ROS'))[T.WDN] 0.6675 0.104 6.410 0.000 0.463 0.872
C(IJ_CODE, Treatment('ROS'))[T.WFJ] -2.1972 0.116 -19.015 0.000 -2.424 -1.971
C(IJ_CODE, Treatment('ROS'))[T.WI] 0.0135 0.081 0.166 0.868 -0.146 0.173
C(IJ_CODE, Treatment('ROS'))[T.WJM] 0.2426 0.084 2.882 0.004 0.078 0.408
C(IJ_CODE, Treatment('ROS'))[T.WKH] 0.1092 0.104 1.054 0.292 -0.094 0.312
C(IJ_CODE, Treatment('ROS'))[T.WKZ] -0.1421 0.085 -1.669 0.095 -0.309 0.025
C(IJ_CODE, Treatment('ROS'))[T.WLN] 0.8273 0.146 5.668 0.000 0.541 1.113
C(IJ_CODE, Treatment('ROS'))[T.WPJ] 0.8988 0.131 6.835 0.000 0.641 1.157
C(IJ_CODE, Treatment('ROS'))[T.WS] 1.2894 0.126 10.198 0.000 1.042 1.537
C(IJ_CODE, Treatment('ROS'))[T.WVW] 1.7489 0.074 23.523 0.000 1.603 1.895
C(IJ_CODE, Treatment('ROS'))[T.WWS] 1.1430 0.142 8.050 0.000 0.865 1.421
C(IJ_CODE, Treatment('ROS'))[T.YKA] -1.2535 0.124 -10.136 0.000 -1.496 -1.011
C(IJ_CODE, Treatment('ROS'))[T.ZZD] 0.3458 0.138 2.513 0.012 0.076 0.615
C(UPDATE_SITE, Treatment('NYC'))[T.ADL] -1.0468 0.291 -3.602 0.000 -1.616 -0.477
C(UPDATE_SITE, Treatment('NYC'))[T.AGA] -0.2214 0.157 -1.407 0.159 -0.530 0.087
C(UPDATE_SITE, Treatment('NYC'))[T.ATL] -0.9078 0.078 -11.655 0.000 -1.060 -0.755
C(UPDATE_SITE, Treatment('NYC'))[T.BAL] 0.2433 0.059 4.128 0.000 0.128 0.359
C(UPDATE_SITE, Treatment('NYC'))[T.BLM] -0.1341 0.105 -1.277 0.202 -0.340 0.072
C(UPDATE_SITE, Treatment('NYC'))[T.BOS] 0.4463 0.063 7.073 0.000 0.323 0.570
C(UPDATE_SITE, Treatment('NYC'))[T.BUF] 0.2415 0.185 1.308 0.191 -0.120 0.603
C(UPDATE_SITE, Treatment('NYC'))[T.CHI] 0.4426 0.071 6.225 0.000 0.303 0.582
C(UPDATE_SITE, Treatment('NYC'))[T.CHL] -1.5921 0.069 -23.165 0.000 -1.727 -1.457
C(UPDATE_SITE, Treatment('NYC'))[T.CLE] -0.0021 0.075 -0.029 0.977 -0.148 0.144
C(UPDATE_SITE, Treatment('NYC'))[T.DAL] -0.0598 0.073 -0.821 0.411 -0.202 0.083
C(UPDATE_SITE, Treatment('NYC'))[T.DEN] 0.4388 0.093 4.735 0.000 0.257 0.620
C(UPDATE_SITE, Treatment('NYC'))[T.DET] -0.3473 0.062 -5.594 0.000 -0.469 -0.226
C(UPDATE_SITE, Treatment('NYC'))[T.ELO] -2.1629 0.330 -6.562 0.000 -2.809 -1.517
C(UPDATE_SITE, Treatment('NYC'))[T.ELP] 0.9626 0.094 10.284 0.000 0.779 1.146
C(UPDATE_SITE, Treatment('NYC'))[T.ELZ] -0.2739 0.112 -2.435 0.015 -0.494 -0.053
C(UPDATE_SITE, Treatment('NYC'))[T.HAR] -0.0098 0.082 -0.119 0.905 -0.170 0.150
C(UPDATE_SITE, Treatment('NYC'))[T.HLG] -0.4010 0.082 -4.915 0.000 -0.561 -0.241
C(UPDATE_SITE, Treatment('NYC'))[T.HON] 0.1479 0.164 0.901 0.368 -0.174 0.470
C(UPDATE_SITE, Treatment('NYC'))[T.HOU] 0.2543 0.057 4.458 0.000 0.143 0.366
C(UPDATE_SITE, Treatment('NYC'))[T.IMP] 0.0626 0.151 0.415 0.678 -0.233 0.358
C(UPDATE_SITE, Treatment('NYC'))[T.KAN] -0.6447 0.076 -8.527 0.000 -0.793 -0.497
C(UPDATE_SITE, Treatment('NYC'))[T.KRO] -1.2726 0.186 -6.856 0.000 -1.636 -0.909
C(UPDATE_SITE, Treatment('NYC'))[T.LAD] -1.6707 0.310 -5.390 0.000 -2.278 -1.063
C(UPDATE_SITE, Treatment('NYC'))[T.LOS] 0.5924 0.037 15.825 0.000 0.519 0.666
C(UPDATE_SITE, Treatment('NYC'))[T.LOU] 0.3443 0.080 4.290 0.000 0.187 0.502
C(UPDATE_SITE, Treatment('NYC'))[T.LVG] -0.0088 0.089 -0.100 0.921 -0.183 0.165
C(UPDATE_SITE, Treatment('NYC'))[T.MEM] -0.2239 0.063 -3.567 0.000 -0.347 -0.101
C(UPDATE_SITE, Treatment('NYC'))[T.MIA] -0.2537 0.033 -7.680 0.000 -0.318 -0.189
C(UPDATE_SITE, Treatment('NYC'))[T.NEW] -0.5880 0.062 -9.554 0.000 -0.709 -0.467
C(UPDATE_SITE, Treatment('NYC'))[T.NOL] -0.5615 0.115 -4.867 0.000 -0.788 -0.335
C(UPDATE_SITE, Treatment('NYC'))[T.NYV] -1.0289 0.078 -13.113 0.000 -1.183 -0.875
C(UPDATE_SITE, Treatment('NYC'))[T.OMA] -0.0678 0.091 -0.744 0.457 -0.246 0.111
C(UPDATE_SITE, Treatment('NYC'))[T.ORL] -0.0341 0.044 -0.767 0.443 -0.121 0.053
C(UPDATE_SITE, Treatment('NYC'))[T.PHI] 0.2468 0.054 4.611 0.000 0.142 0.352
C(UPDATE_SITE, Treatment('NYC'))[T.PHO] 0.6658 0.098 6.784 0.000 0.473 0.858
C(UPDATE_SITE, Treatment('NYC'))[T.PIS] -1.6370 0.254 -6.450 0.000 -2.134 -1.140
C(UPDATE_SITE, Treatment('NYC'))[T.POO] 0.0435 0.087 0.503 0.615 -0.126 0.213
C(UPDATE_SITE, Treatment('NYC'))[T.SAJ] -0.0247 0.196 -0.126 0.900 -0.410 0.360
C(UPDATE_SITE, Treatment('NYC'))[T.SEA] 0.0190 0.045 0.422 0.673 -0.069 0.107
C(UPDATE_SITE, Treatment('NYC'))[T.SFR] 0.6663 0.046 14.518 0.000 0.576 0.756
C(UPDATE_SITE, Treatment('NYC'))[T.SLC] -0.6922 0.109 -6.363 0.000 -0.905 -0.479
C(UPDATE_SITE, Treatment('NYC'))[T.SNA] 0.1758 0.076 2.315 0.021 0.027 0.325
C(UPDATE_SITE, Treatment('NYC'))[T.SND] -0.5912 0.058 -10.199 0.000 -0.705 -0.478
C(UPDATE_SITE, Treatment('NYC'))[T.TUC] -0.3801 0.124 -3.057 0.002 -0.624 -0.136
C(UPDATE_SITE, Treatment('NYC'))[T.WAS] 0.5069 0.054 9.365 0.000 0.401 0.613
C(UPDATE_SITE, Treatment('NYC'))[T.YOR] -0.6704 0.227 -2.955 0.003 -1.115 -0.226
C(NAT, Treatment('CH'))[T.AF] 1.0316 0.096 10.764 0.000 0.844 1.219
C(NAT, Treatment('CH'))[T.AG] 0.1491 0.097 1.540 0.123 -0.041 0.339
C(NAT, Treatment('CH'))[T.AL] 0.1476 0.025 5.795 0.000 0.098 0.197
C(NAT, Treatment('CH'))[T.AM] 0.0726 0.032 2.253 0.024 0.009 0.136
C(NAT, Treatment('CH'))[T.AR] -0.5504 0.069 -7.925 0.000 -0.687 -0.414
C(NAT, Treatment('CH'))[T.AZ] 0.7437 0.108 6.886 0.000 0.532 0.955
C(NAT, Treatment('CH'))[T.BG] 0.2410 0.038 6.395 0.000 0.167 0.315
C(NAT, Treatment('CH'))[T.BH] 0.3509 0.129 2.725 0.006 0.099 0.603
C(NAT, Treatment('CH'))[T.BL] 0.6715 0.095 7.036 0.000 0.484 0.859
C(NAT, Treatment('CH'))[T.BM] 1.0498 0.062 16.886 0.000 0.928 1.172
C(NAT, Treatment('CH'))[T.BO] 0.8004 0.086 9.309 0.000 0.632 0.969
C(NAT, Treatment('CH'))[T.BR] -0.3096 0.037 -8.392 0.000 -0.382 -0.237
C(NAT, Treatment('CH'))[T.BS] 1.2130 0.103 11.808 0.000 1.012 1.414
C(NAT, Treatment('CH'))[T.BU] 0.4948 0.068 7.304 0.000 0.362 0.628
C(NAT, Treatment('CH'))[T.BY] 0.2675 0.119 2.245 0.025 0.034 0.501
C(NAT, Treatment('CH'))[T.BZ] 1.2974 0.102 12.678 0.000 1.097 1.498
C(NAT, Treatment('CH'))[T.CA] 0.3051 0.068 4.474 0.000 0.171 0.439
C(NAT, Treatment('CH'))[T.CB] -0.0054 0.112 -0.048 0.962 -0.226 0.215
C(NAT, Treatment('CH'))[T.CC] 0.0679 0.095 0.712 0.477 -0.119 0.255
C(NAT, Treatment('CH'))[T.CD] 1.0979 0.152 7.225 0.000 0.800 1.396
C(NAT, Treatment('CH'))[T.CE] 0.1925 0.054 3.590 0.000 0.087 0.298
C(NAT, Treatment('CH'))[T.CF] 0.6739 0.060 11.200 0.000 0.556 0.792
C(NAT, Treatment('CH'))[T.CG] 0.5698 0.129 4.425 0.000 0.317 0.822
C(NAT, Treatment('CH'))[T.CI] 0.4459 0.103 4.330 0.000 0.244 0.648
C(NAT, Treatment('CH'))[T.CM] 0.7681 0.039 19.785 0.000 0.692 0.844
C(NAT, Treatment('CH'))[T.CO] -0.0404 0.018 -2.183 0.029 -0.077 -0.004
C(NAT, Treatment('CH'))[T.CS] 0.5064 0.108 4.701 0.000 0.295 0.718
C(NAT, Treatment('CH'))[T.CU] 2.5355 0.028 89.205 0.000 2.480 2.591
C(NAT, Treatment('CH'))[T.CV] 0.3052 0.144 2.117 0.034 0.023 0.588
C(NAT, Treatment('CH'))[T.DC] 0.2477 0.098 2.524 0.012 0.055 0.440
C(NAT, Treatment('CH'))[T.DR] 0.0457 0.037 1.230 0.219 -0.027 0.118
C(NAT, Treatment('CH'))[T.EC] -0.6150 0.039 -15.746 0.000 -0.691 -0.538
C(NAT, Treatment('CH'))[T.EG] 1.3147 0.037 36.008 0.000 1.243 1.386
C(NAT, Treatment('CH'))[T.ER] 1.2148 0.058 21.106 0.000 1.102 1.328
C(NAT, Treatment('CH'))[T.ES] -0.7149 0.015 -47.263 0.000 -0.745 -0.685
C(NAT, Treatment('CH'))[T.ET] 0.9576 0.033 28.605 0.000 0.892 1.023
C(NAT, Treatment('CH'))[T.FJ] 0.4588 0.071 6.490 0.000 0.320 0.597
C(NAT, Treatment('CH'))[T.GA] 0.8409 0.061 13.854 0.000 0.722 0.960
C(NAT, Treatment('CH'))[T.GE] 0.1469 0.110 1.341 0.180 -0.068 0.361
C(NAT, Treatment('CH'))[T.GH] 0.1828 0.056 3.260 0.001 0.073 0.293
C(NAT, Treatment('CH'))[T.GO] 0.3617 0.067 5.366 0.000 0.230 0.494
C(NAT, Treatment('CH'))[T.GT] -0.5818 0.016 -36.319 0.000 -0.613 -0.550
C(NAT, Treatment('CH'))[T.GV] 0.3212 0.039 8.260 0.000 0.245 0.397
C(NAT, Treatment('CH'))[T.GY] -0.1489 0.068 -2.176 0.030 -0.283 -0.015
C(NAT, Treatment('CH'))[T.HA] -0.9526 0.020 -47.677 0.000 -0.992 -0.913
C(NAT, Treatment('CH'))[T.HO] -1.1712 0.020 -59.329 0.000 -1.210 -1.133
C(NAT, Treatment('CH'))[T.ID] -0.7054 0.025 -28.382 0.000 -0.754 -0.657
C(NAT, Treatment('CH'))[T.IN] -0.2308 0.020 -11.319 0.000 -0.271 -0.191
C(NAT, Treatment('CH'))[T.IR] 0.6773 0.039 17.240 0.000 0.600 0.754
C(NAT, Treatment('CH'))[T.IS] 0.3764 0.068 5.530 0.000 0.243 0.510
C(NAT, Treatment('CH'))[T.IV] 0.4885 0.060 8.161 0.000 0.371 0.606
C(NAT, Treatment('CH'))[T.IZ] 1.3838 0.044 31.251 0.000 1.297 1.471
C(NAT, Treatment('CH'))[T.JM] 0.4448 0.040 11.051 0.000 0.366 0.524
C(NAT, Treatment('CH'))[T.JO] 0.5991 0.057 10.496 0.000 0.487 0.711
C(NAT, Treatment('CH'))[T.KE] 0.3036 0.047 6.526 0.000 0.212 0.395
C(NAT, Treatment('CH'))[T.KG] 1.0636 0.112 9.463 0.000 0.843 1.284
C(NAT, Treatment('CH'))[T.KS] 0.0430 0.058 0.738 0.460 -0.071 0.157
C(NAT, Treatment('CH'))[T.KV] 0.5753 0.133 4.322 0.000 0.314 0.836
C(NAT, Treatment('CH'))[T.KZ] 0.9906 0.094 10.497 0.000 0.806 1.176
C(NAT, Treatment('CH'))[T.LA] 0.2637 0.085 3.109 0.002 0.097 0.430
C(NAT, Treatment('CH'))[T.LE] 0.6112 0.054 11.366 0.000 0.506 0.717
C(NAT, Treatment('CH'))[T.LH] 0.0126 0.121 0.104 0.917 -0.224 0.249
C(NAT, Treatment('CH'))[T.LI] 0.8832 0.058 15.181 0.000 0.769 0.997
C(NAT, Treatment('CH'))[T.MD] 0.7486 0.080 9.383 0.000 0.592 0.905
C(NAT, Treatment('CH'))[T.MG] -0.3401 0.075 -4.517 0.000 -0.488 -0.193
C(NAT, Treatment('CH'))[T.ML] 0.3333 0.061 5.457 0.000 0.214 0.453
C(NAT, Treatment('CH'))[T.MM] -0.1192 0.092 -1.301 0.193 -0.299 0.060
C(NAT, Treatment('CH'))[T.MO] 0.9727 0.076 12.750 0.000 0.823 1.122
C(NAT, Treatment('CH'))[T.MR] -0.3944 0.040 -9.925 0.000 -0.472 -0.317
C(NAT, Treatment('CH'))[T.MX] 0.2113 0.014 15.116 0.000 0.184 0.239
C(NAT, Treatment('CH'))[T.MY] 0.0890 0.126 0.705 0.481 -0.158 0.336
C(NAT, Treatment('CH'))[T.NG] 0.2499 0.116 2.145 0.032 0.022 0.478
C(NAT, Treatment('CH'))[T.NI] 0.4264 0.038 11.095 0.000 0.351 0.502
C(NAT, Treatment('CH'))[T.NP] 0.3974 0.035 11.272 0.000 0.328 0.466
C(NAT, Treatment('CH'))[T.NU] 0.2070 0.039 5.244 0.000 0.130 0.284
C(NAT, Treatment('CH'))[T.PE] 0.4044 0.032 12.568 0.000 0.341 0.467
C(NAT, Treatment('CH'))[T.PK] 0.3409 0.029 11.892 0.000 0.285 0.397
C(NAT, Treatment('CH'))[T.PL] 0.1484 0.063 2.350 0.019 0.025 0.272
C(NAT, Treatment('CH'))[T.PM] 0.6881 0.115 6.001 0.000 0.463 0.913
C(NAT, Treatment('CH'))[T.PO] 0.2523 0.117 2.160 0.031 0.023 0.481
C(NAT, Treatment('CH'))[T.RO] 0.4729 0.056 8.515 0.000 0.364 0.582
C(NAT, Treatment('CH'))[T.RP] 0.5824 0.038 15.128 0.000 0.507 0.658
C(NAT, Treatment('CH'))[T.RU] 0.9020 0.032 28.403 0.000 0.840 0.964
C(NAT, Treatment('CH'))[T.RW] 1.0881 0.106 10.277 0.000 0.881 1.296
C(NAT, Treatment('CH'))[T.SF] 0.6042 0.103 5.851 0.000 0.402 0.807
C(NAT, Treatment('CH'))[T.SG] 0.2713 0.070 3.866 0.000 0.134 0.409
C(NAT, Treatment('CH'))[T.SL] 0.0872 0.058 1.501 0.133 -0.027 0.201
C(NAT, Treatment('CH'))[T.SO] 0.4761 0.042 11.273 0.000 0.393 0.559
C(NAT, Treatment('CH'))[T.SS] 0.5528 0.085 6.509 0.000 0.386 0.719
C(NAT, Treatment('CH'))[T.SU] 0.7053 0.070 10.132 0.000 0.569 0.842
C(NAT, Treatment('CH'))[T.SY] 1.0766 0.070 15.472 0.000 0.940 1.213
C(NAT, Treatment('CH'))[T.TD] 0.7608 0.065 11.762 0.000 0.634 0.888
C(NAT, Treatment('CH'))[T.TH] 0.5876 0.111 5.276 0.000 0.369 0.806
C(NAT, Treatment('CH'))[T.TO] 0.5789 0.066 8.732 0.000 0.449 0.709
C(NAT, Treatment('CH'))[T.TS] 0.9330 0.126 7.379 0.000 0.685 1.181
C(NAT, Treatment('CH'))[T.TU] 0.2206 0.072 3.065 0.002 0.080 0.362
C(NAT, Treatment('CH'))[T.TZ] 0.3182 0.119 2.672 0.008 0.085 0.552
C(NAT, Treatment('CH'))[T.UE] 0.5898 0.048 12.393 0.000 0.497 0.683
C(NAT, Treatment('CH'))[T.UG] 0.6678 0.075 8.910 0.000 0.521 0.815
C(NAT, Treatment('CH'))[T.UK] 0.3685 0.085 4.347 0.000 0.202 0.535
C(NAT, Treatment('CH'))[T.UR] 1.0622 0.053 19.914 0.000 0.958 1.167
C(NAT, Treatment('CH'))[T.UZ] 0.7145 0.058 12.267 0.000 0.600 0.829
C(NAT, Treatment('CH'))[T.VE] 0.3001 0.029 10.508 0.000 0.244 0.356
C(NAT, Treatment('CH'))[T.VM] 0.1128 0.061 1.864 0.062 -0.006 0.231
C(NAT, Treatment('CH'))[T.YE] 0.1601 0.077 2.078 0.038 0.009 0.311
C(NAT, Treatment('CH'))[T.YO] 0.2948 0.043 6.880 0.000 0.211 0.379
C(NAT, Treatment('CH'))[T.YS] 0.4937 0.107 4.627 0.000 0.285 0.703
C(NAT, Treatment('CH'))[T.ZI] 0.6402 0.064 9.933 0.000 0.514 0.767

Reading our results#

That's a lot lot lot lot of variables! Let's convert them to an easier-to-read dataframe with odds ratios and p values.

import numpy as np

coefs = pd.DataFrame({
    'coef': result.params.values,
    'odds ratio': np.exp(result.params.values),
    'pvalue': result.pvalues,
    'column': result.params.index
}).sort_values(by='odds ratio', ascending=False)
coefs.head(10)
coef odds ratio pvalue column
C(NAT, Treatment('CH'))[T.CU] 2.53550 12.62271 0.00000 C(NAT, Treatment('CH'))[T.CU]
C(IJ_CODE, Treatment('ROS'))[T.TAB] 2.14198 8.51629 0.00000 C(IJ_CODE, Treatment('ROS'))[T.TAB]
C(IJ_CODE, Treatment('ROS'))[T.PMM] 2.12543 8.37650 0.00000 C(IJ_CODE, Treatment('ROS'))[T.PMM]
C(IJ_CODE, Treatment('ROS'))[T.ICD] 2.08921 8.07852 0.00000 C(IJ_CODE, Treatment('ROS'))[T.ICD]
C(IJ_CODE, Treatment('ROS'))[T.NB] 1.78492 5.95909 0.00000 C(IJ_CODE, Treatment('ROS'))[T.NB]
C(IJ_CODE, Treatment('ROS'))[T.WVW] 1.74893 5.74842 0.00000 C(IJ_CODE, Treatment('ROS'))[T.WVW]
C(IJ_CODE, Treatment('ROS'))[T.OLC] 1.66564 5.28903 0.00000 C(IJ_CODE, Treatment('ROS'))[T.OLC]
C(IJ_CODE, Treatment('ROS'))[T.SSC] 1.60332 4.96951 0.00000 C(IJ_CODE, Treatment('ROS'))[T.SSC]
C(IJ_CODE, Treatment('ROS'))[T.PD1] 1.56840 4.79897 0.00000 C(IJ_CODE, Treatment('ROS'))[T.PD1]
C(IJ_CODE, Treatment('ROS'))[T.RAC] 1.53783 4.65448 0.00000 C(IJ_CODE, Treatment('ROS'))[T.RAC]

While we could pick through that list to get the top n, let's look at each slice separately. Judges, sites, and nationalities, one at a time.

Judges#

# Highest odds ratio
coefs[coefs.column.str.contains("IJ_CODE")].sort_values(by='odds ratio', ascending=False).head()
coef odds ratio pvalue column
C(IJ_CODE, Treatment('ROS'))[T.TAB] 2.14198 8.51629 0.00000 C(IJ_CODE, Treatment('ROS'))[T.TAB]
C(IJ_CODE, Treatment('ROS'))[T.PMM] 2.12543 8.37650 0.00000 C(IJ_CODE, Treatment('ROS'))[T.PMM]
C(IJ_CODE, Treatment('ROS'))[T.ICD] 2.08921 8.07852 0.00000 C(IJ_CODE, Treatment('ROS'))[T.ICD]
C(IJ_CODE, Treatment('ROS'))[T.NB] 1.78492 5.95909 0.00000 C(IJ_CODE, Treatment('ROS'))[T.NB]
C(IJ_CODE, Treatment('ROS'))[T.WVW] 1.74893 5.74842 0.00000 C(IJ_CODE, Treatment('ROS'))[T.WVW] 
# Lowest odds ratio
coefs[coefs.column.str.contains("IJ_CODE")].sort_values(by='odds ratio', ascending=True).head()
coef odds ratio pvalue column
C(IJ_CODE, Treatment('ROS'))[T.MS1] -2.65239 0.07048 0.00000 C(IJ_CODE, Treatment('ROS'))[T.MS1]
C(IJ_CODE, Treatment('ROS'))[T.HAR] -2.64110 0.07128 0.00000 C(IJ_CODE, Treatment('ROS'))[T.HAR]
C(IJ_CODE, Treatment('ROS'))[T.TCR] -2.52388 0.08015 0.00000 C(IJ_CODE, Treatment('ROS'))[T.TCR]
C(IJ_CODE, Treatment('ROS'))[T.RMH] -2.45111 0.08620 0.00000 C(IJ_CODE, Treatment('ROS'))[T.RMH]
C(IJ_CODE, Treatment('ROS'))[T.VBM] -2.41153 0.08968 0.00000 C(IJ_CODE, Treatment('ROS'))[T.VBM]

Those odds ratios let us how each other judge compares to Raphael Ortiz-Segura, controlling for city and nationality.

Cities/Sites#

Same with the cities - they're each compared to New York.

# Highest odds ratio
coefs[coefs.column.str.contains("SITE")].sort_values(by='odds ratio', ascending=False).head()
coef odds ratio pvalue column
C(UPDATE_SITE, Treatment('NYC'))[T.ELP] 0.96261 2.61851 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.ELP]
C(UPDATE_SITE, Treatment('NYC'))[T.SFR] 0.66628 1.94698 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.SFR]
C(UPDATE_SITE, Treatment('NYC'))[T.PHO] 0.66577 1.94599 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.PHO]
C(UPDATE_SITE, Treatment('NYC'))[T.LOS] 0.59240 1.80833 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.LOS]
C(UPDATE_SITE, Treatment('NYC'))[T.WAS] 0.50693 1.66018 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.WAS] 
# Highest odds ratio
coefs[coefs.column.str.contains("SITE")].sort_values(by='odds ratio', ascending=True).head()
coef odds ratio pvalue column
C(UPDATE_SITE, Treatment('NYC'))[T.ELO] -2.16287 0.11499 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.ELO]
C(UPDATE_SITE, Treatment('NYC'))[T.LAD] -1.67067 0.18812 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.LAD]
C(UPDATE_SITE, Treatment('NYC'))[T.PIS] -1.63704 0.19456 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.PIS]
C(UPDATE_SITE, Treatment('NYC'))[T.CHL] -1.59208 0.20350 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.CHL]
C(UPDATE_SITE, Treatment('NYC'))[T.KRO] -1.27262 0.28010 0.00000 C(UPDATE_SITE, Treatment('NYC'))[T.KRO]

Nationalities#

We'll need to pull in the list of nationalities so we know what we're looking at.

from io import StringIO

content = open("data/tblLookupAlienNat.csv").read().replace('"','')

nationalities = pd.read_csv(StringIO(content), sep='\t')
nationalities[['strCode', 'strDescription']].sort_values(by='strCode')
strCode strDescription
0 <A> <All>
1 ?? UNKNOWN NATIONALITY
2 AB ARUBA
3 AC ANTIGUA AND BARBUDA
4 AF AFGHANISTAN
5 AG ALGERIA
6 AL ALBANIA
7 AM ARMENIA
8 AN ANDORRA
9 AO ANGOLA
10 AR ARGENTINA
11 AS AUSTRALIA
12 AU AUSTRIA
13 AV ANGUILLA
14 AZ AZERBAIJAN
15 BA BAHRAIN
16 BB BARBADOS
17 BC BOTSWANA
18 BD BERMUDA
19 BE BELGIUM
20 BF BAHAMAS
21 BG BANGLADESH
22 BH BELIZE
23 BI BOSNIA-HERZEGOVINA
24 BL BOLIVIA
25 BM BURMA (MYANMAR)
26 BN BENIN
27 BO BURKINA FASO
28 BP SOLOMON ISLANDS
29 BR BRAZIL
30 BS BYELORUSSIA (BELARUS)
31 BT BHUTAN
32 BU BULGARIA
33 BV BOUVET ISLAND
34 BW BRITISH WEST INDIES
35 BX BRUNEI
36 BY BURUNDI
37 BZ BELARUS
38 CA CANADA
39 CB KAMPUCHEA
40 CC CAMBODIA
41 CD CHAD
42 CE SRI LANKA
43 CF PEOPLE'S REPUBLIC OF THE CONGO
44 CG ZAIRE
45 CH CHINA
46 CI CHILE
47 CJ CAYMAN ISLANDS
48 CK COCOS ISLAND
49 CM CAMEROON
50 CN COMOROS
51 CO COLOMBIA
52 CR CZECH REPUBLIC
53 CS COSTA RICA
54 CT CENTRAL AFRICAN REPUBLIC
55 CU CUBA
56 CV CAPE VERDE
57 CW COOK ISLANDS
58 CX CROATIA
59 CY CYPRUS
60 CZ CZECHOSLOVAKIA
61 DA DENMARK
62 DC DEMOCRATIC REPUBLIC OF CONGO
63 DJ DJIBOUTI
64 DM PEOPLE'S REPUBLIC OF BENIN
65 DO DOMINICA
66 DR DOMINICAN REPUBLIC
67 EC ECUADOR
68 EG EGYPT
69 EI IRELAND
70 EK EQUATORIAL GUINEA
71 EO ESTONIA
72 ER ERITREA
73 ES EL SALVADOR
74 ET ETHIOPIA
75 FA FALKLAND ISLANDS
76 FG FRENCH GUIANA
77 FI FINLAND
78 FJ FIJI
79 FM FEDERATED STATES OF MICRONESIA
80 FO FAEROE ISLAND
81 FP FRENCH POLYNESIA
82 FR FRANCE
83 FS FRENCH SOUTHERN AND ANTARCTIC LANDS
84 FW FRENCH WEST INDIES
85 GA GAMBIA
86 GB GABON
87 GC EAST GERMANY
88 GE GERMANY
89 GH GHANA
90 GI GIBRALTAR
91 GJ GRENADA
92 GL GREENLAND
93 GO GEORGIA
94 GP GUADELOUPE
95 GR GREECE
96 GT GUATEMALA
97 GV GUINEA
98 GY GUYANA
99 GZ GAZA STRIP
100 HA HAITI
101 HK HONG KONG
102 HL HOLLAND
103 HM HEARD AND MCDONALD ISLANDS
104 HO HONDURAS
105 HU HUNGARY
106 IC ICELAND
107 ID INDONESIA
108 IN INDIA
109 IO BRITISH INDIAN OCEAN TERRITORY
110 IR IRAN
111 IS ISRAEL
112 IT ITALY
113 IU IN ABSENTIA/UNKNOWN
114 IV IVORY COAST (COTE D'IVOIRE)
115 IZ IRAQ
116 JA JAPAN
117 JM JAMAICA
118 JO JORDAN
119 KE KENYA
120 KG KIRGHIZIA (KYRGYZSTAN)
121 KN NORTH KOREA
122 KO KOSRAE, FEDERATED STATES OF MICRONESIA
123 KR KIRIBATI
124 KS SOUTH KOREA
125 KT CHRISTMAS ISLANDS
126 KU KUWAIT
127 KV KOSOVO
128 KZ KAZAKHSTAN
129 LA LAOS
130 LE LEBANON
131 LH LITHUANIA
132 LI LIBERIA
133 LS LIECHTENSTEIN
134 LT LESOTHO
135 LU LUXEMBOURG
136 LV LATVIA
137 LY LIBYA
138 MA MADAGASCAR
139 MB MARTINIQUE
140 MC MACAU
141 MD MOLDAVIA (MOLDOVA)
142 MG MONGOLIA
143 MH MONTSERRAT
144 MI MALAWI
145 MJ Montenegro
146 ML MALI
147 MM MACEDONIA
148 MN MONACO
149 MO MOROCCO
150 MP MAURITIUS
151 MQ MIDWAY ISLANDS
152 MR MAURITANIA
153 MT MALTA
154 MU OMAN
155 MV MALDIVES
156 MX MEXICO
157 MY MALAYSIA
158 MZ MOZAMBIQUE
160 NC NEW CALEDONIA
161 ND Not Designated by IJ
162 NE NIUE
163 NF NORFOLK ISLAND
164 NH VANUATU
165 NI NIGERIA
166 NL NETHERLANDS
167 NN NO NATIONALITY
168 NO NORWAY
169 NP NEPAL
170 NQ TRUST TERRITORY OF THE PACIFIC ISLANDS
171 NR NAURU
172 NS SURINAME
173 NU NICARAGUA
174 NZ NEW ZEALAND
175 PA PARAGUAY
176 PC PITCAIRN ISLANDS
177 PE PERU
178 PI PALAU
179 PK PAKISTAN
180 PL POLAND
181 PM PANAMA
182 PN PALESTINIAN
183 PO PORTUGAL
184 PP PAPUA NEW GUINEA
185 PS THE REPUBLIC OF PALAU
186 PU GUINEA BISSAU
187 QA QATAR
188 RE REUNION
189 RM REPUBLIC OF THE MARSHALL ISLANDS
190 RO ROMANIA
191 RP PHILIPPINES
192 RU RUSSIA
193 RW RWANDA
194 SA SAUDI ARABIA
195 SB ST. PIERRE AND MIQUELON
196 SC ST. KITTS AND NEVIS
197 SE SEYCHELLES
198 SF SOUTH AFRICA
199 SG SENEGAL
200 SH ST. HELENA
201 SK ST. KIITTS, WEST INDIES
202 SL SIERRA LEONE
203 SM SAN MARINO
204 SN SINGAPORE
205 SO SOMALIA
206 SP SPAIN
207 SR SLOVAK REPUBLIC
208 SS STATELESS - ALIEN UNABLE TO NAME A COUNTRY
209 ST ST. LUCIA
210 SU SUDAN
211 SV SLOVENIA
212 SW SWEDEN
213 SY SYRIA
214 SZ SWITZERLAND
215 TA TAJIKISTAN (TADZHIK)
216 TC UNITED ARAB EMIRATES
217 TD TRINIDAD AND TOBAGO
218 TH THAILAND
219 TK TURKS AND CAICOS ISLANDS
220 TL TOKELAU
221 TM EAST TIMOR
222 TN TONGA
223 TO TOGO
224 TP SAO TOME AND PRINCIPE
225 TR TURKMENISTAN
226 TS TUNISIA
227 TU TURKEY
228 TV TUVALU
229 TW TAIWAN
230 TZ TANZANIA
231 UE UKRAINE
232 UG UGANDA
233 UK UNITED KINGDOM
234 UR SOVIET UNION
235 US UNITED STATES OF AMERICA
236 UV UPPER VOLTA
237 UY URUGUAY
238 UZ UZBEKISTAN
239 VC ST. VINCENT AND THE GRENADINES
240 VE VENEZUELA
241 VI BRITISH VIRGIN ISLANDS
242 VM VIETNAM
243 WA NAMIBIA
244 WI WESTERN SAHARA
245 WS WESTERN SAMOA
246 WZ SWAZILAND
247 XS SOUTH SUDAN
248 XX BE REMOVED FROM UNITED STATES
249 YE YEMEN
250 YO YUGOSLAVIA
251 YS SERBIA MONTENEGRO
252 ZA ZAMBIA
253 ZI ZIMBABWE
159 NaN NETHERLANDS ANTILLES 
# Highest odds ratio
coefs[coefs.column.str.contains("NAT")].sort_values(by='odds ratio', ascending=False).head()
coef odds ratio pvalue column
C(NAT, Treatment('CH'))[T.CU] 2.53550 12.62271 0.00000 C(NAT, Treatment('CH'))[T.CU]
C(NAT, Treatment('CH'))[T.IZ] 1.38377 3.98990 0.00000 C(NAT, Treatment('CH'))[T.IZ]
C(NAT, Treatment('CH'))[T.EG] 1.31471 3.72368 0.00000 C(NAT, Treatment('CH'))[T.EG]
C(NAT, Treatment('CH'))[T.BZ] 1.29744 3.65991 0.00000 C(NAT, Treatment('CH'))[T.BZ]
C(NAT, Treatment('CH'))[T.ER] 1.21485 3.36977 0.00000 C(NAT, Treatment('CH'))[T.ER]

Cuba, Iraq, Egypt, and Belarus are the top five (BZ and BS are both Belarus).

# Lowest odds ratio
coefs[coefs.column.str.contains("NAT")].sort_values(by='odds ratio', ascending=True).head()
coef odds ratio pvalue column
C(NAT, Treatment('CH'))[T.HO] -1.17124 0.30998 0.00000 C(NAT, Treatment('CH'))[T.HO]
C(NAT, Treatment('CH'))[T.HA] -0.95258 0.38575 0.00000 C(NAT, Treatment('CH'))[T.HA]
C(NAT, Treatment('CH'))[T.ES] -0.71489 0.48925 0.00000 C(NAT, Treatment('CH'))[T.ES]
C(NAT, Treatment('CH'))[T.ID] -0.70545 0.49389 0.00000 C(NAT, Treatment('CH'))[T.ID]
C(NAT, Treatment('CH'))[T.EC] -0.61495 0.54067 0.00000 C(NAT, Treatment('CH'))[T.EC]

Honduras, Haiti, El Salvador, Indonesia, and Ecuador are the bottom five, all of which approximately half the chance or less than an asylum-seeker from China.

Trying again#

The thing is, though, these values probably depend on how we filter. Let's say we filter and perform our regression again, this time the only change being we look for nationalities and judges that show up 100 times instead of 300.

common_nats_100 = list(merged.NAT.value_counts()[merged.NAT.value_counts() >= 100].index)
has_common_nat_100 = merged.NAT.isin(common_nats_100)

common_judges_100 = list(merged.IJ_CODE.value_counts()[merged.IJ_CODE.value_counts() >= 100].index)
has_common_judge_100 = merged.IJ_CODE.isin(common_judges_100)

# We'll use the same two filters for judges and sites as before
# But use our new nationalities list
filtered = merged[has_common_judge_100 & has_common_site & has_common_nat_100]

print("ORIGINAL NATIONALITIES")
print(common_nats)
print("")
print("ADDED NATIONALITIES")
print(list(set(common_nats_100) - set(common_nats)))

ORIGINAL NATIONALITIES
['CH', 'MX', 'ES', 'GT', 'HA', 'HO', 'CO', 'CU', 'IN', 'AL', 'ID', 'PK', 'RU', 'VE', 'ET', 'EG', 'NP', 'AM', 'PE', 'RP', 'DR', 'IR', 'CM', 'BG', 'BR', 'GV', 'NI', 'IZ', 'EC', 'NU', 'JM', 'MR', 'UR', 'SO', 'YO', 'UE', 'ER', 'KE', 'CE', 'LE', 'RO', 'GA', 'BM', 'LI', 'UZ', 'JO', 'GH', 'IV', 'ML', 'CF', 'SY', 'SL', 'KS', 'VM', 'FJ', 'TD', 'ZI', 'PL', 'TO', 'BU', 'SU', 'GO', 'CA', 'MO', 'IS', 'SG', 'UG', 'GY', 'AR', 'TU', 'MD', 'BO', 'YE', 'MG', 'SS', 'AF', 'BZ', 'KZ', 'BS', 'LA', 'UK', 'KG', 'BL', 'RW', 'MM', 'CC', 'AG', 'AZ', 'DC', 'YS', 'SF', 'CI', 'TH', 'CS', 'GE', 'TS', 'PM', 'CB', 'BY', 'PO', 'NG', 'KV', 'TZ', 'MY', 'LH', 'BH', 'CG', 'CV', 'CD']

ADDED NATIONALITIES
['CX', 'BN', 'DO', 'TN', 'BI', 'FR', 'CT', 'HU', 'HK', 'TR', 'ZA', 'GJ', 'BB', 'CR', 'IT', 'EO', 'ST', 'PN', 'GR', 'TW', 'NS', 'AO', 'LV', 'CZ', 'JA', 'SP', 'BF', 'UY', 'LY', 'PA', 'SA', 'KU', 'SR', 'AS', 'TA']

print(len(common_judges_100), "judges, up from", len(common_judges))

487 judges, up from 359

print("Old version", merged[has_common_judge & has_common_site & has_common_nationality].shape)
print("New version", filtered.shape)

Old version (548433, 14)
New version (578438, 14)

Okay, looks like an additional 30,000 cases, only about a 5% increase.

Performing the regression#

%%time

import statsmodels.formula.api as smf

# Create and run our regression
model = smf.logit("""
    granted ~ C(IJ_CODE, Treatment('ROS')) + C(UPDATE_SITE, Treatment('NYC')) + C(NAT, Treatment('CH'))
""", data=filtered)
result = model.fit(method='bfgs', maxiter=1000)
# Not going to print the summary because it's SO LONG
#result.summary()

Optimization terminated successfully.
         Current function value: 0.553239
         Iterations: 482
         Function evaluations: 483
         Gradient evaluations: 483
CPU times: user 16min 55s, sys: 33.7 s, total: 17min 29s
Wall time: 9min 25s

# Build the coefficients dataframe
coefs = pd.DataFrame({
    'coef': result.params.values,
    'odds ratio': np.exp(result.params.values),
    'pvalue': result.pvalues,
    'column': result.params.index
}).sort_values(by='odds ratio', ascending=False)

# Highest odds ratio
coefs[coefs.column.str.contains("NAT")].sort_values(by='odds ratio', ascending=False).head()
coef odds ratio pvalue column
C(NAT, Treatment('CH'))[T.CU] 2.47675 11.90248 0.00000 C(NAT, Treatment('CH'))[T.CU]
C(NAT, Treatment('CH'))[T.IZ] 1.36025 3.89717 0.00000 C(NAT, Treatment('CH'))[T.IZ]
C(NAT, Treatment('CH'))[T.LY] 1.32737 3.77110 0.00000 C(NAT, Treatment('CH'))[T.LY]
C(NAT, Treatment('CH'))[T.EG] 1.32115 3.74772 0.00000 C(NAT, Treatment('CH'))[T.EG]
C(NAT, Treatment('CH'))[T.BZ] 1.25971 3.52441 0.00000 C(NAT, Treatment('CH'))[T.BZ] 
# Lowest odds ratio
coefs[coefs.column.str.contains("NAT")].sort_values(by='odds ratio', ascending=True).head()
coef odds ratio pvalue column
C(NAT, Treatment('CH'))[T.HO] -1.19281 0.30337 0.00000 C(NAT, Treatment('CH'))[T.HO]
C(NAT, Treatment('CH'))[T.HA] -0.98694 0.37271 0.00000 C(NAT, Treatment('CH'))[T.HA]
C(NAT, Treatment('CH'))[T.ES] -0.76023 0.46756 0.00000 C(NAT, Treatment('CH'))[T.ES]
C(NAT, Treatment('CH'))[T.ID] -0.72259 0.48549 0.00000 C(NAT, Treatment('CH'))[T.ID]
C(NAT, Treatment('CH'))[T.EC] -0.65532 0.51928 0.00000 C(NAT, Treatment('CH'))[T.EC]

Discussion topics#

By lowering the threshold from 300 to 100, we gained over 30 "new" countries. Is there a downside to leaving more countries in there? If we think there aren't enough to make a valid conclusion, isn't that what p values are for?

About the site

Hi, I'm Soma, welcome to Data Science for Journalism a.k.a. investigate.ai!

There's been a lot of buzz about machine learning and "artificial intelligence" being used in stories over the past few years. It's mostly not that complicated - a little stats, a classifier here or there - but it's hard to know where to start without a little help.

If you know a little Python programming, hopefully this site can be that help! Learn more about this project here.

Our newsletter

Links

Thanks to Columbia Journalism School, the Knight Foundation, and many others.

investigate.ai
data science for everybody
  1. Types of text analysis
    1. Simple word counting
    2. Counting words across many documents
    3. Segmenting words in East Asian languages
    4. Project: Caixin museums
    5. Using scikit-learn vectorizers with East Asian languages
    1. Upgraded word counts with TF-IDF
    2. Multi-word phrases and n-grams
    3. Standardizing text with stemming and lemmatization
    4. Using TF-IDF with Chinese text
    1. Comparing sentiment analysis tools
    2. Design your own sentiment analyzer
    3. Improving your tool
    4. NRC Emotional Lexicon
    5. Project: UpShot State of the Union
    6. Project: NYT Trump tweets
    1. Converting documents to text (English)
    2. Converting documents to text (non-English)
    1. Extracting topics from documents
    2. Choosing the right number of topics
    3. Topic models with Gensim
    4. Topic models vs clustering
    5. Entity recognition
    6. Intro to word embeddings
    7. Conceptual document similarity
    8. Comparing documents in different languages
  1. Introduction to Classification
    1. Evaluating classifiers
    2. Categorical features
    3. Classifiers with text
    4. Correcting for imbalanced datasets
    1. BuzzFeed: Spy planes
    2. WaPo chat: App reviews
    3. NYT: Faulty airbag search
    4. LA Times: crime classifier
  1. Finding relationships with regression
    1. Linear regression (Quickstart)
    2. Linear regression for humans
    3. Putting regression to use
    4. Evaluating regressions
    5. Associated Press: Life expectancy and unemployment
    1. Logistic regression (Quickstart)
    2. Logistic regression for humans
    3. More complex logistic regressions
    4. Evaluating logistic regressions
    5. Boston Globe: Speeding tickets
    6. APM Reports: Jury selection
  1. Introduction
  2. Vectorizing
  3. Text Analysis
  4. Regression
  5. Classification
  1. Project Summaries
    1. Summary
    2. A simplistic reproduction of the NYT's research using logistic regression
    3. A decision-tree reproduction of the NYT's research
    4. Combining a text vectorizer and a classifier to track down suspicious complaints
    1. Summary
    2. Predicting downgraded assaults with machine learning
    3. Taking a closer look at our classifier and its misclassifications
    4. Trying out and combining different classifiers
    1. Summary
    2. Chinese museum dataset cleanup
    3. Chinese museums per capita analysis
    4. Counting words in Chinese museum names
    1. Summary
    2. Scrape and combine app store reviews
    3. Build a classifier to detect reviews about bad behavior
  6. AJC: Doctors and sex abuse
    1. Summary
    2. An introduction to the NRC Emotional Lexicon
    3. Reproducing The UpShot's Trump State of the Union visualization
    1. Summary
    2. Downloading one million pieces of legislation from LegiScan
    3. Taking a million pieces of legislation from a CSV and inserting them into Postgres
    4. Download Word, PDF and HTML content and process it into text with Tika
    5. Import content into Solr for advanced text searching
    6. Checking for legislative text reuse using Python, Solr, and ngrams
    7. Checking for legislative text reuse using Python, Solr, and simple text search
    8. Search for model legislation in over one million bills using Postgres and Solr
    9. Using topic modeling to categorize legislation
  9. FCC comment bots
    1. Summary
    2. Downloading all 2019 tweets from Democratic presidential candidates
    3. Using topic modeling to analyze presidential candidate tweets
    4. Assigning categories to tweets using keyword matching
    5. Building streamgraphs from categorized and dated datasets
  11. NYT: Trump tweets
    1. Summary
    2. Simple logistic regression using statsmodels (formula version)
    3. Simple logistic regression using statsmodels (dataframes version)
  13. FiveThirtyEight: P-values
    1. Summary
    2. Pothole geographic analysis and linear regression, complete walkthrough
    3. Pothole demographics linear regression, no spatial analysis
    1. Summary
    2. Finding outliers with standard deviation and regression
    3. Finding outliers with regression residuals (short version)
    4. Reproducing the graphics from The Dallas Morning News piece
    1. Summary
    2. Linear regression on Florida schools, complete walkthrough
    3. Linear regression on Florida schools, no cleaning
    1. Summary
    2. Feature selection and engineering
    3. Combine Excel files across multiple sheets and save as CSV files
    4. Create make model weights csv
    5. Find car data from VINs
    6. Combine VINs and weights
    7. Clean combine and filter data
  18. ProPublica: Opportunity Gap
    1. Summary
    2. Logistic regression for speeding tickets
  20. Stanford: Open Policing Data
    1. Summary
    2. Feature engineering - BuzzFeed spy planes
    3. Drawing flight paths on maps with cartopy
    4. Finding surveillance planes using random forests
    1. Summary
    2. Cleaning and combining data for the Reveal Mortgage Analysis
    3. Wild formulas in statsmodels using Patsy (short version)
    4. Reveal Mortgage Analysis - Logistic Regression using statsmodels formulas
    5. Reveal Mortgage Analysis - Logistic Regression
    1. Summary
    2. Combining and cleaning the initial dataset
    3. Picking what matters and what doesn't in a regression
    4. Analyzing data using statsmodels formulas
    5. Alternative techniques with statsmodels formulas
    1. Summary
    2. Preparing the EOIR immigration court data for analysis
    3. How nationality and judges affect your chance of asylum in immigration court
  25. ProPublica: Presidential pardons
    1. Summary
    2. Breaking down machine bias
  27. data.world: The FOIA Predictor