Lending disparities using Logistic Regression#

The story: https://www.revealnews.org/article/for-people-of-color-banks-are-shutting-the-door-to-homeownership/

Author: Aaron Glantz and Emmanuel Martinez

Topics: Logistic regression, odds ratios

Read online Download notebook Interactive version

Our data#

• philadelphia-mortgages.csv: Philadelphia mortgage data for 2015
  • A subset of HMDA LAR data from FFEIC
  • Codebook is 2015HMDACodeSheet.pdf
  • A guide to HMDA reporting
  • I've massaged it slightly to make processing a bit easier
• nhgis0006_ds233_20175_2017_tract.csv:
  • Table B03002: Hispanic or Latino Origin by Race
  • 2013-2017 American Community Survey data US Census Bureau, from NHGIS
  • Codebook is nhgis0006_ds233_20175_2017_tract_codebook.txt
• lending_disparities_whitepaper_180214.pdf: the whitepaper outlining Reveal's methodology

What's the goal?#

Do banks provide mortgages at disparate rates between white applicants and people of color? We're going to look at the following variables to find out:

• Race/Ethnicity
  • Native American
  • Asian
  • Black
  • Native Hawaiian
  • Hispanic/Latino
  • Race and ethnicity were not reported
• Sex
• Whether there was a co-applicant
• Applicant’s annual income (includes co-applicant income)
• Loan amount
• Ratio between the loan amount and the applicant’s income
• Ratio between the median income of the census tract and the median income of the Metro area
• Racial and ethnic breakdown by percentage for each census tract
• Regulating agency of the lending institution

Setup#

Import pandas as usual, but also import numpy. We'll need it for logarithms and exponents.

Some of our datasets have a lot of columns, so you'll also want to use pd.set_option to display up to 100 columns or so.

What is each row of our data?#

If you aren't sure, you might need to look at either the whitepaper or the codebook. You'll need to look at them both eventually, so might as well get started now.

Read in your data#

Read in our Philadelphia mortgage data and take a peek at the first few rows.

• Tip: As always, census tract columns like to cause problems if they're read in as numbers. Make sure pandas reads it in as a string.

Check your column types#

I mentioned it above, but make sure census_tract is an object (a string) or merging isn't going to be any fun later on.

Engineering and cleaning up features#

Income-related columns#

When we plotted the number of applicants, how much money they made and the size of the loan, we found that it skewed to the left, meaning the majority of applicants were clustered on the lower end of the income and loan amount scales. This was especially true for applicants of color. We took the logarithm transformation of income and loan amount to normalize the distribution of those variables and limit the effect of extreme outliers.

A few of the columns you'll need to calculate yourselves. Calculate these values and assign them to three new columns.

• Applicant’s adjusted annual income (includes co-applicant income)
• Adjusted loan amount
• Ratio between the loan amount and the applicant’s income

Instead of using the raw income and loan amount, you'll want the log of both income and loan amount. Call these new columns log_income and log_loan_amount. The third column will be loan_income_ratio.

• Tip: np.log gives you the logarithm

Co-applicants#

Right now we have a column about the co-applicant's sex (see the codebook for column details). We don't want the sex, though, we're interested in whether there is a co applicant or not. Use the co-applicant's sex to create a new column called co_applicant that is either 'yes', 'no', or 'unknown'.

• Hint: If the co-applicant's sex was not provided or is not applicable, count it as unknown.
• Hint: The easiest way is to use .replace on the co-applicant sex column, but store the result in your new column

Filter loan applicants#

If you read the whitepaper - lending_disparities_whitepaper_180214.pdf - many filters are used to get to the target dataset for analysis.

Loan type

While we recognize the substantial presence of applicants of color in the FHA market, we focused on conventional home loans for several reasons.

Property type

Prospective borrowers submit loan applications for various types of structures: one- to four-unit properties, multi-family properties and manufactured homes. For this analysis, we focused on one- to four-unit properties.

Occupancy

We included only borrowers who said they planned to live in the house they were looking to buy. We did this to exclude developers or individuals who were buying property as an investment or to subsequently flip it.

Action Type

We wanted to look at the reasons lending institutions deny people a mortgage. After conversations with former officials at HUD, we decided to include only those applications that resulted in originations (action type 1) or denials (action type 3)

Income

An applicant’s income isn’t always reported in the data. In other cases, the data cuts off any incomes over \$9.9 million and any loan amounts over \\$99.9 million, meaning there’s a value in the database, but it’s not precise. We focused only on those records where income and loan amount have an accurate estimation. This meant discarding about 1 percent of all conventional home loans in the country for 2016. [Note: I already edited this]

When we plotted the number of applicants, how much money they made and the size of the loan, we found that it skewed to the left, meaning the majority of applicants were clustered on the lower end of the income and loan amount scales. This was especially true for applicants of color. We took the logarithm transformation of income and loan amount to normalize the distribution of those variables and limit the effect of extreme outliers.

Lien status

We included all cases in our analysis regardless of lien status.

Race and ethnicity

At first, we looked at race separate from ethnicity, but that approach introduced too many instances in which​ ​either the ethnicity or race was unknown. So we decided to combine race and ethnicity. Applicants who marked their ethnicity as Hispanic were grouped together as Hispanic/Latino regardless of race. Non-Hispanic applicants, as well as those who didn’t provide an ethnicity, were grouped together by race: non-Hispanic white, non-Hispanic black, etc. [Note: This has already been taken care of]

Loan purpose

We decided to look at home purchase, home improvement and refinance loans separately from each other. [Note: please look at home purchase loans.]

Use the text above (it's from the whitepaper) and the 2015HMDACodeSheet.pdf code book to filter the dataset.

• Tip: there should be between 5-8 filters, depending on how you write them.

When you're done filtering, save your dataframe as a "copy" with df = df.copy() (if it's called df, of course). This will prevent irritating warnings when you're trying to create new columns.

Confirm that you have 10,107 loans with 19 columns#

Create a "loan denied" column#

Right now the action_type category reflects whether the loan was granted or not, and either has a value of 1 or 3.

Create a new column called loan_denied, where the value is 0 if the loan was accepted and 1 if the loan was denied. This will be our target for the machine learning algorithm.

• Tip: You should have 8,878 successful loans and 1,229 denied loans

Deal with categorical variables#

Let's go ahead and take a look at our categorical variables:

• Applicant sex (male, female, na)
• Applicant race
• Mortgage agency
• Co-applicant (yes, no, unknown)

Before we do anything crazy, let's use the codebook to turn them into strings.

• Tip: We already did this with the co_applicant column, you only need to do the rest
• Tip: Just use .replace

Double-check these columns match these values in the first three rows (and yes, you should have a lot of other columns, too).

Dummy variables#

Let's say we're at the end of this, and we have a column called sex, where 0 is female and 1 is male. After we've done our regression, we can look at the coefficient/odds ratio for sex and say something like "being male gives you a 1.5x odds of being denied a loan."

We can say this because we're looking at one column, and changing sex from 0 to 1 would turn the applicant male and give them a 1.5x chance of being denied (the odds ratio).

But let's say we're looking at a column called race instead. We could do the same 0/1 thing with white/minority, but what about white/black/asian? If we try to give them 0/1/2 our coefficient/odds ratio interpretation stops working, because we don't have a nice True/False dichotomy any more, it's now a real number.

• 0: White
• 1: Black
• 2: Asian

Usually with numbers you can say "...for every increase of 1...", but we can't anymore - changing from White to Black (+1) isn't the same as changing from Black to Asian (+1). And you can't subtract Black from Asian to get White. And no, you also can't average together White and Asian to get Black. Just recognize that these aren't numbers, they're categories!

How can we turn races off and on like we can turn the sex variable off and on? A good option is to make a 0/1 column for each race. We can then flip each race off and on. These are called dummy variables.

Seems to take up a lot of space, but it works a lot better.

• The first person is white, so they have a 1 for white and a 0 for every other race
• The second person is N/A, so they have a 1 for N/A and a 0 for every other race
• The next three are white, asian, and asian, so they have a 1 under the appropriate column.

When you're looking at the regression output, each column has its own coefficient (and odds ratio). Since each race now has a column, each race will also have its own odds ratio. Asian would have one, Black would have one, Latino would have one - now we can look at the effect of each race separately. For example, you could then say something like "being Asian (e.g., race_asian going from 0 to 1) gives you a 1.2x greater chance of being denied, and being Black gets you a 2.1x chance of being denied."

And no, you're never going to have more than one 1 in a row at the same time.

After you've created your dummy variables, there's one more step which has a real fun name: one-hot encoding.

One-hot encoding#

When we have two sexes - male and female - we can flip between them with one binary digit, 0 and 1.

If we had three races - White, Asian, Black - using pd.get_dummies would make three columns, which makes sense on the surface. But why can we put TWO values in ONE column for sex, and it takes THREE columns for the THREE race values?

The truth is, it doesn't have to!

Instead of having three columns, we're only going to have two: asian and black. And if both of them are 0? The applicant is white! This is called a reference category, and it means the coefficients/odds ratios for asian and black are in reference to a white applicant. So it isn't "being black gets you a 2.1x chance of being denied," it's being black gets you a 2.1x chance of being denied compared to a white person. For example:

To create a one-hot encoded variable with a reference category, you write code like this:

We usually use .drop(columns=...) to drop columns, but I'm using axis=1 here because you should be familiar with it

Make a one-hot encoded sex category with female as the reference category#

You should end up with two columns: sex_male and sex_na.

Using one-hot encoded columns#

Since these one-hot encoded variables are standalone dataframes, we eventually need to combine them into our original dataframe.

We have four categorical variables - sex, race, co-applicant, and the loan agency - so we need you to make four one-hot encoded variables. Name them like this:

• dummies_sex - reference category of white
• dummies_race - reference category of female
• dummies_co_applicant - reference category of no
• dummies_agency - reference category of FDIC

Typically your reference category is the most common category, because it makes for the most interesting comparisons.

Tip: if you're cutting and pasting from above, watch out for .head()

Tip: After you've made them, use .head(2) to check the first couple rows of each to make sure they look okay

Cleaning up our old dataframe#

Take a look at your original dataframe real quick.

We don't need all of those columns! If we look at the list of columns we'll be using for the regression:

• Race/Ethnicity
• Sex
• Whether there was a co-applicant
• Applicant’s annual income (includes co-applicant income)
• Loan amount
• Ratio between the loan amount and the applicant’s income
• Ratio between the median income of the census tract and the median income of the Metro area
• Racial and ethnic breakdown by percentage for each census tract
• Regulating agency of the lending institution

We can keep anything in that list, and remove everything else. For example, we can drop the variables we used to create the dummy variables, as we'll be adding the one-hot encoded versions in for the next step.

For "Racial and ethnic breakdown by percentage for each census tract" we'll need to join with some census data later, so we need to also keep census tract, county code and state code.

Build a new dataframe with only the columns we're interested in, call it numeric. We're calling it numeric because it's mostly numeric columns after the categorical ones have been removed.

Tip: You can either use .drop(columns= to remove unwanted columns or df = df[['col1', 'col2', ... 'col12']] to only select the ones you're interested in

Confirm that numeric has 8 columns.

Combining our features#

We now have 1 dataframe of numeric features (and some merge columns), and 4 one-hot-encoded variables (each with their own dataframe). Combine all five dataframes into one large dataframe called loan_features.

Confirm that loan_features has 10,107 rows and 23 columns.

Census data#

Now we just need the final piece to the puzzle, the census data. Read in the census data file, calling the dataframe census.

Tip: As always, be sure to read the tract column in as a string. Interestingly, this time we don't need to worry about the state or county codes in the same way.

Tip: You're going to encounter a problem that you find every time you read in a file from the US government!

Rename some columns#

If you like to keep your data extra clean, feel free to rename the columns you're interested in. If not, feel free to skip it!

Tip: Make sure you're using the estimates columns, not the margin of error columns

Computed columns#

According to Reveal's regression output, you'll want to create the following columns:

• Percent Black in tract
• Percent Hispanic/Latino in tract (I hope you know how Hispanic/Latino + census data works by now)
• Percent Asian in tract
• Percent Native American in tract
• Percent Native Hawaiian in tract

Notice that we don't include percent white - because all of the other columns add up to percent white, we ignore it! It's similar to a reference category.

If we want to use buzzwords here, the technical reason we're not using percent white is called collinearity. We'll talk more about it on Friday.

Only keep what we need to join and process#

We're only interested in the percentage columns that we computed. Create a new dataframe called census_features that is only those columns along with the one we'll need for joining with the mortgage data.

• Tip: Remember we saved state, county and tract codes when working on the loan data

Confirm that your first few rows look something like this:

Your column headers might be different but your numbers should match.

Merge datasets#

Merge loan_features and census_features into a new dataframe called merged.

Unfortunately something is a little different between our loan_features and census_features census tract columns. You'll need to fix it before you can merge.

Cleaning#

Do the merge#

Confirm you have 10107 rows and 31 columns in the merged dataframe.

Our final dataframe#

Drop all of the columns we merged on and save it as train_df.

Confirm that train_df has 10107 rows and 25 columns.

Final cleanup#

Because we can't have missing data before we run a regression, check the size of train_df, then drop any missing data and check the size again. Confirm you don't lose any rows.

Performing our regression#

Try with statsmodels#

First try to run a linear regression with statsmodels, because even though sometimes it complains and breaks, the output just looks so nice. Instead of sm.OLS we'll use sm.Logit.

Try again with sci-kit learn#

But some people might like sklearn. Using the coefficient to build a dataframe seems much different, so let's give it a shot.

Tip: When you build your model, use LogisticRegression(C=1e9, solver='lbfgs', max_iter=4000) - for if you don't increase max_iter (how long/hard it works) it'll complain it can't find an answer.

Getting your coefficients and odds ratios#

After you run your regression using sklearn, you can use code like the below to print out an ordered list of features, coefficients, and odds ratios.

feature_names = X.columns
coefficients = clf.coef_[0]

pd.DataFrame({
    'feature': feature_names,
    'coefficient (log odds ratio)': coefficients,
    'odds ratio': np.exp(coefficients)
}).sort_values(by='odds ratio', ascending=False)

Wait, what's the odds ratio again?#

It's how much that variable affects the outcome if all other variables stay the same.

Interpreting and thinking about the analysis#

Question 1#

Our results aren't exactly the same as Reveal's, as I pulled a slightly different number of rows from the database and I'm not sure what exact dataset they used for census information. How are we feeling about this reproduction? You might want check their 2015 results in the whitepaper.

Question 2#

In the opening paragraph to the flagship piece, Aaron and Emmanuel write:

Fifty years after the federal Fair Housing Act banned racial discrimination in lending, African Americans and Latinos continue to be routinely denied conventional mortgage loans at rates far higher than their white counterparts.

If you look at the results, Hawaiian/Pacific Islanders (and maybe Native Americans) have an even higher odds ratio. Why do they choose to talk about African Americans and Latinos instead?

Question 3#

Write a sentence expressing the meaning of the odds ratio statistic for Black mortgage applicants. Find a line in the Reveal piece where they use the odds ratio.

Question 4#

Write a similar sentence about men.

Question 5#

Why did Aaron and Emmanuel choose to include the loan-to-income ratio statistic? You might want to read the whitepaper.

Question 6#

Credit score is a common reason why loans are denied. Why are credit scores not included in our analysis? You might want to read the whitepaper.

Question 7#

This data was just sitting out there for anyone to look at, they didn't even need to FOIA it. Why do you think this issue had not come up before Reveal's analysis?

Question 8#

As a result of this series, a lot has happened, although recent changes don't look so good. If you were reporting this story, what groups of people would you want to talk to in order to make sure you're getting the story right?

Question 9#

When they were consulting experts, Aaron and Emmanuel received a lot of conflicting accounts about whether they should include the "N/A" values for race (they ended up including it). If the experts disagreed about something like that, why do you think they went forward with their analysis?

Question 10#

What if we were working on this story, and our logistic regression or input dataset were flawed? What would be the repercussions?