Gender Bias in Graduate Admissions

Berkeley

In 1973, The University of California - Berkeley was one of the top-ranked universities in the United States. As such Berkeley attracted thousands of applicants to its graduate school. But how many made the cut?

We will start off by loading the UCBAdmissions dataset, which is included in base R. This shows the number of students male and female who were admitted or rejected from the six largest departments at Berkeley. Since the dataset takes the unwieldly form of a three-dimensional array, we will convert it to tidy format using the tidy function from the tidytext package. Then we will be ready to start doing some analysis.

# Loading UCBAdmissions dataset
data(UCBAdmissions)

Printing the dataset to the console:

Dept A B C D E F
Admit Gender
Admitted Male 512 353 120 138 53 22
Female 89 17 202 131 94 24
Rejected Male 313 207 205 279 138 351
Female 19 8 391 244 299 317
# Loading broom package
library(broom)

# Converting UCBAdmissions to tidy format
ucb_tidy <- tidy(UCBAdmissions)

Printing the dataset to the console:

Admit Gender Dept n
Admitted Male A 512
Rejected Male A 313
Admitted Female A 89
Rejected Female A 19
Admitted Male B 353
Rejected Male B 207
Admitted Female B 17
Rejected Female B 8
Admitted Male C 120
Rejected Male C 205
Admitted Female C 202
Rejected Female C 391
Admitted Male D 138
Rejected Male D 279
Admitted Female D 131
Rejected Female D 244
Admitted Male E 53
Rejected Male E 138
Admitted Female E 94
Rejected Female E 299
Admitted Male F 22
Rejected Male F 351
Admitted Female F 24
Rejected Female F 317

Acceptance rate for men and women

The data is more readable now that it is in tidy format, but since it is split by department and displays raw counts, it is difficult for us to infer any kind of gender bias. To do that, we need to aggregate over department and ask ourselves, overall, what proportion of men and women were accepted into Berkeley in 1973.

Here we make use of the dplyr package for all of our data manipulation tasks. We aggregate over department using the group_by function to get the total number of men and women who were accepted into or rejected from Berkeley that year, as well as the proportion accepted in each case. That will leave us in a better place to understand any accusations of gender bias.

# Loading the dplyr library
library(dplyr)

# Aggregate over department
ucb_tidy_aggregated <- ucb_tidy %>% 
  group_by(Admit, Gender) %>% 
  summarize(n = sum(n)) %>% 
  ungroup() %>% 
  group_by(Gender) %>% 
  mutate(prop = n/sum(n)) %>% 
  filter(Admit == "Admitted")

# Print aggregated dataset
ucb_tidy_aggregated
## # A tibble: 2 x 4
## # Groups:   Gender [2]
##   Admit    Gender     n  prop
##   <chr>    <chr>  <dbl> <dbl>
## 1 Admitted Female   557 0.304
## 2 Admitted Male    1198 0.445

Visualizing the discrepancy

From the previous task, we can see that 44.5% of male applicants were accepted into Berkeley, as opposed to 30.4% of female applicants. Now we can start to see the problem. Did Berkeley’s graduate admissions department really discriminate against women in 1973?

Before we consider alternative explanations, let’s visualize the discrepancy through a simple bar chart using ggplot2. This won’t add much to our understanding of the problem right now, but it will act as a useful reference point later on.

For clarity, we will format acceptance rate as a percentage using the percent function from the scales package.

# Load the ggplot2 and scales packages
library(ggplot2)
library(scales)

# Prepare the bar plot
gg_bar <- ucb_tidy_aggregated %>% 
    ggplot(aes(x = Gender, y = prop, fill = Gender)) +
    geom_col() +
    geom_text(aes(label = percent(prop)), vjust = -1) +
    labs(title = "Acceptance rate of male and female applicants",
         subtitle = "University of California, Berkeley (1973)",
         y = "Acceptance rate") +
    scale_y_continuous(labels = percent, limits = c(0,0.5)) +
    guides(fill = FALSE)

# Print the bar plot
print(gg_bar)

Acceptance rate by department

The bar plot confirms what we already knew, a higher proportion of men were accepted than women. But what happens when we separate the graph out by department?

Now we can return to our ucb_tidy dataset. After calculating the proportion of acceptances/rejections, we will plot a separate chart for each department using the facet_wrap() function in ggplot2 This will give us an idea of how acceptance rates differ by department, as well as by gender.

# Calculate acceptance/rejection rate
ucb_by_dept <- ucb_tidy %>% 
    group_by(Gender, Dept) %>% 
    mutate(prop = n/sum(n)) %>% 
    filter(Admit == "Admitted")

# Print the dataset
print(ucb_by_dept)
## # A tibble: 12 x 5
## # Groups:   Gender, Dept [12]
##    Admit    Gender Dept      n   prop
##    <chr>    <chr>  <chr> <dbl>  <dbl>
##  1 Admitted Male   A       512 0.621 
##  2 Admitted Female A        89 0.824 
##  3 Admitted Male   B       353 0.630 
##  4 Admitted Female B        17 0.68  
##  5 Admitted Male   C       120 0.369 
##  6 Admitted Female C       202 0.341 
##  7 Admitted Male   D       138 0.331 
##  8 Admitted Female D       131 0.349 
##  9 Admitted Male   E        53 0.277 
## 10 Admitted Female E        94 0.239 
## 11 Admitted Male   F        22 0.0590
## 12 Admitted Female F        24 0.0704
# Prepare the bar plot for each department
gg_bar_faceted <- ucb_by_dept %>% 
  ggplot(aes(Gender, prop, fill = Gender)) +
  geom_col() +
  geom_text(aes(label = percent(prop)), vjust = -1) +
  labs(title = "Acceptance rate of male and female applicants",
       subtitle = "University of California, Berkeley (1973)",
       y = "Acceptance rate") +
  scale_y_continuous(labels = scales::percent, limits = c(0, 1)) +
  facet_wrap(~Dept)  +
  guides(fill = FALSE)

# Print the bar plot for each department
print(gg_bar_faceted)

## Alternative explanations Now that we have separated out our analysis by department, the interpretation has changed rather dramatically. Although men were indeed more likely to be admitted into Departments C and E, women were more likely to be admitted into all other departments. So what’s really going on here?

If you turn your attention to the first two plots, you can see that Department A and B were quite easy to get into. However, relatively few women applied to these departments – only 108 women applied to Department A, as opposed to 825 men!

At this stage, we can hypothesise that the effect of gender on acceptance is null when you control for department. We can test that hypothesis using binary logistic regression, but first we need to de-aggregate the dataset so that each row represents one student. That should leave us with 4,526 rows – one row for each student who applied to Berkeley that year.

# Define function that repeats each row in each column n times
multiply_rows <- function(column, n) {
  rep(column, n)
}

# Create new de-aggregated data frame using the multiply_rows function
ucb_full <- data.frame(Admit = multiply_rows(ucb_tidy$Admit, ucb_tidy$n),
                      Gender = multiply_rows(ucb_tidy$Gender, ucb_tidy$n),
                      Dept = multiply_rows(ucb_tidy$Dept, ucb_tidy$n))

# Check the number of rows equals the number of students
nrow(ucb_full)
## [1] 4526

Binary logistic regression: part i

The data is now in the right format for us to do some hypothesis testing. Great! But first let’s try to predict admittance using gender alone. We will use the built-in glm() function to fit a generalised linear model, making sure to set family = "binomial" because the outcome variable is binary (Admitted or Rejected).

By default, Admit is coded such that Admitted is level 1 and Rejected is level 2 (because of their alphabetical order). Since glm() will assume that level 2 represents ‘success’, we will reverse the coding of Admit so we are predicting the probability of admittance rather than rejection.

To change the coding of a variable, you can use the fct_relevel() function from the forcats package.

# Load the forcats library
library(forcats)

# Reverse the coding of the Admit variable
ucb_full$Admit <- fct_relevel(ucb_full$Admit, "Rejected", "Admitted")

# Run the regression
glm_gender <- glm(Admit ~ Gender, data = ucb_full, family = "binomial")

# Summarize the results
summary(glm_gender)
## 
## Call:
## glm(formula = Admit ~ Gender, family = "binomial", data = ucb_full)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.0855  -1.0855  -0.8506   1.2722   1.5442  
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -0.83049    0.05077 -16.357   <2e-16 ***
## GenderMale   0.61035    0.06389   9.553   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 6044.3  on 4525  degrees of freedom
## Residual deviance: 5950.9  on 4524  degrees of freedom
## AIC: 5954.9
## 
## Number of Fisher Scoring iterations: 4

Binary logistic regression: part ii

Sure enough, when you predict the probability of admission as a function of gender alone, the effect is statistically significant (p < 0.01). Specifically, you are exp(0.61035) = 1.84 times more likely to be admitted if you are a man. However, what happens if we control for department?

# Run the regression, including Dept as an explanatory variable
glm_genderdept <- glm(Admit ~ Gender + Dept, data = ucb_full, family = "binomial")

# Summarize the results
summary(glm_genderdept)
## 
## Call:
## glm(formula = Admit ~ Gender + Dept, family = "binomial", data = ucb_full)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.4773  -0.9306  -0.3741   0.9588   2.3613  
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept)  0.68192    0.09911   6.880 5.97e-12 ***
## GenderMale  -0.09987    0.08085  -1.235    0.217    
## DeptB       -0.04340    0.10984  -0.395    0.693    
## DeptC       -1.26260    0.10663 -11.841  < 2e-16 ***
## DeptD       -1.29461    0.10582 -12.234  < 2e-16 ***
## DeptE       -1.73931    0.12611 -13.792  < 2e-16 ***
## DeptF       -3.30648    0.16998 -19.452  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 6044.3  on 4525  degrees of freedom
## Residual deviance: 5187.5  on 4519  degrees of freedom
## AIC: 5201.5
## 
## Number of Fisher Scoring iterations: 5

Behold Simpson’s paradox

Finally, we can see Simpson’s paradox at play – when you control for the effect of department on the probability of admission, the effect of gender disappears. In fact, it even reverses, suggesting that – controlling for department – you were actually more likely to be admitted as a woman! However, this effect is not statistically significant (p > 0.05), so we conclude that there was not a campus-wide bias against applicants of either gender in 1973.

That said, individual departments often handle their own admissions processes, so it is plausible that bias exists in one department but not another. Let’s take a look at Department A, where 82.4% of women were admitted but only 62.1% of men. Is the difference statistically significant?

# Filter for Department A
dept_a <- ucb_full %>% filter(Dept == "A")

# Run the regression
glm_gender_depta <- glm(Admit ~ Gender, data = dept_a, family = "binomial")

# Summarize the results
summary(glm_gender_depta)
## 
## Call:
## glm(formula = Admit ~ Gender, family = "binomial", data = dept_a)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.8642  -1.3922   0.9768   0.9768   0.9768  
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept)   1.5442     0.2527   6.110 9.94e-10 ***
## GenderMale   -1.0521     0.2627  -4.005 6.21e-05 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 1214.7  on 932  degrees of freedom
## Residual deviance: 1195.7  on 931  degrees of freedom
## AIC: 1199.7
## 
## Number of Fisher Scoring iterations: 4

Bias or discrimination?

Well then! If we take Department A in isolation, we find there is a statistically significant bias in favour of women. So does that mean that the department discriminated against men?

Not necessarily. After all, the bias might exist simply because the female applicants to Department A were better qualified that year. In their article dealing with this issue, Bickel, Hammel & O’Connell (1975) define discrimination as “the exercise of decision influenced by the sex of the applicant when that is immaterial to the qualifications for entry”. Since we do not have any data on the respective qualifications of the candidates, we cannot say whether any gender bias in their admissions process amounted to discrimination.

Although now more than 40 years old, the Berkeley problem is a useful reminder about the dangers of aggregation and omitted variable bias, especially in relation to matters of such legal and ethical importance as discrimination. Where bias does exist – as it does in the case of Department A – it is always worth considering whether there are any other factors that could explain the discrepancy.

# Define bias
bias <- "A pattern of association between a particular decision and a particular sex of applicant, of sufficient strength to make us confident that it is unlikely to be the result of chance alon"

# Define discrimination
discrimination <- "The exercise of decision influenced by the sex of the applicant when that is immaterial to the qualifications for entry"

# Is bias equal to discrimination?
bias == discrimination
## [1] FALSE

This project was completed as part of Datacamp projects