Evaluating a Science Diversity Program at UC Berkeley: More Questions Than Answers

INTRODUCTION

The population of the United States is undergoing dramatic demographical changes. In 2000, the U.S. population numbered over 280 million people, increasing 38% from 1990, with dramatic increases in the numbers of racial/ethnic minorities. In 1999, the U.S. Department of Education reported that approximately 44% of the population between 18 and 24 years of age was enrolled in a 4-year degree-granting institution. Although underrepresented minorities, including African Americans, Hispanics, and Native Americans, constitute 25% of the U.S. population, they received a disproportionately low percentage (16%) of all bachelors degrees awarded. The percentage decreases further when examining degrees awarded in the fields of math, science, and engineering. For example, of the 65,000 bachelors degrees awarded nationally in the biological sciences in 1999, underrepresented minorities received less than 13%. Although the numbers of academically successful underrepresented minorities have grown over the past four decades, there still exists a significant graduation gap between majority and minority students, especially in the sciences.

For more than 30 years, much attention has been focused on developing programs designed to improve academic success for underrepresented minorities. A large percentage of these programs has been designed for college students entering the fields of math, science, and engineering. In 1999, a report commissioned by the College Board, a not-for-profit educational association, described and assessed 24 college and university programs involved in promoting the high achievement of underrepresented minority students (Gandara and Maxwell-Jolly, 1999). Although the study describes “what works,” few quantitative data are provided as to the effectiveness of the programs in increasing graduates. In addition, a separate study commissioned by the College Board in the same year reported that only a few promising programs were found to have undergone extensive external evaluation (Cota-Robles and Gordan, 1999).

The University of California (UC), Berkeley, is well known for both its academic programs and its diverse student population. However, between 1992 and 1999, of the students majoring in the biological sciences, only 4% were African American and 9% were Hispanic. In 1992, The Biology Scholars Program (BSP) was established in the Department of Molecular and Cell Biology at UC Berkeley. Funded by the Howard Hughes Medical Institute, BSP is a program designed to promote the success of undergraduates from economic, gender, ethnic, and cultural groups historically underrepresented in the biological sciences. BSP is an academically centered program administratively housed in UC Berkeley's Department of Molecular and Cell Biology. It is a continuum of resources available to help its members address critical transitions (e.g., making the high school-to-university academic and social adjustment, declaring a major, applying to graduate or professional school) throughout their undergraduate years. Program components include academic support for lower-division courses in the biology major, an academic and career seminar series, social events, access to on- and off-campus paid research opportunities, academic and personal advising, and mentoring. By design, BSP also addresses issues that encompass students' lives “beyond academics,” such as the impact of family, financial, personal, etc., issues on their performance at Berkeley. Overall, the goal of BSP is to create a community of scholars with both high academic expectations and high academic support, which allows its members to network with culturally sensitive faculty, staff, more senior undergraduates, and graduate students in an academic context.

Any student admitted to the UC Berkeley undergraduate program expressing an interest in the biological sciences is qualified to join BSP. Underrepresented high school students applying to UC Berkeley with an interest in the biological sciences, as indicated on their application, are mailed information including an invitation to apply to the program. UC Berkeley students from low-income and/or first-generation college backgrounds are identified through various on-campus programs, provided an informational meeting, and encouraged to apply. Current students having difficulties adjusting to the university are referred to BSP by professors and college/major advisors. Finally, current BSP members can recommend students for acceptance into the program. Selection into the program is based on an informational one-on-one meeting with either the Director or the Assistant Director of BSP that includes a written application and a final interview.

Between 1992 and 1999, African American and Hispanic students made up 28 and 31% of the program participants, respectively. The goal of BSP is to diversify those participating in science by targeting Berkeley undergraduates from these underrepresented groups. Through collaborative academic workshops, research opportunities, career/course advising, and a student center located in the heart of the main science building on the UC Berkeley campus, students develop a community through academic as well as social activities.

The specific aims of this article are (1) to present the results of a quantitative evaluation of the success of BSP students, (2) to discuss programmatic research questions raised by these results, and (3) to develop an outline for future studies.

METHODS

To assess the impact of BSP on its members to date, we compared program and nonprogram members in terms of the following.

1. Academic preparation—as measured by their uncapped high-school GPA and their combined score for the math and verbal SAT tests.

2. Success—as measured by

   1. The percentage of intended biology majors who graduated with a biology degree.

   2. Comparisons of final UC GPA.

Data on biology graduates were taken from the Central Campus Student Database.

Research Participants

The data used for this analysis span the academic years 1992 through 1999, and using these data, we compare BSP and non-BSP students graduating with a degree in the biological sciences between 1994 and 1999. Spring of 1994 saw the first BSP graduates. Our comparison data sets include all 1994–1999 BSP (n = 143) and non-BSP (n = 1904) biology graduates (Table 1). In our study,“ intended biology majors” include those students who state on their admissions application that they intend to major in 1 of 11 majors: bioengineering, conservation and resource studies, environmental science, forestry, genetics and plant biology, integrative biology, microbial biology, molecular and cell biology, molecular environmental biology, nutritional sciences, and resource management. It should be noted that intended majors in biology who do not graduate with a biology degree have not necessarily left the university but may have switched to an alternative major.

In addition to Intended Major, the data sets of 1994–1999 BSP and non-BSP graduates also include Ethnicity, Math–Verbal–Total SAT Scores, Uncapped High School GPA (i.e., GPAs that exceed a maximum of 4.0 if a student takes honors and/or advanced placement courses), and Final UC GPA at graduation.

All students in the study are “intended biology majors,”“ All Groups” includes “Majority” +“ Minority” students, “Majority” includes Asian and White students, and “Minority” includes African-American and Hispanic (Chicano/Latino) students. Native American and Pacific Island students were not included in this analysis because of their small numbers.

Regarding graduates who were once transfer students, because most of them completed their lower-division science and mathematics courses at their community colleges before their admission to the university, we cannot assume that their academic experiences (e.g., lower-division class size, testing, workload) were similar to those of students who entered Berkeley as freshmen. Therefore, in this study students who transferred to the university in advanced standing were omitted from the analyses. In a future paper, the success of BSP and non-BSP transfer students will be compared.

Statistical Analysis

Uncapped high school GPA, SAT test scores, and final UC GPA were analyzed with the aid of Statview software. Data were analyzed via ANOVA followed by Scheffé's (1953) F post hoc test to determine significance between groups. Statistical significance was accepted at P < 0.05.

Percentages of intended biology majors graduating with a biology degree were analyzed using the G-statistic (Sokal and Rohlf, 1981). The percentages of non-BSP students graduating with a biology degree were used to generate expected values for the analysis of BSP groups. P values were obtained from a χ² table and significance was accepted at P < 0.05.

RESULTS

The goal of this analysis was to determine if participation in BSP had a positive effect on student success in biological sciences disciplines while at UC Berkeley. Thus, the evaluation required an analysis of the students' preparation on arrival as well as their performance while at Berkeley.

Academic Preparation: SAT and High School Grade Point Averages—BSP vs. Non-BSP Students

The purpose of this comparison was to determine if BSP members were comparable in academic preparation to students not in BSP, where preparation is narrowly defined by high school GPA and performance on standardized tests. Table 2 summarizes the results. This analysis addressed the question of whether, in our selection of BSP members, we had “socially engineered” a more academically prepared (and therefore more likely to succeed) population of students by“ skimming” only those students with higher SAT scores and high school GPAs.

Overall, BSP students had lower high school GPAs and total SAT scores than non-BSP members. Majority BSP students had lower total SAT scores than non-BSP Majority students, while African-American and Hispanic students had statistically similar high school GPAs and total SAT scores compared to African-American and Hispanic students not in the program. These results suggest that BSP members are similar, in terms of high school GPAs, or less prepared, in terms of total SAT scores, compared to non-BSP students of the same race/ethnicity.

First Measure of Success—Percentage of Intended Biology Majors Who Graduate with a Biology Degree

One goal of BSP is to increase the number of students graduating with undergraduate degrees in the biological sciences. Indeed, both Majority and Minority BSP members graduate in significantly higher percentages than majority and minority students, respectively, not in the program (Figure 1). Also, when Minority is disaggregated into African-American and Hispanic students, BSP members in each case graduate in biology in significantly higher percentages than non-BSP members (Figure 1). The possible explanations for this first measure of BSP member success (e.g., the greater motivation of BSP vs. non-BSP students, or a positive program effect, or both) will be the focus of future research (see Discussion).

Second Measure of Success—Final UC Grade Point Average of Graduates in Biology

While graduation per se is a goal for BSP students, graduating with academic excellence is an even more stringent measure of success. The final GPA upon graduation often dramatically influences a student's ability to continue into graduate or professional school and is one measure of the level of learning achieved during the undergraduate years. Thus, final UC GPAs were compared among BSP members and nonmembers in biology. The data indicate that BSP Minority (African American and Hispanic) students graduated with significantly higher UC GPAs than students of similar background not in the program (Figure 2). All Groups and Majority students in BSP graduated with similar UC GPAs compared to non-BSP members. Thus, Minority BSP members are not only graduating at higher rates, but graduating with statistically similar GPAs.

Summary of Results—BSP vs. non-BSP Comparisons

Table 3 and the accompanying text summarize a comparison of BSP and non-BSP graduates in terms of their preparation (SAT and High School GPA) and success (Completion of a Biology Degree and Final UC GPA).

1. For All Graduates—While BSP members show significantly lower high school GPAs and total SAT scores upon entering UC Berkeley (Table 2), they graduate with a biology degree in significantly higher percentages than non-BSP members regardless of race/ethnicity (Figure 1). For all racial/ethnic groups combined (All Groups), there is no statistical difference in final UC GPA (Figure 2).

2. For Majority Graduates—BSP Majority students enter UC Berkeley with significantly lower total SAT scores (Table 2) yet graduate with a higher percentage of biology degrees and a UC GPA equivalent to that of non-BSP majority students (Figures 1 and 2). BSP Majority students have similar high school GPAs compared to non-BSP Majority students.

3. For Minority Graduates—Minority BSP students enter UC Berkeley with equivalent high school SAT scores and GPAs and graduate with a higher percentage of biology degrees and a higher UC GPA than nonmembers (Figures 1 and 2).

4. For African-American Graduates—BSP and non-BSP African-American students enter UC Berkeley with similar high school GPAs and SAT scores (Table 2), yet BSP members graduate with a higher percentage of biology degrees and a higher UC GPA than students not in the program (Figures 1 and 2).

5. For Hispanic Graduates—BSP and non-BSP Hispanic students enter UC Berkeley with similar high school GPAs and SAT scores (Table 2), yet BSP members graduate with a higher percentage of biology degrees and a higher UC GPA than students not in BSP (Figures 1 and 2).

DISCUSSION

Our analysis of student performance suggests that underrepresented students in BSP have attained parity with nonprogram majority students in terms of graduating with a biology degree and in terms of their GPA at graduation. In fact, BSP Minority students graduate with statistically similar final UC GPAs compared to non-BSP Majority students (data not shown). In other words, by their success in biology at UC Berkeley, BSP graduates have closed the minority–majority gap. This raises the important questions of (1) What possible role has BSP had in their success? and (2) What can other institutions do to help their undergraduates replicate this success?

What kinds of activities characterize BSP? Many are familiar to those with program goals similar to those of BSP, including study groups, paid research opportunities, and mentoring by culturally sensitive faculty and advisors who teach students “how to think about” their major and career choices and provide “reality checks” regarding their readiness to succeed in specific courses, research, and summer internships. In addition, the BSP Center is housed within the academic biology community, and the students immediately are part of that physical identification with biology. Finally, a student advisory committee has been integral in voicing the needs and concerns of BSP members, planning activities each semester that reflect their interests and needs.

However, no listing or description of components can explain the effectiveness of BSP. “Building community,”“ mentoring,” “addressing critical transitions,” and“ providing research opportunities and academic support” are common phrases used to describe what programs like BSP do to address underrepresentation in science (e.g., see the inventory of common characteristics of successful science diversity programs in Gandara and Maxwell-Jolly [1999]). If it is as simple as providing students with this fairly agreed-upon list of conditions for their success, then why, after three decades of effort across the United States, is there still underrepresentation in science at the undergraduate, graduate, and professional levels?

It is within this context that we raise a number of questions about the overall success of BSP for further study.

1. Are BSP students simply more motivated than Berkeley students at large, and, if so, might this difference account for the results in Figures 1 and 2?

2. In Figure 2, what is the“ real-world” significance of these statistically significant differences in final UC GPA? At UC Berkeley, undergraduates know their GPAs to the 1000th decimal point: Might a 3.160 vs. a 2.949 GPA make a real difference in students' self-concept and their ability to succeed in science?

3. Many BSP members participate in research while at Berkeley. Is there a correlation among research, graduation rates, and final GPA for BSP members and/or nonmembers?

4. Is gender a significant factor in the success of BSP vs. non-BSP students? The majority of participants in BSP are women, over 65%. Based on our current measures of preparation and success, statistical analyses show no significant differences between men and women within and outside the program.

From nearly a decade of working successfully with BSP members, it is our strong sense that our single greatest challenge is to go beyond simply describing what we have done to explaining why it has worked, in applicable ways/formats that we can share with our colleagues. Toward elevating BSP to “the next level” as we go into the 11th year of the program, we view this study as a formative step toward understanding BSP in terms of what is working, what is not, and for whom.

Our next step will be to expand our quantitative analysis to include a more in-depth look at student background, preparation, and performance using more available data elements (e.g., parental income and education, SAT2 subject scores) in our Central Campus Student Database. At the same time, we will expand our assessment with assistance from qualitative researchers to help us understand (beyond the numbers) those factors that affect student success in biology at Berkeley. Our plan is to use both quantitative and qualitative data to refine the structure, operation, and effectiveness of BSP and, through publications and presentations, to share our understandings with colleagues engaged in equity and access issues in science across the country.

FOOTNOTES

ACKNOWLEDGMENTS