declining to address the plaintiffs’ Title VI claim because it was unnecessary given the resolution of the identical claim under Title VII and therefore, anything written about the Title VI claim would constitute “an advisory opinion”
How later courts described this case
- declining to address the plaintiffs’ Title VI claim because it was unnecessary given the resolution of the identical claim under Title VII and therefore, anything written about the Title VI claim would constitute “an advisory opinion”
- ruling on whether Title VI and VII apply even though the issues were unnecessary to the disposition of the case
- noting approval of cases “refusing to award costs to a prevailing party” on the basis of “the losing party’s limited financial resources”
- “[D]ivesting district courts of discretion to limit or to refuse such overwhelming costs in important, close, but ultimately unsuccessful civil rights cases like this one might have the regrettable effect of discouraging poten- tial plaintiffs from bringing such cases at all.”
Written by the judges who cited it.
The opinion
REINHARDT, Circuit Judge,
with whom Circuit Judges SCHROEDER and THOMAS join, concurring in part and dissenting in part:
The result we reach is most unfortunate. Not only is it wrong as a matter of law, but it may lead some to conclude that we are insensitive to the needs and problems of minority educators and minority students. While that may not be so, one thing is certain. The loser in the case before us is clearly the already beleaguered public education system of the state of California.
Although I concur in Section IA and Parts II, III, and TV of the majority opinion, I cannot agree with the majority’s *594 holding that the CBEST was properly validated under Title VII. As a result of this ruling, qualified minority educators— teachers, administrators, librarians, and other officials — will be denied the opportunity to work in California’s severely understaffed public schools, simply because they failed to pass a test that concededly has a disparate impact on minority group members. Despite Title VII’s clear requirement that a test that has a disparate impact on minorities must be validated by reference to a particular job, the majority accepts validation studies that fail to differentiate among different school jobs, ranging from bilingual education teachers to mathematics teachers to physical education teachers. Further, even though Title VII demands that the defendants demonstrate “by professionally acceptable methods” that the test items are predictive of or significantly correlated to job-related skills, the majority finds that the CBEST met this requirement through a study that the district court found to be “unscientific” and “not particularly helpful.”
A sampling of the plaintiff class illustrates the disservice the majority decision does not only to minority educators but also to the students in the California public school system. The plaintiff class includes a Cambodian-born applicant with a postgraduate teacher preparation certificate and a Bachelor’s degree in Liberal Studies from a California State University, who is seeking to teach bilingual elementary school classes to address the needs of over 24,000 Khmer-speaking students in California public schools. He is qualified but for the fact that he is four points short of obtaining the required score on the reading section of the CBEST. The class also includes a Latino resident of Los Angeles County, in which Latinos comprise the largest ethnic group. The man seeks a position as a counselor in the public schools, and has an undergraduate degree from University of California at Irvine and a Master’s degree in Counseling from California State University at Los Angeles, but has failed the reading section of the CBEST. He too is qualified, but for the test. Another class member is an African-American Assistant Principal and physical education teacher with a Bachelor’s degree and a Master’s degree in Physical Education from Stanford, who was promoted from his position as a teacher to Assistant Principal, and who received excellent performance reviews as an Assistant Principal. He has been denied the opportunity to return to his post as Assistant Principal because he is six points short of passing the math portion of the CBEST. The result of the failure to validate the CBEST in a lawful manner is to perpetuate discrimination against minority teachers who could be of particular assistance to minority students and serve as role models for them. It is significant in this regard that minority students outnumber white students in many of California’s largest school districts, in some cases by substantial amounts.
As the majority notes, we follow the three-step approach set forth in Craig v. County of Los Angeles, 626 F.2d 659, 662 (9th Cir.1980) to determine whether a test is job related. I believe that the district court erred in applying the second and third steps of the Craig test, because (1) the studies relied on by the CTC (Commission on Teacher Credentialing) do not differentiate among different school jobs in determining the skills to be tested; and (2) the studies do not demonstrate by professionally acceptable methods that the test items actually measure the job-related skills identified by the CTC. Either of these errors in the method of validation would be sufficient to render the CBEST invalid. Finally, although I believe that the district court’s determination of job-relatedness should be reversed even under the clear error standard adopted by the majority, the proper standard for reviewing the determinations at issue in this case is de novo.
I. Failure to validate for separate jobs
The use of a test that has a disparate impact under Title VII is impermissible *595 ‘job unless it is proven that the test is related for the position in question.” 42 U.S.C. § 2000e-2(k)(l)(A)(i) (emphasis added). The Uniform Guidelines provide that “[a]ny validity study should be based upon a review of information about the job for which the selection procedure is to be used.” 29 C.F.R. § 1607.14 (A). The CBEST is invalid because it was not validated with specific reference to the various particular jobs for which the CBEST is used.
The Supreme Court clarified the job-specific validation requirement in Albemarle Paper Co. v. Moody, holding that “[a] test may be used in jobs other than those for which it has been professionally validated only if there are no significant differences between the studied and unstudied jobs.” 422 U.S. 405, 432 , 95 S.Ct. 2362 , 45 L.Ed.2d 280 (1975). In Albe-marle, employment tests designed to measure general intelligence were administered to ten different job groups, which were primarily grouped together on the basis of their proximity to each other in the line of progression, rather than in terms of the specific skills that the jobs required. Id. at 429-30 , 95 S.Ct. 2362 . The Court held the tests invalid in part because the validation study focused on higher-skilled positions on the production lines without adequately examining whether the tests were valid for workers entering lower level jobs. Id. at 433-34 , 95 S.Ct. 2362 . Further, the Court noted that one of the tests required for the skilled lines of progression correlated to job performance in only three of the eight lines at issue. Id. at 431 , 95 S.Ct. 2362 . Because the study had not shown that there were “no significant differences” between the various jobs this test purported to evaluate, the test was held invalid. Id. at 432 , 95 S.Ct. 2362 .
We required a similar level of job specificity in Craig , which involved, in relevant part, a height requirement for prospective sheriffs. Craig, 626 F.2d at 666-68 . Although the height requirement was justified by reference to the need for officers to “command respect,” we rejected the generalization that “all police officers expend a significant portion of their time and effort in confrontation and control conditions” because “[n]o study proved the generalization to be a fact.” Craig, 626 F.2d at 667 . We therefore required the defendants to validate the requirement by reference to more than a general description of a police officer’s functions, because different police officers have different duties. Other circuits have also rejected attempts to use a single test to measure skills for jobs that were generally similar but had appreciably different specific duties. 1
The CBEST was validated without regard to the important differences among the various jobs at issue, which range from twelfth grade mathematics teachers to seventh grade bilingual education teachers to guidance counselors, librarians, and administrators. The 1982 Wheeler and Elias study encompassed skills relevant to the job of “teaching at any grade level;” the 1985 Practioner’s Review study divided the “particular jobs” into four groups, specifically (1) elementary school teachers, (2) secondary school teachers, (3) pupil personnel and librarians, and (4) administrators; and the Lundquist study divided the pool into teachers and administrators. Thus, even the study that provided the *596 most specific analysis of job categories did not differentiate among the job duties of bilingual education teachers, mathematics teachers, and physical education teachers, for example.
The majority states that the CBEST establishes only “a minimum level of competence” in skills that are job related for all the positions in question. While it may be possible to construct a test that measures skills relevant for all teachers, Albe-marle holds that such a test may only be used if it has been validated for each “particular job” involved or if there are “no significant differences” among the jobs at issue. Albemarle, 422 U.S. at 432 , 95 S.Ct. 2362 . The positions at issue in AMAE bear less resemblance to each other than those in Craig, Albemarle, or the cases from other circuits. If a validation study must differentiate between different officers in the sheriffs department or workers on different lines at the paper production factory, it is certainly improper for the CBEST to be validated by reference to undifferentiated but significantly different teaching jobs. A validation study that focuses on the “particular job” of “a teacher” or even “a secondary school teacher” fails to satisfy the legal requirement that the examination predict job performance for each particular job position in question. Teachers’ jobs are too diverse to be lumped into one or two categories for validation purposes. There is no legal or logical basis to support the majority’s abandonment of the job-specific validation requirement employed in Albemarle and Craig . In doing so, it clearly erred.
II. Failure to match test items to skills important to the job
As the majority notes, the third step of the Craig test requires employers to “demonstrate by ‘professionally acceptable methods’ that the selection device is ‘predictive of or significantly correlated’ with the element of work behavior identified [by the job analysis].” Craig, 626 F.2d at 662 (quoting Albemarle Paper, 422 U.S. at 431 , 95 S.Ct. 2362 ). In other words, the employer must demonstrate not only that the skills identified in the job analysis are job related, but also that the test items actually measure those skills. In Contreras v. City of Los Angeles, for example, this requirement was satisfied by an independent “examination review phase” in which a new group of job experts reviewed each test question and decided whether it tested one of the critical elements identified in the preceding “job analysis” phase. 656 F.2d 1267 , 1282 (9th Cir.1981); cf. Craig, 626 F.2d at 667 (finding that testimony from the sheriff and a physical training instructor that height was important in commanding respect and applying restraint was insufficient to show by professionally acceptable methods that the height requirements were predictive of the ability to carry out the “control functions” of the job). As the majority acknowledges, “a key requirement of [the] third step, a requirement essential to the proof of job relatedness, is that the validation method be professionally acceptable.” Contreras, 656 F.2d at 1282. Here, even assuming that the job analysis was sufficiently specific to generate skills important for all the positions in question, the CBEST is invalid because the CTC failed to demonstrate in a professionally acceptable manner that the test actually measures the skills identified in the job analysis.
The majority appears to concede that neither the Practitioner’s Review study nor the Lundquist study attempted to match the test items to the job skills as required under Craig. Maj. op. at 589. To support the district court’s finding of validity, the majority cites evidence describing the process of test development employed by ETS. However, this evidence merely purports to show that the test items were expertly developed; it does not demonstrate, as the third step of the Craig test requires, that the final version of the CBEST administered to job applicants is “predictive of or significantly correlated” with the skills identified in the job analysis *597 as important for the job. Where an employer seeks to use a test that has a disparate impact on minorities, it must present evidence that the test actually measures the skills that have been identified as important for the job, and may not rely solely on the fact that it used a “well known and respected as a developer of standardized tests.” Maj. op. at 588 (quoting the report of Dr. William A. Mehrens).
As evidence that the test questions were “matched” to the skills they were intended to measure, the majority points only to the Curriculum Matching Project, in which two ETS employees matched CBEST test specifications to material found in textbooks purportedly used in California public schools. The district court criticized this study as “unscientific” and “not particularly helpful,” but nevertheless upheld the test. AMAE v. State of California, 937 F.Supp. 1397 , 1412 n. 21 (N.D.Cal.1996). We have explicitly held that it is “essential” that the item validation method be professionally acceptable. Contreras, 656 F.2d at 1282. “Unscientific” and “not particularly helpful” tests clearly do not meet that standard. Thus, even under a clear error standard, the district court’s determination that step three of the Craig test was satisfied requires reversal.
III. Standard of review
It is plain from a reading of the majority’s opinion that it is heavily influenced by the fact that we are reviewing district court determinations that it believes to be of some complexity. Indeed, in oral argument, the state relied almost exclusively on the fact that the district court had considered the matter carefully and reached its conclusions following a long trial. This may have caused the majority to fail to examine sufficiently the legal inadequacies of the validation studies pointed out in the preceding sections of this dissent.
At each step in its analysis of job-relatedness, the majority emphasizes that the district court’s determination was not clearly erroneous. However, as the majority recognizes, courts have not discussed the appropriate standard of review for a district court’s ruling on test validation issues. In fact, the standard of review here should be the same as it was in Albemarle and Craig , where the Supreme Court and this court reversed a district court’s determination that a test was valid based on errors in the application of the job-relatedness test similar to those committed by the district court here. At no point did either court suggest that the clearly erroneous test applied.
Although I believe that the district court’s determination of job relatedness should be reversed regardless of the applicable standard of review, the proper standard is de novo. The majority fails to separate errors regarding the application of the various prongs of the Craig test, which involve primarily legal issues, from errors regarding the underlying factual evaluations of expert testimony and statistical significance. The errors here fall in the former category and are therefore subject to de novo review. See United States v. McConney, 728 F.2d 1195, 1201 (9th Cir.1984) (en banc). 2
*598 IV. Conclusion
The defendants failed to meet their burden of demonstrating that the CBEST, with its discriminatory impact, is job related. The studies relied on by the CTC to validate the test did not differentiate among different teaching jobs, and the CTC did not demonstrate by professionally acceptable methods that the test items actually measured the skills identified in the job analysis. Title VII’s requirement that tests be validated properly must be vigilantly enforced to ensure that tests that have a disparate impact on minorities are used only if they are job related for the position in question and if no alternative nondiscriminatory selection device exists. Plaintiffs’ challenge to the CBEST implicates one of the most important objectives of Title VII — preventing employers from erecting arbitrary barriers which interfere with minority access to employment opportunities. Griggs v. Duke Power Co., 401 U.S. 424, 432 , 91 S.Ct. 849 , 28 L.Ed.2d 158 (1971). By erroneously affirming the district court’s decision, we allow the State of California to perpetuate discrimination against qualified minority teachers, who are already seriously underrepresented in the California public school system, and, derivatively, against minority students as well. In the end, all Californians are the victims of the discriminatory tests the majority upholds. I respectfully dissent.
. See Bernard v. Gulf Oil, 841 F.2d 547, 567 (5th Cir.1988) (remanding for specific findings because test for craftsmen positions was not validated for some positions, such as welding and carpentry, even though it was validated for others, such as pipefitting and boilermaking); Walston v. County Sch. Bd., 492 F.2d 919 , 926 (4th Cir.1974) (holding, under a business necessity standard, that for the purposes of validating the use of the National Teacher Examinations a "job analysis for one teaching position (and the appropriate test for it) would not necessarily be suitable for another.”); cf. Williams v. Ford Motor Co., 187 F.3d 533, 536-37, 542 (6th Cir.1999); (holding content-validation study acceptable for unskilled workers at auto plant because workers were not hired for specific unskilled worker positions and were rotated between positions on a regular basis).
. Clady v. County of Los Angeles, cited by the majority for the proposition that we have applied the clear error standard in cases involving test validation, is not to the contrary. In Clady, we applied the clear error standard because that case turned on subsidiary factual issues — primarily, credibility determinations of opposing witnesses — rather than the application of the Craig test to those factual findings. See Clady v. County of Los Angeles, 770 F.2d 1421, 1431 (9th Cir.1985) (noting that the district court rejected the testimony of the appellants' expert, and stating that "[w]hen a trial judge's finding is based on his decision to credit the testimony of one or two or more witnesses, ... that finding, if not internally inconsistent, can virtually never be clear error.” (citation omitted)).
The approach of the Fifth Circuit also does not support the majority’s position. In Bernard v. Gulf Oil, 841 F.2d 547, 567 (5th Cir.1988), the court remanded for specific findings because the district court upheld tests that were not validated for some positions, such as welding and carpentry, even though they were validated for pipefitting and *598 boilermaking. In doing so, the Fifth Circuit did not apply a clear error test. However, after the district court made findings based on expert testimony with respect to each of the particular jobs, the Fifth Circuit employed the clear error standard in reviewing the correctness of those factual findings. Bernard v. Gulf Oil, 890 F.2d 735, 743 (5th Cir.1989). It is this second opinion that the majority cites.