# Petition — James v. Tennessee Valley Authority > Briefs, arguments, decisions, and more. URL: https://www.frixlaw.com/law-library/documents/brief%3Amicro_IA40385011_0773%3A1 ## Record - **Collection:** Supreme Court brief - **Document type:** Petition - **Published:** January 1, 1984 - **Citation:** 465 U.S. 1066 ## Text 83-989 mx , much less consider, the affidavits of those same experts, who pointed out that the court had seriously misunderstood and misapplied their work? 2. Having misunderstood the extra-record works of the three experts, was it error for the court of appeals to reject statistical evidence relating to the class as a whole and to limit evidence of racially biased pay rate disparities only to blacks and whites occupying the same job categories, where that form of analysis would inevitably mask the very discrimination alleged? il PARTIES TO THE PROCEEDINGS The petitioners are William N. James, Robert H. Nash and Louie J. Sheffield, on behalf of themselves and a!l members of the certified class, consisting of all past and present black salary employees represented by the Salary Policy Employee Panel of TVA’s Muscle Shoals, Alabama, Office of Agricultural and Chemical Development, except Schedule A employees, from January 17, 1973, until the present. The respondents are the Tennessee Valley Authority, S. David Freeman, Richard M. Freeman and Charles H. Dean, Jr. ili TABLE OF CONTENTS Page I i OO scscnsnrrosunctisesulectcmabenndteinesaiiabaaandas v IIE MEIN 126nncctenisnseesesncetbdisiunssiisvedededneansinaaaminital 2 NER CUOUUIIOINE a seiicscssiscocneodsonservettergemicvdaapnainal 2 NE NOFOUI oie cscs ccnevessnorsscugpitigpaincsnucecumimunhibaidtsesonizeds 2 INE OE CD COO ooiscsssitcsscrcssclshtcinsscnnerascbichisiniesnmioe 3 I. The Proceedings Below...................... oon jasiebacdindeaes 3 II. Class-Wide Evidence of Discrimination .............. 4 III. The Multiple Regression Analyses ...................... 6 IV. The Petition for Rehearing ...................0:c:ccccccceeee 3 Reasons for Granting the WFit..................:ccccccceceeeeeeeeeeeees 12 I. The Court Should Grant the Writ to Correct a Grave Procedural Irregularity That Deprived Petitioners of Their Due Process Rights........... 12 II. The Court Should Grant the Writ to Correct a Serious Error in the Use of Statistical Evi- dence in an Employment Discrimination Case, to Provide Guidance to the Lower Courts and to Resolve a Conflict Between the CRIES vascssccsonctecsiisnhsulopeaiiitioes socniaaieesiianimie 14 CII ciscsccessinitecsccnesictidiitinelapeinsliccageliaadeanttagiammaias 19 Appendices A. Opinion of the Court of Pea Granting in a Denyi Petition aring and a ‘or Rehearing En 14 F.2d 1066 (I ith Cir. 1983) A-l B. Order of the Court of tion for Leave to mara on Appeal zs iv Order of the Court of Appeals Striking Affi- davits from Petition for Rehearing............... C-1 Opinion of the Court of Appeals, 704 F.2d CEP CAI. BPO cher bintenicviccsrvegnetesPines D-1 Opinion of the District Court on the Class Claims, 528 F. Supp. 862 (N.D. Ala. 1981) E-1 Affidavit of Professor Franklin M. Fisher........ F-1 Affidavit of Professors David C. Baldus and ye EE te” ERR pore ee amie 0 cre G-1 Affidavit of Professors Michael O. Finkelstein SEE UTNE ciinis cciedccabiaspinssnavendacbiornabeinidind H-1 v TABLE OF AUTHORITIES Cases: Carroll v. Sears, Roebuck & Co., 514 F. Supp. 788 (W.D. La. 1981), rev'd in part and aff'd in part, 708 gh Ri ge Le |: ) SE ona a ere Citizens for a Better Environment v. United States EPA, Ge ee Uke CFE Cas BONER D atincnsvediincthcantitioabocenentiiiens Conway v. Chemical Leaman Tank Lines, Inc., 610 F.2d NN 2 8 ARIE AIRES Pies ea Se Tare EEOC vy. Federal Reserve Bank of Richmond, 698 F.2d 633 (4th Cir. 1982), cert. granted sub nom. Cooper v. Federal Reserve Bank of Richmond, 52 U.S.L.W. 3342 (U.S. Oct. 31, 1983) (No. 83-185) oo. eee Eastland v. TVA, 398 F. Supp. 541 (N.D. Ala. 1974), rev'd in part and aff'd in part, 553 F.2d 364 (Sth Cir. ), cert. denied, 434 U.S. 985 (1977) wo.eececeeeeceeeeeeee Eastland v. TVA , 553 F.2d 364 (Sth Cir. ), cert. denied, Co Be as Ce a ee ee eee Eynde, Application of, 480 F.2d 1364 (C.C.P.A. 1973) .. James v. Stockham Valves & Fittings Co., 559 F.2d 310 (5th Cir. 1977), cert. denied, 434 U.S. 1034 (1978) ... Ohio Bell Telephone Co. v. Public Utilities Commission SE Be he CUDSED ssscccentaiteetnectninipeonetacninonne Sanchez v. Standard Brands, Inc., 431 F.2d 455 (5th ICT INIT ‘csincrosiiiduiiinkstabinsdadancqsinseeepeciddahipelinnasssiamiaiacict Sobel v. Yeshiva University, 566 F. Supp. 1166 (S.D.N.Y. 1983) ..........+. sian taliaa gaeahiciadbaatlacacatin Texas Department of Community Affairs v. Burdine, 450 Bic IE UIE Pt shcctictsainicssinnnectareqrocenpinghitinlpinmiiidetindives Trout v. Hidalgo, 517 F. Supp. 873 (D.D.C. 1981), aff'd in part and rev'd in part sub nom. Trout v. Lehman, POD F.2d 1094 (D.C. Cir.), petition for cert. filed, 52 U.S.L.W. 3387 (U.S. Oct. 28, 1983) (No. 83-706) a Trout v. Lehman, 702 F.2d 1094 (D.C. Cir.), petition for cert. filed, 52 U.S.L.W. 3387 (U.S. Oct. 28, 1983) (No. 83-706) 13 13 15 15, 16, 17 13 12 15 i, 15, 16 vi Page Turtle Mountain Band of Chippewa Indians v. United States, 490 F.2d 935 (Ct. Cl. 1974) ......ccccccoscosssssssseceee 13 United States v. De Rose Industries, Inc., 519 F.2d 1066 RENEE ITEP Ta ins sans ctuinlanssadetoashesiiel eceatodiahiniuciaioumen 13 United States v. Greschner, 647 F.2d 740 (7th Cir. PDI: cians dicaiccsaasenntnsn Sais hanee MbecviatiaeMechiedbiddaiibaneetieins 13 United States Postal Service Board of Governors v. Aikens, 103 §.Ct. 1478, 75 L.Ed.2d 403 (1983) .......... 6 Valentino v. United States Postal Service, 674 F.2d 56 SUPT R a E) ROMUPD sis dcesrs odictessccobshchieebdieanesteventianasedsibiassbses 16 Vuyanich v. Republic National Bank of Dallas, 505 F. Supp. 224 (N.D. Tex. 1980), mod. in part, reh. denied, en banc, 521 F. Supp. 656 (N.D. Tex. 1981)... 17 Statutes and Rules: ITN asc niicsdanseionbisassaignidatbavaicibtnkesictniebe 3 CREE TD ss ossich of ocanebannibahiboaiousare ateinsossboeibiion 2 I ass scat nn sind eseiipeapngadeboukvabopseendapabalin 3 ERM MN ne 3 ii It INN clicteriicecncestrscnséaiicombenisamsibebiotestoasesiabeegunmmsiidcben 3 EN I oc suka: cabaccnnssepeteniciiansaieteiaaboleeubibbienen 3 Se es A SPL TIE 3D Psivscessesenvinsaphecotestessesvensestanéviboues 3 A RPO TED vecsercovcvescoddncnnvencasopesssoodsinessiodadons 2 eI SE PDA GD saiccsdatoicvcsosincabbisaccesestnscatecsesbavembbstate 13 Sy ED PHIIICALD sansovicSnopidercenexsupasanibiocubbeteansbiinighnasanian 13 Miscellaneous: D. Baldus & J. Cole, Statistical Proof of Discrimination CESS RAS TORE BOO.) ccnccveriscorstereseinasseioetvessecsstobireteess 6 Finkelstein, The Judicial Reception of Multiple Regres- sion Studies in Race and Sex Discrimination Cases, 80 Colum. L. Rev. 737 (1980) o.....cccccceescesseeceeeeeeseeeees 10 Fisher, Multiple Regression in Legal Proceedings, 80 Colum. L. Rev. 702 (1980) .0.......ccccscecscesseesseescereeeeseeees 6 B. Schlei & P. Grossman, Employment Discrimination BACHE GE. WITS Y : seccicecsictsecdsindctaocthaaintnssctiatanicsticldiso 14 Annual Report of the Director, Administrative Office of the United States Courts (1983) .....c.cccccecceesessesseereereeees 16, 18 ? . A ; | ERS No. 83— IN THE SUPREME CourRT OF THE UNITED STATES OctToBer TERM, 1983 WILLIAM N. JAMES, et al., Petitioners, —against— TENNESSEE VALLEY AUTHORITY, et al., Respondents. PETITION FOR A WRIT OF CERTIORARI TO THE UNITED STATES COURT OF APPEALS FOR THE ELEVENTH CIRCUIT Petitioners William N. James, et al., respectfully pray that a writ of certiorari issue to review the judgment and opinion of the United States Court of Appeals for the Eleventh Circuit entered on May 2, 1983, as modified on rehearing on Septem- ber 12, 1983. 2 OPINIONS BELOW The September 12, 1983, opinion of the Eleventh Circuit granting in part and denying in part the petition for rehearing and denying the suggestion for rehearing en banc is attached as Appendix A and is reported at 714 F.2d 1066. The July 18, 1983, order of the Eleventh Circuit denying petitioners’ motion to supplement the record on appeal is attached as Appendix B and is unreported. The June 27, 1983, order of the Eleventh Circuit striking the affidavits of the experts is attached as Appendix C and is unreported. The May 2, 1983, opinion of the Eleventh Circuit affirming in part and reversing in part is attached as Appendix D and is reported at 704 F.2d 613. The December 15, 1981, opinion of the district court on the class claims is attached as Appendix E and is reported at 528 F. Supp. 862. JURISDICTIONAL STATEMENT The opinion and judgment of the Eleventh Circuit were entered on May 2, 1983. A timely petition for rehearing with a suggestion for rehearing en banc was filed, and it was granted in part and denied in part on September 12, 1983. The jurisdiction of this Court rests on 28 U.S.C. § 1254(1). STATUTE INVOLVED Section 717(a) of Title VII of the Civil Rights Act of 1964, as amended by the Equal Employment Opportunity Act of 1972, 42 U.S.C. § 2000e-16(a), provides in pertinent part: “All personnel actions affecting employees or appli- cants for employment . . . in executive agencies as defined in section 105 of Title 5 (including employees and appli- cants for employment who are paid from nonappropriated funds), . . . shall be made free from any discrimination based on race, color, religion, sex or national origin.” 3 STATEMENT OF THE CASE I. The Proceedings Below This is a class action alleging unlawful and pervasive discrimination by the Tennessee Valley Authority (“TVA”) against blacks in promotion, training, assignment and other conditions of employment.' The jurisdiction of the United States District Court for the Northern District of Alabama was invoked pursuant to 42 U.S.C. § 2000e-5(f)(3); 42 U.S.C. § 1981; 16 U.S.C. § 83lc(b); 28 U.S.C. § 1343(a)(4); 28 U.S.C. § 1361; and 28 U.S.C. § 1331. On December 15, 1981, following a trial on the merits, the district court entered judgment in favor of defendant TVA on all claims.2 On May 2, 1 In addition, individual claims of two plaintiffs who the district court held were not proper class representatives, one alleging dis- crimination in hiring and the other alleging discrimination in training and promotion, were consolidated for trial with the claims of the class and class representatives. Both of those individual plaintiffs were ultimately successful in their individual claims, and they are not petitioners here. 2 The original complaint was filed on May 21, 1973, by Frank L. Eastland. In addition to his individual allegations of discrimination, Eastland’s complaint requested broad injunctive and monetary relief on behalf of a class of employees and applicants for employment at TVA’s Muscle Shoals facilities. On October 13, 1973, an amended complaint added eleven plaintiffs and two defendants, the Salary Policy Employee Panel and the Tennessee Valley Trades and Labor Council, neither of which remains a defendant. On December 31, 1974, and June 5, 1975, the district court granted summary judgment against all but one of the named plaintiffs. The district court held that review of the remaining claims would be limited to the administrative record and that, as a matter of law, a class action could not be maintained by federal sector employees. Eastland v. TVA, 398 F. Supp. 541 (N.D. Ala. 1974). On February 28, 1977, the Fifth Circuit reversed in part and affirmed in part. The Fifth Circuit held that three of the named plairtiffs— Messrs. Eastland, James and Sheffield—were entitled to a trial de novo on their claims and that they could seek to maintain a class action. Eastland v. TVA, 553 F.2d 364 (Sth Cir.), cert. denied, 434 U.S. 985 (1977). 7 4 1983, the Eleventh Circuit affirmed the judgment against the class.3 Petitioners sought rehearing and rehearing en banc. The Eleventh Circuit agreed to modify one footnote of its opinion but otherwise denied the petition for rehearing and suggestion for rehearing en banc. Il. Class-Wide Evidence of Discrimination At trial, petitioners proved discrimination four ways: (1) they proved that TVA used a highly subjective personncl system giving uncontrolled di-cretion to a vir- tually all-white group of managers and proved that the system had in fact been abused by managers to favor the persons they wanted to favor on racial grounds; (2) they presented the testimony of more than 20 black witnesses as to the discrimination against them; (3) they presented qualification-by-qualification pay rate analyses showing that blacks with the same education as whites, the same seniority as whites and even the same combination of education and seniority as whites were assigned to lower paying jobs than those same whites; and (4) they presented multiple regression analyses show- ing that when all qualifications were considered simulta- neously, black race had a substantial and statistically significant negative effect on pay, demonstrating that blacks were assigned to lower paying jobs than whites with identical qualifications. 3 The court of appeals reversed the district court’s judgment in favor of TVA with respect to the claims of the two individual plaintiffs, see note |, supra, finding that they had in fact been victims of unlawful racial discrimination. The court of appeals affirmed the judgment in favor of TVA, however, with respect to all of the class claims and the individual claims of the other plaintiffs. The court of appeals also affirmed the district court’s refusal to certify plaintiff Eastland as a class representative of a class of applicants. 5 The district court rejected essentially all of that evidence.‘ With respect to the multiple regression analyses, which were at the heart of petitioners’ statistical proof at trial and are at the heart of this petition, the district court said only this: “Plaintiffs’ regression analyses also compare salaries of jobs requiring a wide range of skills, education, training, and experience. Once again, the variables selected by the plaintiffs for the regression analysis failed to adequately reflect job comparability. Plaintiffs’ analyses can be given little weight in light of the many important variables excluded. Furthermore, the plaintiffs’ data cannot be used to establish a prima facie case of either disparate treatment or disparate impact.”” Appendix E at E-27.5 4 With respect to the subjectivity of the personnel system as a factor contributing to discrimination, the court ruled that in order to accept that argument it would have to “establish a presumption that left to their own devices whites will, at least -dnconsciously, dis- criminate against blacks”, a presumption it was unwilling to establish. Appendix E at E-32. In rejecting the anecdotal evidence of dis- crimination, the district court found only that “plaintiffs failed to show by testimony at trial or by other evidence presented to the court a single valid claim of any black who should have been promoted and was not”. Appendix E at E-18. The district court also rejected the “qualification-by-qualification” statistical analyses, because they did not establish that “blacks in the same kind and level of work were paid any less than whites” (a claim, incidentally, that petitioners never made, since this was not an equal pay case) and because “these comparisons ignored important differences in skills, education, train- ing, and experience”. Appendix E at E-27. Despite that finding, however, neither the district court nor TVA ever identified a single qualification that was not later analyzed in petitioner’s evidence. None of the factors suggested by TVA ever changed the result of petitioners’ showing of discrimination. ‘5 Although the trial court held that petitioners had failed to establish a “prima facie” case, it reached this conclusion only after a full trial on the merits. To the extent that respondents’ evidence purporting to discredit petitioners’ statistical evidence might be char- acterized as “rebuttal” evidence, see Appendix D at D-7, the district court was in error in considering the question of whether a prima facie case was established. Under the rule of United States Postal Service 6 The court of appeals affirmed all of the district court’s findings and conclusions with respect to class-wide discrimination. Most of the court of appeals’ opinion was devoted to the multiple regression analyses. Citing an article published by Professor Franklin M. Fisher in 1980, Multiple Regression in Legal Proceedings, 80 Colum. L. Rev. 702 (1980), and a supplement to a treatise published by Professors David C. Baldus and James W. L. Cole in 1982, Statistical Proof of Discrimination (1980 and 1982 Supp.), the court held that petitioners had not established a sufficient theoretical founda- tion for their multiple regression model and had failed to “control for job category”. Appendix D at D-15 to D-19.6 III. The Multiple Regression Analyses To show that blacks disproportionately occupy lower pay- ing jobs than similarly qualified whites, petitioners offered multiple regression analyses that measured the effect each of several factors had on pay rates of employees similarly situated with respect to the other factors in the analysis. The factors considered were age, length of service, non-technical degree, Board of Governors v. Aikens, 103 S.Ct. 1478, 75 L.Ed.2d 403 (1983), the proper inquiry flowing from such a characterization is whether petitioners proved discrimination by a preponderance of the evidence. Whether respondents’ evidence is characterized as rebuttal evidence or—as botii the district court and the court of appeals treated it—as evidence going to the sufficiency of petitioners’ prima facie case, petitioners’ argument on the merits leads to the same result. If respondents’ st*tistical evidence is treated as rebuttal evidence, it is unsound, and petitioners have carried their ultimate burden of persuasion under Aikens. If respondents’ evidence is not treated as rebuttal evidence, it is still inadequate to cast doubt on petitioners’ prima facie case, which stands unrebutted and thus of itself satisfied petitioners’ ultimate burden of persuasion, Texas Community Affairs v. Burdine, 450 U.S. 248, 254 (1981). 6 As we show below, those two grounds are simply different facets of the same issue. See p. 9, infra. 7 technical degree, and race. An example of the result of that analysis for one of the years, 1975, is as follows: é Effect on Statistically Variable Salary Significant Bach year Of O96 ...:cccccsccssssescseeseess $ 167 Yes Each year Of Service ................000 +) Yes Non-Technical Degree ................ + 5,446 Yes Technical Degree .............cccssseres +11,014 Yes POT Wee sccovutcenn: ---abdscaseiiios - 3,838 Yes Thus, with all of these qualifications held constant, one can predict at a statistically significant level that a black employee with any combination of those qualifications would be assigned to a job at TVA paying $3,838 less than a white employee with the same combination of qualifications. TA criticized that analysis on three grounds. It argued that petitioners used the wrong factors, that the inclusion of the F (lowest paid) and M (highest paid) schedules might have distorted the results and that it was improper to do a single multiple regression analysis for the entire class—that instead petitioners should have run separate regressions for each of the five job categories in issue (i.e., “controlled for job category” ). TVA’s expert, a psychologist, then performed his own multiple regression analyses, one for each job category, testing for the effects of a different set of factors. He admitted that even his own analysis showed a statistically significant race effect on at least one schedule, E, which was one of two schedules at the heart of petitioners’ discrimination claim.’ In 7Schedule E consists primarily of laboratory aides, many of whom have college degrees in the sciences. TVA’s analysis of Schedule E showed that TVA had assigned blacks to jobs paying $2,254 less than comparably qualified whites. The other schedule of greatest concern to petitioners was Schedule D, consisting primarily of laboratory analysts, most of whom have science degrees. At the class certification stage of the case, TVA argued for a class consisting only of Schedules D and E, and TVA performed a regression analysis on addition, he was unable to do a regression analysis for another job schedule, A, since TVA had never had a black employee on it. For the other schedules, B and F (a schedule consisting almost entirely of blacks), he found no significant race effect in favor of whites. In rebuttal, petitioners prepared new multiple regression analyses in order to meet TVA’s hypothetical challenges. First, petitioners used TVA’s suggested variables, and the results continued to show significant pay rate disparities in favor of whites when the class as a whole was considered. An example of this result for 1979 appears as follows: Effect on Statistically a_i Salary _ ___J Each year of service.............. ee Fo fs yes Each year of education. ............ + 756.41 yes Each year of age ...................... + 177.27 yes Physical science degree............ + 7,348.44 yes Natural science degree ............ + 6,931.64 yes Ph.D in any field or bachelor’s degree in chemical engi- i itlbicetesstindercntvecseceniens + 7,361.32 yes Engineering science degree ..... + 4,925.74 yes Social science degree ............... + 2,784.68 yes dat selideameideneseccndde -—3,019.50 yes Petitioners then excluded, in turn, the F schedule and the M schedule, and each time the results were the same—statistically significant disparities in favor of whites. ; The court of appeals did not question the factors tested by petitioners either in their own analyses or in the rebuttal analyses. Rather, it held that petitioners should have estab- lished a “theoretical foundation” for their multiple regression those two schedules together. If proper statistical methods are used, see Appendix F ¥ 18, TVA’s own analysis establishes that black race had a statistically significant effect on pay rates in the two schedules combined (.0453 probability of finding the observed race effect by chance, with .05 being statistically significant), as well as in Schedule E alone (.0273 probability). 9 analyses and should have “controlled for job category”. Citing and quoting from the work by Professors Baldus and Cole, the court said that “a regression should account for the ‘interaction’ between the kind of job and the other variables”. Specifically, the court observed that, for example, “length of service” may have a greater effect on salary in some schedules than in others. Appendix D at D-17 to D-18. As a result, the court concluded: “Given the weakness of the theoretical foundation and the failure to control for job category, the district court did not err in determining that Eastland’s regressions were insufficient to establish a prima facie case.” Appendix D at D-19.8 The two bases on which the court of appeals relied in rejecting petitioners’ multiple regression analyses—the sup- posed lack of a “theoretical foundation” and the failure to control for job category—are in fact one and the same. Since petitioners’ rebuttal analyses considered all factors suggested by respondents, they differed from respondents’ analyses only in their “failure to control for job category”, and any alleged flaw in the theoretical foundation for petitioners’ statistical model must reside in that difference. See Appendix D at D-17. Consequently, the court’s rejection of petitioners’ evidence rests entirely on its understanding of statistical science as gleaned from the works of Professors Fisher, Baldus and Cole. IV. The Petition for Rehearing Petitioners sought a rehearing and a rehearing en banc and, in support of their petition, submitted the affidavits of the same three experts relied on by the court, Professors Fisher, Baldus and Cole, as well as the affidavit of Professor Michael O. ®Since a full plenary trial was held, the court’s emphasis on whether petitioners had established a prima facie case is subject to the same possible objection as the district court’s similar analysis. If this was crror, however, it should have no effect on the proper resolution of this case, since petitioners’ evidence both establishes a prima facie case and proves discrimination by a preponderance of the evidence. See note 5, supra. 10 Finkelstein (the avthor of a companion work to Professor Fisher’s 9) and Professor Bruce Levin, a colleague of Professor Finkelstein. (Copies of these affidavits are attached to this petition as Appendices F, G and H, respectively.) Each of those experts concluded that the court of appeals had made a serious analytical error that affected the outcome of the case. Their affidavits establish three central points: —It is statistically unsound, as the court of appeals required, to test for discrimination in job assignment and promotion by multiple regressions that “control for job category”, since such a control would mask the dis- crimination alleged, particularly where, as here, it would result in extremely small sample sizes. —Even if it were correct to do so as a theoretical matter, it was unnecessary to do so here, since TVA’s own regression analyses demonstrate the error of the court’s hypothesis that the effect of length of service on pay rate might be different for the different job schedules. '° —The record clearly shows that even when separate job categories are considered, race continues to have a significant effect on Schedules D and E."! 9 Finkelstein, The Judicial Reception of Multiple Regression Studies in Race and Sex Discrimination Cases, 80 Colum. L. Rev. 737 (1980). 10 TVA’s regression analyses showed the following effects of length of service on salary for each job schedule: A—unknown; B—$220.01; D—$227.82; E—$174.72; F—unknown; M—$212.99. ( Defendants’ Exhibit (“DX”) 201.) These figures clearly belie the court’s objection to petitioners’ methodology. "7 “( B]lack race is such a significant negative factor affecting job placement among equally qualified black and white employ- ees on Schedules, D and E, that TVA’s own results in DX201—showing that blacks on Schedule D earn $1,220, and blacks on Schedule E earn $2,250, less than their similarly qualified white counterparts—could have occurred by chance only 22 times in every 10,000 cases.” Appendix F ¥ 19. jet Approximately a month later, the court of appeals, sua sponte, struck the affidavits: i “The Appellants have filed a petition for rehearing and suggestion for rehearing en banc and, in support, without leave of court have attached affidavits of statistical experts who disagree with various parts of the court’s opinion. The court will not consider these ex parte, post-decision state- ments. The court sua sponte ORDERS that they are stricken from the petition, and no opposing affidavits need be filed.” Appendix C. Immediately thereafter, petitioners moved for leave to supplement the record on appeal with the stricken affidavits, but that motion was denied. When the court of appeals ruled on the merits of the petition for rehearing, it modified one footnote in response to a suggestion by the amicus NAACP Legal Defense and Educational Fund, Inc. and denied the rest of the petition without comment. '2 “[T]hese observations taken together suggest that the re- sults of the defendants’ separate analyses of Schedules D and E tend to support conclusions not substantially different from those drawn from the plaintiffs’ analysis of the pooled schedules, at least to the degree that the pooled analysis indicates the presence of disparate treatment in Schedules D and E.” Appendix G 4 21. “[WlJe must reject the hypothesis that race was not an influence on salary in either schedule.” Appendix H 4 10; see also q 12. 12 The court modified its rationale for affirming the district court’s exclusion of initial assignment claims from the class action. As modified, the court’s opinion held that initial assignment claims should not be considered because they were not “like or related to” the claims made in the administrative complaints of the class repre- sentatives. The court noted that none of the class representatives could have raised an initial assignment claim because all were hired before Title VII became applicable to the federal government, and that from the record, it did not appear that the TVA had considered discrimination in initial assignments in investigating the claims of the class representatives. The court of appeals’ rejection of initial assignment claims was wrong for several reasons. 12 REASONS FOR GRANTING THE WRIT I. The Court Should Grant the Writ to Correct a Grave Procedural Irregularity that Deprived Petitioners of their Due Process Rights The court of appeals’ misplaced reliance on material outside the record, coupled with its refusal to allow the parties an opportunity to correct the error, constituted a grave procedural irregularity with far-reaching implications for em- ployment discrimination cases and other cases as well. By refusing even to receive, much less consider, the affidavits of the very experts on whom it relied in its erroneous statistical analysis, the court of appeals has deprived petitioners of their fundamental rights to due process. First, the court of appeals committed plain and dispositive error by relying on material not considered by the trial court. First, the district court had rejected the initial assignment claims for a reason entirely different from that of the court of appeals—on the ground that they were “applicant”, rather than “employee”, claims. The court of appeals essentially made an independent finding of fact unrelated to any findings of the district court. Second, the court of appeals misapplied the “like or related” test. As the court of appeals stated the test, it encompasses “the “scope” of the EEOC investigation which can reasonably be expected to grow out of the charge of discrimination.”” Appendix A at A-2 (quoting Sanchez v. Standard Brands, Inc., 431 F.2d 455 (Sth Cir. 1970)) (emphasis added). It is irrelevant to the application of this test that the TVA may not have investigated discrimination in initial assign- ments—may not, in other words, have done all it could “reasonably be expected” to do. Finally, the court of appeals noted that pervasiveness of a discriminatory practice may support a broader application of the “like or related” rule. Accordingly, if this Court should reverse the court of appeals’ rejection of petitioners’ statistical evidence, finding, in effect, that discrimination pervasive enough to warrant judgment for the class existed or may have existed, the court of appeals should at least be required to reconsider whether a broader application of the “like or related” rule is appropriate in this case. 13 Turtle Mountain Band of Chippewa Indians v. United States, 490 F.2d 935, 945 (Ct. Cl. 1974). More fundamentally, whether at trial or on appeal, courts may take judicial notice only of facts that are not reasonably subject to dispute. Citizens for a Better Environment v. United States EPA, 649 F.2d 522, 526 n.6 (7th Cir. 1981); United States v. Greschner, 647 F.2d 740, 742 n.2 (7th Cir. 1981); United States v. De Rose In- dustries, Inc., 519 F.2d 1066, 1068 (Sth Cir. 1975); Application of Eynde, 480 F.2d 1364, 1370 (C.C.P.A. 1973). Accordingly, both the Federal Rules of Evidence and the Due Process Clause require that a court taking judicial notice of a fact give the parties an opportunity to refute it. Fed. R. Evid. 201(e); Ohio Bell Telephone Co. v. Public Utilities Commission of Ohio, 301 U.S. 292 (1937); Conway v. Chemical Leaman Tank Lines, Inc., 610 F.2d 360, 365 n.5 (Sth Cir. 1980).13 In the present case, this right was not an empty formality but a critical necessity determining the outcome. Because of its unguided inference from extra-record expert evidence, the court of ap- peals judicially noticed not a fact but an error, and as the affidavits of Professors Baldus, Cole, Fisher, Finkelstein and Levin establish, petitioners—if given an opportunity to be heard—could have prevented the error. Petitioners sought such an opportunity to be heard, but were turned away. As the use of statistical evidence in employment dis- crimination cases increases, the use—and misunder- standing—by courts of expert treatises without illumination by the parties will pose grave dangers to the ability of litigants to ensure that their evidence is given proper weight—or is even understood—by the courts. Quite apart from employment discrimination cases, litigation raising complex technical, mathematical and statistical issues is now a commonplace in the federal courts. A precedent authorizing courts to go outside the record, to misconstrue the extra-record materials relied on and then to refuse correction from the authors of those very materials poses the danger of generating many more plainly incorrect decisions. 13 Cf. Fed. R. Evid. 706(a) (expert appointed by court subject to cross-examination by each party). 14 We do not contend that courts of appeals should be quick to permit appellants to supplement the record on appeal with affidavits such as the ones we offered. We fully recognize that it is highly unusual even to seek to supplement the record on appeal. But where the court of appeals has itself gone outside the record to find expert opinion on which to rely, especially when its misreading of that opinion is dispositive, it is fundamentally unfair and improper for the court not to permit the aggrieved purty to submit sworn statements of those very same experts in order t> show that the court misunderstood them in the first place. It may be that the court was concerned about the procedu- ral propriety of inducing a battle of affidavits on appeal, but in that case there was a simple remedy. The court could have, as petitioners suggested, remanded the case to the district court for the taking of additional evidence and for additional findings of fact. Inexplicably, it refused to do so. II. The Court Should Grant the Writ to Correct a Serious Error in the Use of Statistical Evidence in an Employment Discrimination Case, to Provide Guidance to the Lower Courts and to Resolve a Conflict Between the Circuits. The grave procedural irregularity committed by the court of appeals led it into a serious and outcome-determinative error on the merits, which of itself warrants granting the writ. Statistics generally, and multiple regression analyses in particular, are an important—probably indispensable—evi- dentiary tool for Title VII litigants. See B. Schlei & P. Grossman, Employment Discrimination Law 1342-43 & n.179 (2d ed. 1983). Yet they have generated substantial confusion in the lower courts—of which the plain error of the court of appeals in the present case is but one compelling example. The question of when statistical evidence must account for differ- ences in job categories is of particular importance, having arisen in an increasingly large number of cases in recent years. See, 15 e.g., EEOC v. Federal Reserve Bank of Richmond, 698 F.2d 633, 655-57 (4th Cir. 1982), cert. granted sub nom. Cooper v. Federal Reserve Bank of Ri hmond, 52 U.S.L.W. 3342 (US. Oct. 31, 1983) (No. 83-185); James v. Stockham Valves & Fittings Co., 559 F.2d 310, 331-32 (Sth Cir. 1977), cert. denied, 434 U.S. 1034 (1978); Sobel v. Yeshiva University, 566 F. Supp. 1166, 1178-80 (S.D.N.Y. 1983); Carroll v. Sears, Roebuck & Co., 514 F. Supp. 788, 816 (W.D. La. 1981), rev’d in part and aff'd in part, 708 F.2d 183 (Sth Cir. 1983). The guidance of this Court is badly needed by the lower courts. Currently before this Court is the petition of the Solicitor General of the United States for a writ of certiorari in Trout v. Lehman, 702 F.2d 1094 (D.C. Cir. 1983), petition for cert. filed, 52 U.S.L.W. 3387 (U.S. Oct. 28, 1983) (No. 83-706). The Solicitor General’s petition asks this Court to review a decision that the inclusion in multiple regression analyses of nonactionable pre-1972 federal government personnel decisions does not render the analyses inadequate to prove actionable employment discrimination. Although the question in Trout is not identical to that presented here, the two are very closely related: Both concern the standards a multiple regression analysis must meet to qualify as evidence of employment discrimination. If this Court should choose to review the decision of the D.C. Circu‘t in No. 83-706, to evaluate the probative value of a tainted regression analysis, it should at the same time consider the present case, involving untainted and highly probative statistical analyses that were rejected below on the basis of unnecessary and meaningless requirements. Regardless of the Trout case, the writ should issue in the present case to correct the plain error and injustice of the decisions below, which conflict with well reasoned authority in another Circuit. The court of appeals was wrong on the merits, and its error is outcome determinative. The works in question do not lead to the conclusion that the court of appeals drew from them; properly understood, they lead to the opposite conclusion. No other court of which we are aware has made § 16 the same error’, but in the lizht of the opiaion below from a respected court that handles a great number of Title VII cases, there is a danger that others now will. And because the error is such a fundamental one, affecting the use of multiple regression analysis in every single employment discrimination case in which more than one type of job is in issue, there is a serious risk that large numbers of meritorious cases will be jeopardized because of the court’s opinion. (For the twelve months ending on June 30, 1983, 9,097 new employment discrimination cases were filed in the district courts. During the same twelve months, 1,418 employment discrimination cases were appealed to the courts of appeals. Annual Report of the Director, Administrative Office of the United States Courts, Appendix I, pp. A-4, A-20 (1983).) In view of the nature of the error below, it is not surprising that it conflicts with the rule in the Fifth Circuit. In James v. Stockham Valves & Fittings Co., supra, the Fifth Circuit held it was error, in an employment discrimination case involving discrimination in promotion, to give weight to a regression analysis that controlled for job class. The defendant’s regres- sion analysis in Stockham Valves had treated “skill level” 14 The court of appeals stated that one other case, Valentino v. United States Postal Service, 674 F.2d 56 (D.C. Cir. 1982), reached a similar conclusion. But that is incorrect. Valentino holds that where a class definition includes widely disparate jobs, the regression model must include sufficiently detailed variables to account for the different minimum objective job qualifications required for the various jobs. Id. at 61, 70-71. See Trout v. Lehman, supra. Valentino does not require class plaintiffs to perform separate regressions for each job category within the class. See Trout v. Hidalgo, 517 F. Supp. 873, 883 (D.D.C. 1981), aff'd in relevant part sub nom. Trout v. Lehman, 702 F.2d 1094 (D.C. Cir.), petition for cert. filed, 52 U.S.L.W. 3387 (U.S Oct. 28, 1983) (No. 83-706). Here, petitioners’ variables did in fact account for all of the minimum objective job qualifications identified by TVA or any of its witnesses. Ss 17 —derived solely from the employee's job class—as a vari- able.'5 The court of appeals said: “ “Skill level’ was derived from an employee's job class; he had ‘skill’ only if he worked in a job with a rating between JC 10 and 13. The systematic exclusion of blacks from promotion and training opportunities for such jobs, as is alleged here, will automatically produce no black employ- ees with ‘skill level’. A regression analysis defining ‘skill level’ in that way thus may confirm the existence of employment discrimination practices that result in higher earnings for whites.” 559 F.2d at 332. See also Carroll, supra, 514 F. Supp. at 816; Vuyanich v. Republic National Bank of Dallas, 505 F. Supp. 224, 305 n.95 (N.D. Tex. 1980), mod. in part, reh. denied, en banc, 521 F. Supp. 656 (N.D. Tex. 1981). A hypothetical éxample may illustrate the correctness of the Fifth Circuit rule and the error of the court of appeals in this case: Assume that a black with a degree in chemistry is hired on the E schedule as an aide, a position that does not require a degree in chemistry and in which no white person with a degree in chemistry is employed. Because of his race, and despite his credentials and strong performance in his job, he is repeatedly denied promotion to the D schedule, the schedule in which similarly qualified whites are employed, and remains in the E 1S The statistical methodology rejected by the Fifth Circuit in Stockham Valves differs, technically, from that required by the Eleventh Circuit in the present case, but both are infected with the same infirmity. In the present case respondents urge that petitioners must control for job category by performing separate regression analyses for each job category, while in Stockham Valves, the defendant included job category as a variable in its one regression analysis. Both are methods of “controlling” for job category, see Vuyanich v. Republic National Bank of Dallas, 505 F. Supp. 224, 305 n.95 (N.D. Tex. 1980), mod. in part, reh. denied, en banc, 521 F. Supp. 656 (N.D. Tex. 1981), and in a case involving discrimination in assignments and promotions, both have the same unacceptable effect: allowing the defendant to remove from the evidence before the court precisely the effect to be measured. schedule for more than 10 years. Although he may be paid as well as others in the E schedule, he does not belong there, and he is in fact paid less than comparably qualified whites. A regression analysis that controlled for job category would not show any pay rate disparity based on race. It would show only that he is paid as well as other aides, masking the very discrimination he has suffered.'® The Fifth Circuit understands the vice in controlling for job category in cases involving assignments and promotions. The court of appeals in the present case clearly did not. Title VII cases are enormously expensive for the litigants (most of whom cannot alone bear the expense) and time consuming for the courts. As the conflict between the Fifth and Eleventh Circuits here demonstrates, there is confusion among the lower courts—and among litigants as well—as to what types of evidentiary showings are sufficient to establish dis- crimination. If they are entitled to nothing else, the litigants are entitled to predictability. That, unfortunately, they do not have today. Rather, they are repeatedly being faced with changing standards of proof, changes, such as those here, that often make little apparent sense. Clear direction from this Court is needed, now more than ever. Without such direction, there is a clear danger that private enforcement of Title VII will substantially diminish, that private attorneys general (who now account for more than 98% of all new employment discrimination cases'7 ) will find it impossible to continue to bear the heavy costs of making the required statistical showings without knowing in advance how those showings should be structured sO as to succeed. f6 This example is not wholly hypothetical. It closely parallels the career of petitioner Sheffield. 17 Of the 9,097 new employment discrimination cases filed in the district courts during the 12 months ending on June 30, 1983, only 170 were instituted by the government. Of the 1,418 employment discrimination cases appealed to the courts of a during the same period, only 51 involved appeals by the Gove t. Annual of the Director, Administrative Office of the United States Courts, Appendix I, pp. A-4, A-20 (1983). 19 CONCLUSION In view of the foregoing, a writ of certiorari should issue to the United States Court of Appeals for the Eleventh Circuit, so that this Court may resolve the important issues raised in this petition. December 12, 1983. Respectfully submitted, PauL C. SAUNDERS, Counsel of Record, One Chase Manhattan Plaza, 57th Floor, New York, N. Y. 10005 (212) 422-3000 WILLIAM L. ROBINSON, RICHARD T. SEYMOUR, STEPHEN L. Spitz, Lawyers’ Committee for Civil Rights Under Law, 733 Fifteenth Street, N.W., Washington, D.C. 20005 (202) 628-6700 SUSAN W. REEVES, Reeves and Still, 400 Commerce Center, 2027 First Avenue North, Birmingham, Alabama 35203 ALAN E. Kraus, (205) 322-6631 Rosin C. LANDIS, DaNIEL J. LEFFELL, CRAVATH, SWAINE & Moore, One Chase Manhattan Plaza, New York, N. Y. !0005 (212) 422-3000 Attorneys for Petitioners. Of Counsel. OPINION OF THE COURT OF APPEALS GRANTING IN PART AND DENYING IN PART a PETITION FOR REHEARING AND ae DENYING SUGGESTION FOR REHEARING EN BANC REPORTED AT 714 F.2d 1066 (11th Cir. 1983) A-l APPENDIX A UNITED STATES CouRT OF APPEALS For THE ELEVENTH CIRCUIT FRANK L. EASTLAND, Individually, et a/., Plaintiffs-Appellants, — against — TENNESSEE VALLEY AUTHORITY, ef al., Defendants-Appellees. No. 82-7008 Sept. 12, 1983 Appeals from the United States District Court for the Northern District of Alabama, ON PETITION FOR REHEARING AND SUGGESTION FOR REHEARING EN BANC (Opinion May 2, 1983, 11 Cir., 1983, 704 F.2d 613). Before GopBOLD, Chief Judge, Fay and SmitH*, Circuit. Judges. PER CURIAM: The amicus brief filed by the NAACP Legal Defense and Educational Fund, Inc., in support of the petition for rehear- ing/rehearing en banc questions the correctness of footnote 9 of our opinion, 704 F.2d at 620. * Honorable Edward S. Smith, U.S. Circuit Judge for the Federal Circuit, sitting by designation. A-2 Footnote 9 is withdrawn and the following is substituted for it: 9 The district court refused to consider the initial assign- ment claims because they were “applicant claims” ex- cluded by the Fifth Circuit’s opinion in Eastland I, 553 F.2d 364. Eastland I held that “a class action may only be maintained if the requirements of Rule 23 .. . can be complied with, and the only issues that may be raised are those issues that were raised by the representative parties in their administrative complaints, together with those issues that may reasonably be expected to grow out of the administrative investigation of their claims.” Jd. at 372. Eastland argues that initial assignment claims were implicit in the administrative complaints of both Nash and Sheffield and therefore the issue was properly before the court. “* The starting point for determining the permissible scope of the judicial complaint is the EEOC charge and investigation. Evans v. U.S. Pipe & Foundry Co., 696 F.2d 925, 927 (Cir. 1983). In Sanchez v. Standard Brands, Inc., 431 F.2d 455 (Sth Cir. 1970), the Fifth Circuit held “the allegations in a judicial complaint filed pursuant to Title VII ‘may encompass any kind of discrimination like or related to the allegations contained in the charge and growing out of such allegation during the pendency of the case before the Commission.’” Jd. at 466 (quoting King v. Georgia Power Co., 295 F. Supp. 943, 947 (N.D. Ga. 1968) (emphasis added)). Under the “like or related” rule “the ‘scope’ of the judicial complaint is limited to the ‘scope’ of the EEOC investigation which can reasonably be expected to grow out of the charge of discrimination”. Jd. In the instant case none of the class representatives raised an initial assignment claim in his administrative complaint. The class representatives were not in a position to raise such claims because all received their initial assignments years before Title VII was made applicable to TVA. Our review of the record reveals no indication that the administrative investigations considered either dis- crimination in initial assignments or the contention that A-3 widespread discrimination resulted from TVA’s general policy of delegating excessive subjective discretion to its supervisory staff. The “like or related” rule has been subject to varied interpretations. See Evans, 696 F.2d 928-29. Although the pervasiveness of a discriminatory practice or other compelling circumstances may support a broader appli- cation of the rule, on the facts of this case the district court’s refusal to consider the initial assignment claims was not error. In all other respects, the petition for rehearing is DENIED. No member of this panel nor judge in regular active service on the court having requested that the court be polled on rehearing en banc (Rule 35, Federal Rules of Appellate Procedure; Eleventh Circuit Rule 26), the suggestion for re- hearing en banc is DENIED. Big : b t 4 . at ee E-31 been objective. Neither has a single case been shown of an abuse of either of the three review systems in use or of failure of any of the three to function fairly and as intended. It is interesting to note that plaintiffs never contended that their job descriptions or those of the class were subjective or discriminatory or ‘unlawful in their pretrial contentions, even though these job descriptions are the keystone of the TVA personnel system. Furthermore, the plaintiffs’ expert witness, Dr. Ireland, never discussed the job descriptions. Plaintiffs’ claim regarding a subjective “good old boy” network to fill management and supervisory positions is an attempt to create a class claim based upon discriminatory treatment in promotion to top management positions, without having to prove the requisite discriminatory animus, which proof does not exist in the record. Moreover, as stated earlier, promotions to the M schedule are not a part of this case. ( See, orders of October 31, 1979, and July 8, 1980, as amended on July 10, 1980.) Eighty-three percent (83%) of the promotions at OACD are brought about by reclassification. It is difficult for the court to overlook the conflict of interest created by the plaintiffs’ contention that TVA’s employee preference policy is evidence of discrimination, since the class is composed of TVA employ- ees who necessarily benefit from the policy they attack. The court also notes that subjectivity itself is not against the law; a subjective reason for a personnel action that does not hide discrimination satisfies an employer’s burden. See, Wil- kins v. University of Houston, 654 F.2d 388 at 400 (Sth Cir. 1981); Ramirez v. Hofheinz, 619 F.2d 442, 446 ( 5th Cir. 1980). Bay v. Goodyear Tire and Rubber Co., 23 EPD ¥ 30,903 at 15,674 (S.D. Tex. 1980), states: Subjective reasons for the failure to promote Plaintiffs should be given cg — prety tiny tes acon AN discrimination, for courts recognize that employment deci sions often cannot realistically be made on the basis of objective standards alone. Id. An employer's decision may properly be based on subjective factors. E-32 Most importantly, this Court must consider the result of Goodyear’s promotional practices. Regardless of whether objective or subjective factors were utilized, did they result in an adverse treatment or impact upon Black employees? The answer here is “no.” Accordingly, the mere presence of subjective procedures in the promotional process is not discriminatory. Jenkins v. Caddo-Bossier Assn. for Re- tarded Children, 570 F.2d 1227 (Sth Cir. 1978). (Citations omitted). Again, it should be pointed out that no satisfactory proof was ever offered to the court that any particular black should have been promoted and was not (or was otherwise mistreated in a manner within the issues herein). If plaintiffs have a class claim, surely they can find one good example of discriminatory mistreatment of a black employee. See, Wilkins v. University of Houston, at 400-01 (Sth Cir. Aug. 28, 1981). Thus, plaintiffs have not met their burden of proof to show that it is the standard operating procedure of OACD managers to treat blacks differently and unfavorably in promo- tions, reclassifications, training, work assignments, and reduc- tion in force, on account of their race. Plaintiffs attempt to prevail by alleging that white TVA managers make subjective decisions, especially in the handling of work assignments that might lead to faster professional development. In order for the court to find for the plaintiffs on this rationale, the court would have to “establish a presumption that left to their own devices whites will, at least unconsciously, discriminate against blacks.” Grant v. Bethlehem Steel Corp., 22 FEP Cases 687, 689 S.D.N.Y. 1978). This presumption is not the jaw; in fact, plaintiffs’ present chief counsel established such in Grant. In assessing the statistical evidence, the Supreme Court has dictated that plaintiffs cannot carry their burden of proof by use of statistics unless their statistics are both relevant to their contentiox.s and account for no reasonable interpretation other than the conclusion they seek to draw. New York City Transit Authority v. Beazer, 440 U.S. 568, 582-87, 99 S. Ct. 1355, 1364- 66, 59 L. Ed. 2d 587, 600-604 (1979). As discussed infra, the - FO. ae E-33 average salary statistics presented by plaintiffs are lower for blacks than for whites due to the disproportionately large number of blacks hired (through affirmative action) in recent years at entry-level salaries to increase the number of black employees at TVA. Evidence was presented to the court that TVA was quite aware of and sensitive to the desirability of an aggressive affirmative action program. Regardless, defendants have aptly pointed out to the court that according to the plaintiffs’ proof of average salaries over the relevant time period, on or after January 17, 1973, the average salary of blacks at OACD has increased by 88.2 percent, while the average salary of whites has increased by 54.5 percent. In conclusion, because plaintiffs have failed to provide evidence to the court of a prima facie case of disparate treatment or disparate impact, or even an example of a single black who should have been promoted and was not, much less appropriate statistical data on the pool of qualified blacks that could be promoted, this court finds for the defendants. An order consistent with this memorandum opinion will be entered contemporaneously herewith. F-1 APPENDIX F Unrtep STaTes Court OF APPEALS For THE ELEVENTH CIRCUIT FRANK L. EASTLAND, et al., Plaintiffs-Appellants, — against — TENNESSEE VALLEY AUTHORITY, et al., Defendants-Appellees. COMMONWEALTH OF MASSACHUSETTS, a: COUNTY OF MIDDLESEX, ik FRANKLIN M. FisHeR, being duly sworn, deposes and says: 1. I am Professor of Economics at the Massachusetts Institute of Technology, where for a number of years I have taught courses in statistics and econometrics to graduate stu- dents in economics. 2. I am the author of “Multiple Regression in Legal Proceedings”, 80 Col. L. Rev. 702 (1980), cited in the May 2, 1983, decision in this case. 3. I have written, alone or with others, more than 90 published articles and 9 books in the fields of economics, econometric theory and applied econometrics. Econometrics is the application of statistical and quantitative techniques, in- cluding regression analysis, to economic phenomena. From 1968 to 1977 I was the Editor of Econometrica, the journal of the Econometric Society, and in 1979, I was the President of the F-2 A. MY CONCLUSIONS 4. I have reviewed plaintiffs’ exhibit 515, which summa- rizes the results of a large number of statistical tests, and the multiple regression analyses presented by both plaintiffs and defendants. My conclusion is th». the materials of both sides make a strong and statistical’y sound showing that black employees have been assigned . jobs paying substantially less than the jobs to which whites with the identical qualifications have been assigned. The possibi'ity that these showings can be attributed to chance or to any disto:tion caused by aggregating data for different schedules or jobs is so remote that it can be discarded. The reasons for these conclusions are set forth below. 5. The court criticized plaintiffs’ pay rate analyses because “they fail to take into account the fact that a number of factors operate simultaneously to influence the amount of salary an OACD employee receives.” ( Decision, pp. 2872-73). The most appropriate type of analysis capable of taking these factors into account simultaneously is multiple regression. 6. When, as is apparent in this case, an initial glance at pay rates for blacks and whites indicates that there are signifi- cant pay rate disparities in favor of whites, it is appropriate —indeed required—to make more refined analyses of the data to see whether those disparities can be explained by factors other than race. Here, those further refinements were obviously made and they confirm that black race continued to be a significant . negative factor on pay rate. Similarly, where statistically significant results are found in relatively small samples, findings of significance in larger samples—such as Data Sets A, B, MSEO and Four Cities, which include the group corresponding to the class definition but are broader than it—demonstrate that the findings in the small samples were not flukes. F-3 B. THE THEORETICAL FOUNDATIONS OF THE REGRESSION ANALYSES RECEIVED IN EVIDENCE 1. Factors Other Than Job Category and Job Schedule 7. Regressions are used in many different types of situ- ations (see my article at 703-05), and it is important to develop a theory as to which variables are important before performing the regressions. The theory need not be elaborate, and need not be based on the results of a preliminary investigation; one must merely determine on some reasonable basis “those vari- ables that one thinks are actually important” and begin with them. (See my article at pp. 713-14.) In an employment case involving both scientific jobs and nonscientific jobs, the use of technical degree, nontechnical degree, no degree (a category necessarily resulting from the employees who have neither a technical degree nor a nontechnical degree), years of service at TVA, and age (a factor which helps measure in rough fashion the amount of pre-TVA experience) are intuitively reasonable. They are the factors one would most expect to influence the jobs on which TVA’s employees work, if race is not given weight in making those decisions. Race must then be added as an independent variable, to see if race is in fact given weight, after controlling for such other factors. This is an adequate theoretical foundation, particularly in light of the fact that plaintiffs have informed me that the first regressions introduced by plaintiffs (PX 516, 517) were the only ones they had performed and were not simply their first successful results after earlier unsuccessful attempts, and the fact that the variables they later tried (PX 636) produced the same results.' 8. The independent variables used by TVA are supported by the testimony of TVA’s expert, Dr. Martin, at pp. 2603-2625 of the transcript. They differ from plaintiffs’ variables in that ' Plaintiffs’ efforts should be contrasted with TVA’s on this point. According to TVA’s expert, the regressions offered by TVA were not the first ones they performed. Dr. Martin testified at Tr. 2610: “I hate to admit that there were a number of regression analyses that were done as I was trying to build the file.” F-4 they break down degrees by area of major concentration: social science, natural science, physical science, engineering science (other than chemical engineering), Code R (either a doctoral degree in any field or a chemical engineering degree), and highest grade completed. These, too, are intuitively reasonable for the type of jobs in question, and are adequately founded. 9. The fact that plaintiffs’ earlier regressions, which were run using the variables described in paragraph 7 (PX 516-517), produced the same statistically significant results as plaintiffs’ later regressions which were run using TVA’s variables de- scribed in paragraph 8 (PX 636), and that both sets showed . that black race had a substantial negative effect on job place- ment, shows conclusively that the differences in the two sets of variables do not make any practical difference. For purposes of analysis, one set of variables is as good as the other. 2. Taking Job Category and Job Schedule into Account 10. Multiple regressions allow one to study the effect of a particular factor on a dependent variable while controlling for the effect of other factors. This means that multiple regression estimates yield the effect of the factor of interest with other factors held constant. However, while it is very desirable to hold constant the effect of factors other than the one under study, it is self-defeating to hold constant the very factor whose effect is to be measured. 11. In the present instance, the factor whose effect (if any) is to be measured is that of race. Racial discrimination in employment can work in at least two different ways. It can produce systematically different wages for equally qualified whites and blacks in the same job or it can act through the systematic assignment, promotion, or reclassification of blacks to job categories which are lower paying than those to which equally qualified whites are assigned, promoted or reclassified. 12. To test whether blacks are systematically paid less than equally qualified whites in the same job, it is neces- sary—by definition—to control for job category, to make sure F-5 that jobs are indeed the “same” within the limits of the data. This can be done either by performing separate analyses for each job category or (sometimes) in an overall analysis by using variables which reflect job categories. Any procedure which controls for job categories, however, can only reveal whether discrimination acts through differential wages for the same job category; it is incapable of revealing whether dis- crimination acts through differential assignment, promotion or reclassification to job categories.2 13. To test whether discrimination does act through differential job assignment, promotion or reclassification, one must not control for the effect of job categories on wages. To do so is to hold constant the very phenomenon whose differential operation (if any) one seeks to measure. 14. This can be put in different language. If discrimination operates through the assignment, promotion or reclassification of blacks to lower-paying job categories or job schedules, then examination of relative wages in the same job category or schedule can never discover it. Yet controlling for job category or schedule through multiple regression does exactly that by holding job category or schedule constant. Only a multiple regression which does not control for job category or schedule is capable of testing whether discrimination works in this way. Such a regression would indeed control for variables such as education and years of service, and would answer the question whether equally situated blacks and whites are afforded un- equal treatment, whether or not that treatment takes the form of classifying them into different job categories or schedules. 15. In the present case, the regressions done by the plaintiffs properly fail to control for the effect of job category or 2 The case examined for illustrative purposes in my article (at pp. 721-725) is that of discrimination in wages for employees in similar jobs. It is for that reason that I remarked (id n.32) that “[c]ontrolling for job classification is an obvious thing to do and might be done by multiple regression”. I did not there examine the case in which discrimination operates through job classification. F-6 schedule? while those by TVA improperly do control for job schedule. The defendants’ regressions apparently fail to find a statistically significant effect of race on job placement for some schedules,4 while the plaintiffs’ regressions do find such an effect. 16. The discussions of Schedules A, B and M may help make this point clear. I am informed by counsel that TVA regularly promotes or reclassifies employees on Schedule B to jobs in Schedule A, and regularly promotes or reclassifies employees on Schedule A to jobs on Schedule M.5 In most years, there were no blacks in OACD who were working on Schedules A or M; in 1980, there was one black in OACD on Schedule M. To break down an analysis separately by each schedule puts a “thumb on the scale” by making it impossible to measure the effect of any racial discrimination in promotions or reclassifications from Schedule B to Schedule A or from Schedule A to Schedule M. In fact, if there was pervasive discrimination in promotions between schedules but no dis- crimination within schedules, then no schedule-by-schedule regression could, even in theory, detect the racial discrimination in promotions and reclassifications. Only an analysis combining at least Schedules A, B and M would show such discrimination. Similarly, if there were racial discrimination in assigning, promoting and reclassifying employees between Schedules E 3] understand that plaintiffs did perform a regression excluding Schedule F, to respond to TVA’s contention that including this schedule in the regression might produce a distortion and a “false positive” finding of discrimination. There was still a substantial, statistically significant negative effect of black race in job placement. (PX 636) Thus, the inclusion of this schedule did not produce a distortion. The same is true for the regression excluding the M Schedule. (Jd.) There is no basis for concluding that these results would have been any different if the M Schedule or the F Schedule had been deleted from the regressions in PX 516 and PX 517. 4 In reality, there is a significant effect. (See 17 17-20, infra.) For purposes of the present section I suppose for the sake of argument that no significant effect is found. Sin similar fashion, TVA regularly promoted or reclassified employees on Schedule E to jobs in Schedule D. F-7 and D, or between Schedules D and M, only an analysis combining at least these schedules would show such dis- crimination. C. INTERPRETATION OF TVA’S REGRESSIONS 17. My conclusions from TVA’s own regressions are that, despite the weakening of the power of statistical tests to detect significance because of the small numbers of blacks when each schedule is examined in isolation: (a) black race was a significant negative factor affecting pay rate among equally qualified black and white employees on Schedule E; (b) there is a strong likelihood that the same is true for Schedule D and that the failure to find a greater level of significance is simply a “false negative” distortion caused by the small number of blacks working on this schedule; and (c) examination of the results for Schedules D and E together shows that black race was in fact a significant factor affecting job placement among equally qualified black and white employees when Schedules D and E are combined. See DX 201 and DX 202. 18. The level of significance found by TVA’s studies on a “two-tailed” basis for race was .0546 on Schedule E, .1635 on Schedule D, and .0906 on combined Schedules D and E. The “two-tailed” test of significance is appropriate only if there is equal interest in the question of discrimination against whites as there is in the question of discrimination against blacks. Its use doubles the probability level found on a “one-tailed” test.® Because counsel inform me that this case challenged the subjective personnel determinations of white TVA officials as disadvantaging blacks, the only question of interest here is whether blacks have been harmed. A “one-tailed” test of significance is appropriate, and a “two-tailed” test is not. Correcting the doubling of the significance level caused by the TVA program’s automatic printing out of “two-tailed” results, ® Because regression equations are run in many situations, the standard computer programs report the two-tailed level of signifi- cance. If a one-tailed test is appropriate, the reported significance level is divided by 2. F-8 the actual significance levels are .0273 for Schedule E, .0818 for Schedule D and .0453 for combined Schedules D and E. These are all unusual events; and one is entitled to question their fairness even if they do not all meet the arbitrary 95-out-of-100 times corresponding to the .05 level of significance. 19. It is also improper to view the results for Schedule D as totally separate from the results of Schedule E. In each equation, blacks were found to hold lower paying jobs than similarly qualified whites. Because TVA’s regression for Sched- ule D is independent of TVA’s regression for Schedule E, and because both operate under the same personnel system, it is proper to ask what the probability is that both sets of results could have occurred by chance. This is calculated by multiplying .0273 by .0818, for the result of .0022. In other words, black race is such a significant negative factor affecting job placement among equally qualified black and white em- ployees on Schedules, D and E, that TVA’s own results in DX 201—showing that blacks on Schedule D earn $1,220, and blacks on Schedule E earn $2,250, less than their similarly qualified white counterparts—could have occurred by chance only 22 times in every 10,000 cases. 20. It may be objected that this probability is artificially small because the probability that analysis of some pair of schedules will show significant racial effects is substantially bigger than the probability that analysis of Schedules D and E in particular does so. Which calculation is appropriate rests on whether there is reason to be specially interested in Schedules D and E or whether those schedules have been singled out only after seeing the results. I understand from counsel for plaintiffs that in fact there are reasons for being specially interested in Schedules D and E. In any event, it does not matter to this conclusion which view is adopted. Assuming that there was no effect of black race in the regressions for Schedules A, B and F, the other schedules which comprised the class at the time of F-9 trial, 7 the probability of observing racial effects in at least two | schedules as significant as those observed in fact in Schedules D and E is only .034, less than the 5% level at which results are conventionally said to be significant. If one eliminates Schedule A from consideration—an appropriate thing to do since with no blacks on Schedule A no racial effect could be estimated —the probability of observing racial effects by chance as big as those observed on Schedules D and E for at least two out of four schedules is even less, about .021. _SS8/ FRANKLIN M. FISHER Franklin M. Fisher Sworn to before me this 16th day of May 1983. /s/ CLARK HOOPER Notary Public 7 This is a conservative assumption. The stepwise regression technique used by TVA does not permit one to know whether such an effect of race shows up if it is statistically insignificant enough. AFFIDAVIT OF PROFESSORS DAVID C. BALDUS AND JAMES W. L. COLE G-1 > APPENDIX G UNITED STATES COURT OF APPEALS For THE ELEVENTH CIRCUIT Frank L. EASTLAND, et al., Plaintiffs-Appellants, No. 82-7008 AFFIDAVIT —against— TENNESSEE VALLEY AUTHORITY, et al., Defendants-Appellees. STATE OF New York, COUNTY OF ONONDAGA, STATE OF PENNSYLVANIA, on: COUNTY OF ALLEGHENY, ; Davip C. BaLpus and James W. L. Coxe, being duly sworn, depose and say: 1. We wrote the treatise cited by the Circuit Court in this case, D. Baldus and J. Cole, Statistical Proof of Discrimination (1980 and 1982 Supp. ). 2. Mr. Baldus is a lawyer and is on the faculty of the University of Syracuse School of Law. Dr. Cole is a mathematical statistician with a Ph.D. from the University of North Carolina. He taught statistics at the University of lowa from 1968 to 1975. Since 1975, he has been an independent statistical consultant and has testified as an expert in the field of statistics. 3. Two aspects of the interpretation of the statistical evidence in Eastland v. TVA have come to our attention as being worthy of comment. First, the Court appears to place on G-2 the plaintiffs a heavy burden of accounting for interaction terms in a broad class of cases in which multiple regression methods are offered as evidence. This is of particular concern to us, because the Court’s reasoning suggests that our previous efforts in discussing this issue are not sufficiently clear.’ Second, in its final evaluation of the statistical evidence the Court does not take advantage of statistical methods which have been used and accepted in recent cases, which permit a statistical assess- ment of the degree to which separate pieces of suggestive evidence become persuasive when viewed jointly. THE GENERAL CONSEQUENCES OF OMITTING INTERACTIONS 4. In its evaluation of regression analyses put forward by the plaintiffs in Eastland v. TVA, the Circuit Court appears to accept the defendants’ contention that the inclusion of terms accounting for interactions between job types and other quali- fications are an absolute prerequisite for-an adequate multiple regression equation when modeling the promotion decisions in a diverse work force.? In this analysis, the Court appears to have drawn on a statement in our book. 5. Certainly when a single regression is put forward as a representation of a decision process operating throughout a diverse work force, there is a legitimate concern that interaction terms may be needed if the model is to give a representation which is sufficiently accurate in relevant respects as to be useful. For example, when it seems likely that an additional year’s seniority will yield a greater salary increase among managerial employees than among other employees, one may anticipate that a schedule-by-seniority interaction may be needed.* How- 1 The discussion in question is in Baldus and Cole, Statistical Proof of Discrimination, Shepards McGraw, Hill (1980), at 203. 2 Eastland v. TVA, at 2876. 3 Jd. at n.16. 4 E.g., Eastland v. TVA, at 2876. —_ Ree on G-3 ever, what is anticipated is not always what occurs. The world of multiple regression modeling is full of pitfalls and surprises, even for the experienced practitioner. For example, an exam- ination of the defendants’ regression equations representing separate job schedules indicates that in fact marginal differ- ences in seniority were not rewarded more among managerial workers than among all others.5 In this world of complexity the need for interaction terms cannot be established on the basis of a simple rule, but rather by weighing a number of considerations and drawing on several pieces of statistical information. 6. In the discussion which follows, we expand on several points considered in our book at pp. 277-281, using as a point of departure the question of whether “schedule-by-race” and “schedule-by-qualification” ® variables are required in defend- ants’ regression models fitted to the employees in the pool obtained by merging job schedules D and E.’ 7. A preliminary consideration to be noted is that the inclusion or omission of a schedule-by-race interaction and a schedule-by-qualification interaction have essentially different consequences for the probative value of the resulting equation, particularly when the equation omitting the interaction term has a race coefficient large enough to support an inference of disparate treatment against blacks. 8. In this situation, the inclusion of a schedule-by-race term may lead to the inference that the disparate treatment is more pronounced in one schedule than in another, but it cannot make the evidence of disparate treatment vanish in both 5 Compare coefficients for YRSSER for Schedule M, Step 12 (B = 212.99) with corresponding coefficients for Schedule B, Step 4 (B = 220.01), Schedule D, Step 7 (B = 227.82), and Schedule E, Step 5, (B = 174.72) (DX 201). ® In this discussion, “qualification” is a generic term which may denote any of the following variables included in the TVA regression models: AGE, HIGR, YRSSER, SSCI, NSCI, PSCI, ESCI and CODE 7 See DX 202. . G-4 schedules simultaneously. Thus, in general, if a multiple regression without a schedule-by-race interaction indicates the presence of class-wide disparate treatment, the defendants’ position will not be improved by the addition of that inter- action. 9. On the other hand, the consequences of omitting or adding a schedule-by-qualification interaction are less pre- dictable. When a regression omitting such an interaction has a race coefficient supporting an inference of disparate treatment, it is not predictable in general whether the addition of such an interaction will tend to inflate, leave unchanged, or deflate the net disparity associated with race. From the converse view- point, this means that when a schedule-by-qualification inter- action is required, its omission may bias the regression in favor either of the defendant or the plaintiff and neither the direction nor the magnitude of the bias is predictable. 10. In this respect schedule-by-qualification interactions are essentially similar to other qualification variables which the plaintiffs may have overlooked or otherwise omitted from the equation. Accordingly, challenges, in particular the judgment as to the need for a qualification-by-schedule interaction, by the defendants focusing on the omission of one or more schedule- by-qualification interaction should be evaluated according to the same principles, and applying the same burdens, as chal- lenges focusing on the omission of an allegedly significant qualification variable. 11. In this light, procedures which test the statistical significance of the interaction terms at issue provide useful statistical evidence as to the need for the interaction terms, just as stepwise regression procedures use significance tests to aid in the selection of variables for a multiple regression model. 12. However, the most probative evidence as to the adequacy of a regression model in the face of a challenge faulting the omission of schedule-by-qualification variables is a comparision of the challenged model with an alternative put forward by the challenging party, by which one can determine G-5 whether and in what direction the addition of the interaction terms changes the coefficient for the race variable.® INTERPRETATION OF THE STATISTICAL EVIDENCE FROM SCHEDULES D AND E 13. The analysis of the separate regressions in DX 201 presents plaintiffs with a problem which the Court ack- nowledges: “The breakdown by job schedule reduces the size of the data base, thereby making it more difficult to establish statistically significant relationships.”® This problem in the interpretation of statistical evidence has been recognized elsewhere as being potentially a major imped- iment to plaintiffs in discrimination cases.'° Various statistical methods are available which combine the evidence from sepa- rate sets of data, and assess their weight when they are applied jointly to bear on a question common to the analysis for each separate data set. Although these methods have been available to statisticians for a relatively long time, their use in dis- crimination cases is a much more recent development. How- ever, in view of the potential severity of the problem that plaintiffs face when forced to control for variables like job classification, the wider acceptance of such methods in appro- priate circumstances is to be recommended. ® For further discussion of factors to be considered when weigh- ing a challenge centering on the omission of a qualification variable, see Section 8.23 of the 1982 supplement to our book. ® Eastland v. TVA, at 2876. 10 See, e.g., Valentino v. United States Postal Service, 674 F.2d 56, 72, 28 F.E.P. 593, 606 (D.C. Cir. 1982). The absence of a finding ~ rear Pel néadebes ent ec daeenaiesaen eee small sample sizes, such “false negative” findings occur quite G-6 14. The method which has been used and accepted most widely in the cases is sometimes referred to as Fisher’s Method.'' While not necessarily the best method in any given situation, Fisher’s Method has the advantages of being rela- tively simple to apply and understand and of being applicable in a broad class of circumstances. 45. A comparison of the analyses in DX 201 for separate schediiles with a parallel analysis in PX 636 for the pooled Schedules A, B, D, E, and F (labeled “OACD 1979 EX- CLUSIVE OF SCHEDULE M” and found on pp. 43-54 of the exhibit) demonstrate how the analysis of the data within schedules handicaps the plaintiffs by decreasing the probability of obtaining statistically significant associations even when the estimated magnitude of the disparity is not materially changed. In the final step of the analysis for the pooled schedules, the estimated race effect is $1,685.90 with a standard error of $643.15 producing an F ratio of 6.871‘? which is significant at the .01 level. By comparison, the corresponding results from the last steps in Schedules D and E which contain estimates of the race effect are, for Schedule D, estimated race effect, $1,221.56, standard error $871.82, F ratio, 1.96, significance level .1635, and for Schedule E, estimated race effect $2,254.28, 11 After R. A. Fisher, an English statistician. This method is used to combine the results of independent analyses in which individual levels of significance for the race coefficient have been computed. To apply the method, the “natural” logarithm of each level of signifi- cance is multiplied by minus 2, and these products are summed. The resulting sum can then be treated as a Chi-squared random variable with degrees of freedom equal to twice the number of independent analyses. For applications or discussions of this method in dis- crimination cases, see, ¢.g., Jurgens v. Thomas, 29 F.E.P. 1561, 1569 n.21 (N.D. Tex. 1982); Brown v. Delta Airlines, 30 F.E.P. 39, 45 n. 14 (S.D. Tex. 1982); Adams v. Gaudet, 30 F.E.P. 1258, 1294-96 (W.D. La. 1981). For a general discussion of the methodology see Fisher, Statistical Methods For Research Workers (12th Ed.) (Oliver & Boyd 1954), at 99-100; Rosenthal, Combining Results of Independent Studies, 85 Psych. Bull. 185, 186-88 (1978). 12 In this situation, the F ratio can be computed as the square of the ratio between the estimated effect (“B”) and its standard error. G-7 standard error $1,158.40, F ratio, 3.79, and significance level, 0546. 16. Notwithstanding the discussion of interactions in the previous section, the principal reason for analyzing the data within separate job schedules is to avoid being misled by the possibility that blacks are concentrated in lower paying job schedules while whites are disproportionately numerous in higher paying schedules. If this were the case, we would anticipate the estimated race effects to be substantially smaller within schedules than when the schedules are pooled. Clearly this did not occur. In both schedules D and E, the increase in standard error was a major factor in decreasing the statistical significance, and in Schedule E it was clearly the only factor, since the estimated race effect in Schedule E was greater than in the combined pool of five schedules. 17. In view of these observations, this case is clearly one in which some effort is indicated to combine the separate pieces of evidence of a pervasive policy into a single summary figure. 18. Using Fisher’s Method, we may combine the signifi- cance level results from Schedules D and E in DX 201 to obtain a summary Chi-squared statistic of 9.437, which is significant at the .051 level.'3 19. An alternative statistical approach to combining the evidence from the two schedules is based on the calculation of a weighted average of the estimated race effects in the two schedules. '* The resulting average of $1,594.99 is 2.290 times as large as its estimated standard error, so that it is statistically significant approximately at the .025 level.'5 13 The Chi-squared statistic has four degrees of freedom. 14 Specifically we make the weights inversely proportional to the respective variances of the within schedule estimates. By doing so, we obtain an estimate which is maximally precise when, as claimed by the defendants, there is no discrimination in either schedule. 'S This significance was computed on the assumption that the ratio is approximately normally distributed, an assumption which may be considered as valid as it is in many other applications in dis- crimination law. G-8 20. Throughout the foregoing analysis, we have used two- tailed significance levels although I understand that plaintiffs urged that a one-tailed level was appropriate. In the absence of any stated policy of favoring blacks, it may be argued per- suasively that in a case of this type, any single piece of evidence suggesting discrimination in favor of blacks would have been disregarded as a curiosity caused by chance in view of the general pattern of the evidence showing either adverse treat- ment of blacks or equal treatment. If so, it would be appropri- ate to use one-tailed significance levels, in which case the race effects estimated within each schedule would have been signifi- cant at the .0818 level in Schedule D and the 0.0273 level in Schedule E.'® Using Fisher’s Method to combine these results would have produced a Chi-squared statistic of 12.208 which is Statistically significant at the .016 level. Finally, the weighted average race effect would be significant at a level of less than 0.015. 16 The estimated rate effects produced in the last step of DX 202 for the combined Schedules D and E would also have been significant at the 0.0453 level using one-tailed significance levels. G-9 21. In summary, these observations taken together suggest that the results of the defendants’ separate analyses of Sched- ules D and E tend to support conclusions not substantially different from those drawn from the plaintiffs’ analysis of the pooled schedules, at least to the degree that the pooled analysis indicates the presence of disparate treatment in Schedules D and E. /s/_ Davin C. BALDUS David C. Baldus Sworn to before me this 16th day of May 1983. /s/ Caro S. WILLIAMS Notary Public CAROL S. WILLIAMS Notary Public in the State of New York Qualified in Onondaga Co. No. 4734981 My Commission Expires March 30, 1985 _/s/_ Sames W. L. COLE James W. L. Cole Sworn to before me this 13th day of May 1983. __{s/_ Mary ALICE MCCAFFERTY Notary Public MARY ALICE McCAFFERTY, Notary Public Shaler Township, Allegheny County My Commission Expires Dec. 27, 1986 Member, Pennsylvania Association of Notaries H-1 APPENDIX H UnrtTep STaTes CourT OF APPEALS For THE ELEVENTH CIRCUIT FRANK L. EASTLAND, et al., Plaintiffs-Appellants, No. 82-7008 —against— r TENNESSEE VALLEY AUTHORITY, et al., Defendants-Appellees. STATE OF NEw YorK, 9s: COUNTY OF NEw York, x MICHAEL O. FINKELSTEIN and BRUCE LEVIN, being sev- erally duly sworn, do depose and say: 1. Michael O. Finkelstein is a member of the bar of New York and a member of the firm of Barrett Smith Schapiro Simon & Armstrong. He specializes in statistical presentations in legal cases. He is Chairman of the Special Committee on the Use of Empirical Data in Legal Decisionmaking of the Associ- ation of the Bar of the City of New York. For many years he has taught a course entitled Statistics for Lawyers at Columbia Law School and has published numerous articles and one book in this field. In particular, he is the author of The. Judicial Reception of Multiple Regression Studies in Race and Sex Discrimination Cases, 80 Colum. L. Rev. 737 (1980) that appeared as a companion piece to the article by Professor Franklin M. Fisher that was cited by the Panel in this case. H-2 2. Dr. Bruce Levin is Assistant Professor of Clinical Public Health in Biostatistics, Columbia University School of Public Health in New York City. He is a member of Statistica Consulting, Inc., a firm that specializes in providing expert statistical analysis in legal cases and has testified frequently as an expert statistician. He is the author of a mumber of publications on statistics, the most recent being on a problem in the use of statistics in discrimination cases. See Robbins & Levin, A Note on the “Underadjustment Phenomenon”, | Statis- tics & Probability Letters 137 (1983). 3. We have been asked by counsel for the appellants to review the Panel’s opinion, the District Court’s opinion, the briefs, the multiple regression analyses offered by both parties and the trial testimony about them. We conclude that even if the Panel is correct as a theoretical matter that job categories must be taken into account in performing multiple regression analyses in a case such as this (a proposition that is open to serious question since it introduces a variable, assignment te jub category, that itself is alleged to be the subject of dis- crimination), the analyses presented by the TVA, which took job categories into consideration, show clearly that race is a significant factor on at least one of two job schedules, D and E. 4. TVA presented regression studies for separate job schedules (DX 201) and a regression study combining Sched- ules D and E (DX 202). In both Schedules D and E, and in the combined Schedule D and E, the race coefficients in the regressions were negative. That means that on each schedule comparably qualified blacks earned less than whites. The SAS computer program used to produce the regressions automati- cally printed out probability levels (descriptive levels of signifi- cance) on a two-tail basis; this is twice the size of the probability level on a one-tail basis. Which basis is appropriate depends on whether one is equally concerned with both discrimination for blacks and discrimination against them. Since this case alleges discrimination against blacks, the one- tailed basis is the appropriate one, and the probability levels in H-3 the printouts must be divided by two in order to put them on a one-tail basis. On that basis, the race coefficient in the Schedule D regression is close to statistical significance (0.0818), the race coefficient in the Schedule E regression is of substantial signifi- cance (.0273), and the race coefficient for the combined Schedule is also significant (.0453) assuming, as the Panel does, that .05 is the critical level of significance. 5. The number of blacks included in Schedules D and E together is small from a statistical point of view (PX 511 at p. 12 shows that there were only 7 blacks on Schedule E, and 14 on Schedule D, at the end of 1979, some months before the data reflected in TVA’s regression). This raises the possibility that discrimination exists but would not be found statistically significant due to the low power of the method of analysis to detect discrimination when there are such small numbers. The problem is compounded when the numbers of blacks are further reduced by dividing the data into separate schedules; any real difference can be made to lack statistical significance if the data are divided into a sufficient number of small groups. On the other hand, if the Court adheres to its view that job schedules should be analyzed separately, the true level of statistical significance for both the Schedule D regression and the Schedule E regression would have to be calculated by looking simultaneously at the evidence from the two regressions rather than by pooling the data. That can easily be done from the data already in the record, and when it is done, the result shows that race is a statistically significant factor. In the remainder of this affidavit, we show the method and the results of such a computation. 6. Since separate groups of employees were involved in the D and E schedules, the two regression studies are independ- ent in the statistical sense. A basic technique for combining results from independent studies is due to R. A. Fisher, the founder of modern statistics, and is described in his classic work, Statistical Methods for Research Workers, 99-101 (11th Ed., London Oliver & Boyd, 1950). A further discussion may H-4 be found in R. Rosenthal, Combining Results of Independent Studies, 85 Psychological Bulletin, 185-93 (Jan. 1950). 7. To explain Fisher’s technique, we bdcgin by noting that the descriptive level of significance for a single race coefficient is the probability thai, under the hypothesis that race was not a factor in salary, the race coefficient in the regression would indicate a shortfall for blacks as large or larger than that actually observed in the sample. Since two independent regressions are involved, the probability that race coefficients in both studies would be found as large or larger than that observed for each Schedule is the product of the two descriptive levels of significance. The product may be termed the “joint descriptive level of significance”. The smaller that number, the more statistical evidence there is that at least one of the race coefficients is different from zero. 8. In accordance with general statistical techniques of hypothesis testing, we ask the following: if the true race coefficient is in fact zero in both Schedules, what is the probability that the joint descriptive level of significance would be as small or smaller than that observed in the sample? The answer is calculated by computing the natural logarithm of the joint descriptive level of significance multipled by -2. This is a useful test statistic because it has a well-known (chi-squared ) probability distribution so that the probability that it will exceed any given value can be obtained from standard tables of the chi-square distribution found in most elementary statistics texts. If this probability is less than the critical level of significance, the hypothesis that-race was not a factor in either Schedule must be rejected. 9. As we noted above, the descriptive level of significance of the race coefficient is shown on a two-tail basis as 0.1635 for Schedule D and for Schedule E as 0.0546. Since we believe that a one-tail test is appropriate, each of these significance levels is divided by two to obtain the one-tail significance levels of .0818 and .0273, respectively. H-5 10. Fisher’s statistic is then equal to -2 In (.0818 X .0273) = 12.21. From tables of the chi-squared distribution (in this case on 4 degrees of freedom) one finds the probability of observing a joint descriptive level of significance this extreme under the hypothesis that race was not a factor in either Schedule is .016 on a one-tail basis. Thus the joint descriptive level of significance which represents the combined impact of the evidence of both Schedules is stronger than defendant has computed by separating the Schedules and failing properly to combine the evidence. If .05 is accepted as the critical level of significance, we must reject the hypothesis that race was not an influence on salary in either Schedule. 11. Another method of combining the evidence across the two Schedules is to consider the average of the race coefficients. Assuming that the true race coefficient is equal to zero in each case, the expected average is also zero. To test whether the true average is zero, we divide the observed average by its standard error and obtain the so-called z-score, which is the number of standard deviations between the observed and expected aver- ages. The method of standard deviation analysis used here is similar to that approved by the Supreme Court for analyzing claims of discrimination in Castaneda v. Partida, 430 U.S. 482, 496 n. 17 (1977). 12. Here, DX 201 shows that the race coefficient of the D Schedule is $1221.56 (changing signs for convenience) with a standard error of $871.82 (p. 5, Step 6) while the race coefficient for the E Schedule is $2254.28 with a standard error of $1158.40 (p. 8, Step 5). The average is $1737.92 with a standard error of $724.91. The z-score is thus 1737.92/724.91 = 2.40, which is significant at the one-tail .0082 level, and also significant at the two-tail .0164 level. If .05 is accepted as the H-6 critical level of significance. we must reject the hypothesis that the average of the race coefficients in the two schedules ‘s zero. ¢ /s/ MICHAEL O. FINKELSTEIN Michael O. Finkelstein Sworn to before me this 16th day of May 1983. /s/ Dora Lee Epwarps Notary Public DORA LEE EDWARDS Notary Public, State of New York No. 24-4504159 Qualified in Kings County Term Expires March 30, 1985 /s/ BRUCE LEVIN die Bruce Levin Sworn to before me this 16th day of May 1983. /s/ ROBERT W. BEAURY | Notary Public ROBERT W. BEAURY Notary Public, State of New York No. 41-4725819 Qualified in Queens County Certificate Filed in New York County Commission Expires March 30, 1984 | | ia --- Source: Frix Law Library, https://www.frixlaw.com/law-library/documents/brief%3Amicro_IA40385011_0773%3A1. Public record. Not legal advice.