“Daubert . . . does not forbid 17 admission of a report where the weight of the conclusions are subject to challenge.” (internal 18 quotations and citation omitted)
How later courts described this case
- “Daubert . . . does not forbid 17 admission of a report where the weight of the conclusions are subject to challenge.” (internal 18 quotations and citation omitted)
- expert opinion 23 based on “supposed ‘uniqueness’ of a market does not justify substituting a guess for careful 24 analysis”
- “If 25 Defendant wishes to challenge the weight that a fact finder may attribute to the extrapolation, it 26 can employ cross-examination and competing witnesses and testimony to highlight the disparity 27 between claims.”
- court exercised gatekeeping role sua sponte by addressing 10 deficiency in expert report
Written by the judges who cited it.
The opinion
1
2
3
4
5
6
7
8 UNITED STATES DISTRICT COURT
9 FOR THE EASTERN DISTRICT OF CALIFORNIA
10
11 UNITED STATES OF AMERICA, and the No. 2:12-cv-01699-KJM-EFB
STATE OF CALIFORNIA, et al., ex rel.
12 LOYD F. SCHMUCKLEY, JR., ORDER
13 Plaintiffs,
14 v.
15 RITE AID CORPORATION,
16 Defendant.
17
STATE OF CALIFORNIA, ex rel. LOYD
18 F.SCHMUCKLEY, JR.,
19
Plaintiffs,
20 v.
21 RITE AID CORPORATION,
22
Defendant.
23
24 Three related motions are before the court. Defendant Rite Aid moves to exclude
25 plaintiffs’ proposed sampling methodology, Sampling Mot., ECF No. 195, and also moves to
26 exclude and strike allegedly untimely evidence and an expert opinion accompanying plaintiffs’
27 opposition to defendants’ motion to exclude, Def.’s Mot. Strike, ECF No. 208. Plaintiffs move to
28 exclude portions of the expert report and testimony of Rite Aid’s expert Roy J. Epstein, Ph.D.
1 Pls.’ Mot. Exclude, ECF No. 206. On July 15, 2019, the court heard oral argument on the
2 motions. W. Paul Lawrence, Jennifer Bartlett and Brian Barrow appeared on behalf of relator
3 Loyd Schmuckley; Emmanuel Salazar and Bernice Yew appeared on behalf of intervenor State of
4 California; Benjamin Smith appeared on behalf of Rite Aid. At the conclusion of the hearing, the
5 court permitted each party fourteen days to file supplemental briefing addressing two narrow
6 questions posed by the court. Thereafter, the matter was submitted for resolution by written
7 order. Having thoroughly considered the motions, supplemental briefs and arguments at hearing,
8 the court DENIES all the motions for the reasons provided below.
9 I. BACKGROUND
10 A. Relevant Factual Background
11 Given the substance of the pending motions, the court tailors its factual summary
12 to the development of the sampling methodology at issue. In this qui tam action, relator Loyd F.
13 Schmuckley, Jr. and intervenor plaintiff State of California (collectively “plaintiffs”) allege Rite
14 Aid is liable under the False Claims Act (“FCA”) and the California False Claims Act (“CFCA”)
15 for failing to comply with “Code 1 restrictions” as required by Medi-Cal reimbursement
16 regulations. Complaint-in-Intervention (“CII”), ECF No. 75, ¶¶ 4, 6, 118–123. Although it has
17 declined to intervene, the United States remains a real party in interest to this action under
18 31 U.S.C. § 3730(b)(1). See Not. of Declination, ECF No. 38.
19 As relevant here, when the United States and the State of California (collectively
20 “the government”)1 investigated whether to intervene as parties to this action, there were several
21 layers to their investigation. The first layer was focused on the nature of Rite Aid’s obligations as
22 a provider under the Medi-Cal program. Opp’n to Sampling Mot., ECF No. 202, at 5. As an
23 eligible Medi-Cal provider, i.e., a provider able to receive reimbursement for prescription
24 medications covered by Medi-Cal, Rite Aid was required to execute several provider agreements
25 in which it agreed to comply with all relevant rules and regulations, including certain prescription
26 documentation and certification requirements. CII ¶¶ 20, 52. One such agreement, known as the
27
1 Where the State of California acts only on its own behalf, the court refers to it as
28 “California” below.
1 Medi-Cal telecommunications provider and biller agreement, or computer media claim (“CMC”)
2 agreement, allows Rite Aid to submit reimbursement claims electronically through the Medi-Cal
3 system. Id. ¶¶ 21, 52. Here, too, Rite Aid agreed to comply with Medi-Cal documentation rules
4 as a condition to utilizing electronic claim submission. Id. ¶ 21.
5 When a provider such as Rite Aid submits a prescription reimbursement claim
6 through the CMC system, the type of prescription dictates the ease with which the claim is
7 processed. Id. ¶¶ 43–47. If a claim prescription involves a Code 12 drug, the system will
8 automatically reject the claim because of heightened restrictions placed on those drugs. Id. ¶ 45.
9 This automatic rejection occurs only when a Code 1 claim is submitted for the first time. Id. For
10 an initial Code 1 claim to be processed, the provider must resubmit the claim with an affirmative
11 statement that Code 1 requirements have been satisfied. Id. ¶ 46. This resubmission and
12 affirmative statement comes in the form of an override code provided by the CMC system. Id.
13 ¶ 47. Under the Medi-Cal provider manual, to deploy an override code to a rejected Code 1
14 claim, the provider must confirm the Code 1 drug is restricted and the restrictions are satisfied.
15 Id.
16 Rite Aid’s internal policies and computer-based dispensing system largely track
17 these requirements. Id. ¶ 82. Before a Code 1 claim is processed to Medi-Cal, Rite Aid’s system
18 generates a warning to alert the pharmacy of the Code 1 transaction. Id. The Rite Aid associate
19 processing the transaction must then follow a series of verification and documentation
20 instructions to ensure the prescription complies with Code 1 restrictions. Id. If the associate
21 meets these requirements, and the Pharmacist approves, an internally generated override code is
22 used to bypass the Code 1-related warning in Rite Aid’s system. Id.; Opp’n to Sampling Mot. at
23 7. If, however, the Code 1 prescription does not comply with Code 1 restrictions, then the Rite
24 Aid associate must contact the prescriber to determine whether a change in drug therapy is
25
26
27
2 Code 1 drugs are those “identified on the CDL [Medical List of Contact Drugs] with the
asterisk (“*”) symbol” and “require prior authorization in accordance with Cal. Code Regs. tit.
28 22, § 51003.” CII ¶ 38.
1 necessary or a treatment authorization request (“TAR”)3 must be submitted to Medi-Cal for
2 approval. Id. ¶ 84. In either event, if the prescriber modifies the prescription or provides
3 additional information, the modification must be documented by the associate in hard copy. Id.
4 The documentation must then be scanned into Rite Aid’s system for easy retrieval. Id. ¶ 85.
5 The second layer of the government’s pre-intervention investigation pertains to
6 sampling. Because of the sheer volume of claims processed by Rite Aid through the Medi-Cal
7 system, the government deployed statistical sampling techniques to “determin[e] whether
8 intervention in th[is] qui tam matter was worthy and meritorious.” Opp’n to Sampling Mot. at 7.
9 Specifically, the government sought to “estimat[e] the number of, percentage of, and total
10 payments associated with false claims made by Rite Aid to Medi-Cal for Code 1 drugs.” Id. at 8
11 (citing Petron Rep., ECF No. 204-1). To do this, the government developed a sampling
12 methodology testing whether a Rite Aid associate, when met with a Code 1 rejection message,
13 “actually performed the requisite Code 1 review, verification, and documentation” before
14 utilizing the override code to process the claim. Id. at 8; CII ¶¶ 97–106.
15 To develop this methodology, the government asked the California Department of
16 Health Care Services (“DHCS”) to pull a subset of paid claims from its rules-based software
17 program known as Symmetry. Opp’n to Sampling Mot. at 9 (citing Yew Decl. ¶ 5, ECF No. 39-
18 1; Petron Dep., ECF No. 204-3, at 32:4–12). This request produced a batch of 10,810 claims
19 submitted by Rite Aid for Code 1 restricted drugs without a TAR involving service dates from
20 2010 through 2013. Id. (citing July 27, 2018 Resp. to Interrog., ECF No. 204-4, at 14:13–21;
21 Meixner Decl. ¶ 4, ECF No. 204-5). The government then reviewed Rite Aid pharmacy and
22 prescriber medical records and, based on this review, “probed a random sample” from the batch
23 of 10,810 claims. Id. This review revealed a “statewide pattern of non-compliance.” Id. (citing
24 Yew Decl. ¶¶ 5–7).
25 Next, after consulting with “Rite Aid and Medi-Cal subject matter experts,” the
26 government refined its sample testing and broadened the subset of potentially affected claims. Id.
27
3 Treatment authorization requests (“TARs”), as defined in Cal. Code Regs. tit. 22,
28 §51003, are described in greater detail below.
1 To do this, California ran multiple data queries in the Medi-Cal claims database, one of which
2 focused on service dates from 2013 through 2014. Id. at 10 (citing Meixner Decl. ¶ 5). The
3 query for 2013–2014 produced over 3.8 million Code 1-related claims. Id. Because not all Code
4 1 drugs have relevant diagnosis restrictions, the query was further refined. Id. The government
5 then consulted with DHCS to identify all national drug codes (“NDCs”) related to Code 1,
6 diagnosis restricted drugs. Id. There were 2,919 NDCs that aligned with the government’s
7 sampling objective. Id. (citing Meixner Decl. ¶ 6).
8 The government then, after consultation with DHCS, identified certain drugs on
9 the List of Contract Drugs it believes are typically dispensed outside the Code 1 diagnosis
10 restrictions. Id. Out of the 2,919 NDCs that aligned with the sampling objective, 938 of these
11 were identified as having a potential for inconsistent prescriptions; these were placed in a data
12 batch entitled “Off-Formulary.” Id. (citing Petron Dep. at 72:8–73:11; Resp. to Interrog. 15:15–
13 18). The remaining 1,938 NDCs, from the original group of 2,919, were assigned the title
14 “Diagnosis-Related.” Id. The government then filtered these two defined groups against the
15 more than 3.8 million claims for 2013-2014, which ultimately produced the Diagnosis-Related
16 and Off-Formulary sample universes. Id.
17 With the three universes thus defined—the Symmetry universe, the Off-Formulary
18 universe and the Diagnosis-Related universe—the government then chose to employ a stratified
19 random sampling design to test the claims. Id. (citing Yew Decl. ¶ 6; Petron Rep. ¶ 16; Petron
20 Dep. at 87:4–11). In the process known as stratification, the Off-Formulary and Diagnosis-
21 Related universes were each divided into three subcategories, totaling six subcategories, called
22 sample frames. Id. at 11 (citing Petron Rep. ¶ 21). The Diagnosis-Related universe was broken
23 into sample frames “A,” “B,” and “C,” and the Off-Formulary universe was broken into sample
24 frames “D,” “E,” and “F.” The Symmetry universe, however, was not stratified because of its
25 “size, covered period, and ‘outlier nature’”4; therefore, the Symmetry sample frame consisted of
26
27 4 Symmetry comprises “outlier” claims in that, as a third-party software program, it
focuses on “claims where a drug [had] been prescribed to a particular beneficiary whose claim
28 history [did] not support the use of the prescribed drug.” Opp’n to Sampling Mot. at 9
1 the original batch of 10,810 claims created through the process described above. Id. (citing
2 Meixner Decl. ¶ 4). The government also took steps to ensure there were no overlapping claims
3 across the three universes. Id. (citing Petron Rep. ¶ 13; Epstein Rep., ECF No. 201, ¶ 21).
4 In the final step, choosing to apply a 95 percent confidence level5 as within a
5 generally acceptable range to ensure a sampling outcome at a confidence level beyond “a mere
6 guess,” Petron Rep. ¶ 18, the government pulled a sample from each sample frame using RAT-
7 STATS, a software program approved by the U.S. Office of Inspector General for use in the
8 claims review process. Id. (citing Petron Rep. ¶¶ 18, 21). Plaintiffs provide a breakdown of the
9 government’s stratification and sample selection process, which the court reproduces below:
10
11 Sample Frame Time Period Sample Frame Size Sample Size
12 A 1/1/2007 to 6/30/2010 121,735 383
13 B 7/1/2010 to 6/30/2013 137,243 296
14 C 7/1/2013 to 12/31/2014 41,098 88
15 D 1/1/2007 to 6/30/2010 69,880 383
16 E 7/1/2010 to 6/30/2013 92,883 285
17 F 7/1/2013 to 12/31/2014 31,996 98
18 S 1/1/2010 to 12/31/2013 10,810 371
19 Total 505,645 1,904
20
21 Id. (citing Petron Rep. ¶¶ 14, 21; Resp. to Interrog. 15:23–27; Meixner Decl. ¶ 7).
22 In 2015, the government discussed this sampling methodology with Rite Aid, and
23 then tested additional sample claims in 2015 and 2016, and also discussed those results with Rite
24 Aid. Id. at 11–12 (citing Yew Decl., ECF No. 46-1, ¶ 7; Yew Decl., ECF No. 47-1, ¶¶ 5–6). The
25
(alterations in original) (citing Salazar Decl., Ex. A, Petron Rep. ¶ 13; Exh. C, Petron Dep. 32:4-
26 12).
27 5 As Petron’s report explains, “confidence interval is an indication of the probable range
of error associated with a sample value obtained from a probability sample.” Petron Rep. ¶ 18
28 n.20.
1 government represents its consistent practice throughout these testing procedures was to review
2 Rite Aid pharmacy records and prescribers’ medical records connected to each sample claim. Id.
3 Based on its investigation, and after settlement discussions failed, the State of California decided
4 to partially intervene in this action, see ECF No. 69; CII. Relator Mr. Schmuckley adopted by
5 reference California’s claims in his first amended complaint. ECF No. 79.
6 B. Procedural Background
7 The State of California partially intervened in this action on September 21, 2017.
8 CII. On March 23, 2018, the court held a hearing on Rite Aid’s motion to dismiss, ECF No. 100,
9 and motion to stay discovery, ECF No. 112, and also addressed case scheduling, ECF No. 122.
10 On May 29, 2018, the court issued its scheduling order, denying Rite Aid’s motion to stay
11 discovery and adopting the parties’ phased discovery plan. Sched. Order, ECF No. 128, at 2–6.
12 Because plaintiffs made clear their intent to use the 1,904-claim sample explained above, along
13 with Rite Aid’s pharmacy records and prescriber medical records, to establish liability and
14 damages under the FCA, the court approved the following plan for phase one of discovery:
15 During the first phase of discovery, plaintiffs posit the parties should
obtain and analyze all of Rite Aid’s prescription records and all
16 relevant third-party medical records concerning the statistical sample
of 1,904 claims. California plans to subpoena medical records in
17 connection with up to an additional 160 sample claims suspected of
involving beneficiaries who did not have the qualifying condition at
18 the time of dispensing. Plaintiffs maintain that Rite Aid is entitled
and has the information and capability to issue its own medical-
19 records subpoenas to support any of its potential defenses.
20 The court approves plaintiffs’ proposal.
21 At the earliest feasible point during this stage of discovery, plaintiffs
will make disclosures concerning their statistics experts and the
22 design of the statistical sample so that Rite Aid can conduct
discovery concerning the same. Rite Aid will file any motions
23 directed toward the viability of the statistical sample during this stage
of discovery so that preparation of the case for trial will not be
24 significantly delayed if it becomes necessary to redraw the statistical
sample.
25
26 Sched. Order at 5.
27 On April 15, 2019, Rite Aid moved to exclude plaintiffs’ sampling methodology
28 arguing, among other things, plaintiffs’ methodology is unreliable and fails to comply with “total
1 survey design” principles. See generally Sampling Mot. California opposes the motion, Opp’n to
2 Sampling Mot., and Rite Aid has replied, Sampling Reply, ECF No. 210. Additionally, as noted
3 above, each party filed supplemental briefing in response to two narrow questions posed by the
4 court at hearing, as discussed below. See Pls.’ Supp. Resp., ECF No. 233; Def.’s Supp. Resp.,
5 ECF No. 235. The United States, as real party in interest, also filed a statement of interest, under
6 28 U.S.C. § 517,6 addressing the court’s questions. See U.S. Statement of Int., ECF No. 232.
7 Rite Aid also moves to exclude and strike as untimely evidence and expert opinion
8 the government has submitted in opposition to Rite Aid’s sampling motion. Def.’s Mot. Strike.
9 Plaintiffs jointly oppose the motion, Opp’n to Def.’s Mot. Strike, ECF No. 216, and Rite Aid has
10 replied, Reply to Def.’s Mot. Strike, ECF No. 229.
11 Finally, plaintiffs move to exclude portions of the expert report of Roy J. Epstein,
12 Ph.D., arguing, among other things, his opinions address measurement validity rather than
13 sampling validity, with only the latter relevant to phase one discovery. Pls.’ Mot. Exclude. Rite
14 Aid has opposed this motion, Opp’n to Pls.’ Mot. Exclude, ECF No. 215, and plaintiffs’ have
15 replied, Reply to Pls.’ Mot. Exclude, ECF No. 228.
16 II. LEGAL STANDARD
17 Under Federal Rule of Evidence 701, a witness is authorized to provide opinion
18 testimony that is “(1) rationally based on the witness’s perception, and (2) helpful to clearly
19 understanding the witness’s testimony or to determining a fact in issue.” Fed. R. Evid. 701. If an
20 opinion witness’s testimony is based on “scientific, technical, or other specialized knowledge,”
21 admissibility of the opinion is governed by Rule 104, a general rule regarding preliminary
22 questions a court must address, and Rule 702, the rule governing expert opinions. Fed. R. Evid.
23 104, 702. Rule 702 provides that a witness who is a qualified expert based on “knowledge, skill,
24 experience, training, or education” may give opinion testimony if certain prerequisites supporting
25 the expert’s testimony are met. Fed. R. Evid. 702. Taken together, Rules 104 and 702 focus
26 6 “The Solicitor General, or any officer of the Department of Justice, may be sent by the
27 Attorney General to any State or district in the United States to attend to the interests of the
United States in a suit pending in a court of the United States, or in a court of a State, or to attend
28 to any other interest of the United States.” 28 U.S.C. § 517.
1 attention on whether the expert witness is qualified to testify, whether such testimony is relevant,
2 and whether such testimony is reliable. Id.; Daubert v. Merrell Dow Pharm., Inc. (“Daubert I”),
3 509 U.S. 579, 594–95 (1993).
4 In assessing whether an expert has the appropriate qualifications, the court
5 considers whether the expert offers some special knowledge, skills, experience, training, or
6 education on the subject matter of the testimony contemplated. Fed. R. Evid. 702; United States
7 v. Hankey, 203 F.3d 1160, 1168 (9th Cir. 2000). If an expert is not qualified to render an opinion
8 on a particular question or subject, it follows his opinion cannot assist the trier of fact with regard
9 to that particular question or subject. Morin v. United States, 534 F. Supp. 2d 1179, 1185
10 (D. Nev. 2005), aff’d, 244 F. App’x 142 (9th Cir. 2007) (“Just as a lawyer is not by general
11 education and experience qualified to give an expert opinion on every subject of the law, so too a
12 scientist or medical doctor is not presumed to have expert knowledge about every conceivable
13 scientific principle or disease.”). In assessing whether the expert’s testimony will be relevant, the
14 opinion must “logically advance[] a material aspect of the proposing party’s case.” Daubert v.
15 Merrell Dow Pharm., Inc. (“Daubert II”), 43 F.3d 1311, 1315 (9th Cir. 1995). The basic
16 standard of relevance is a liberal one. Daubert I, 509 U.S. at 587.
17 Scientific evidence is reliable if the principles and methodology used by the expert
18 proffering it are supported by “appropriate validation” or “good grounds.” Id. In Daubert I, the
19 Supreme Court provided a non-exhaustive list of factors for determining whether scientific
20 testimony is sufficiently reliable to be admitted into evidence, including (1) whether the theory or
21 methodology can be and has been tested; (2) whether “the theory or technique has been subjected
22 to peer review and publication”; (3) the “known or potential rate of error”; (4) the “existence and
23 maintenance of standards controlling” the methodology’s operation; and, finally, (5) general
24 acceptance in the relevant community. Id. at 593–94.
25 Daubert II elaborated on the Daubert I factors, clarifying that experts may
26 demonstrate scientific reliability of a theory or methodology by showing “the research and
27 analysis supporting the proffered conclusions have been subjected to normal scientific scrutiny
28 through peer review and publication.” Daubert II, 43 F.3d at 1318. Alternatively, testifying
1 experts may also show the validity of a theory by explaining “precisely how [the experts] went
2 about reaching their conclusions and point[ing] to some objective source—a learned treatise, the
3 policy statement of a professional association, a published article in a reputable scientific journal
4 or the like—to show that they have followed the scientific method, as it is practiced by (at least) a
5 recognized minority of scientists in their field.” Id. at 1319.
6 In determining reliability, “the expert’s bald assurance of validity is not enough,”
7 id. at 1316, a rule meant to ensure “junk science” is kept out of the federal courtroom. Id. at 1321
8 n.18. Rather, “the party presenting the expert must show that the expert’s findings are based on
9 sound science, and this will require some objective, independent validation of the expert’s
10 methodology.” Id. at 1316. The trial court is accorded wide discretion when acting as a
11 gatekeeper for the admissibility of expert testimony. Kumho Tire Co., Ltd. v. Carmichael, 526
12 U.S. 137, 151–52 (1999).
13 III. DISCUSSION
14 A. Use of Statistical Sampling to Prove Liability in FCA and CFCA Cases
15 At the July 15, 2019 motion hearing, Rite Aid raised a threshold question of
16 whether it is appropriate to use statistical sampling as an evidentiary tool to establish falsity under
17 the FCA and CFCA. See July 15 Hr’g Tr. at 4:13–8:19. At the conclusion of the hearing, the
18 court provided the parties opportunity to file supplemental briefs addressing the question, along
19 with a secondary question addressed below. See Pls.’ Supp. Resp.; Def.’s Supp. Resp.; see also
20 U.S. Statement of Int. Having considered the parties’ supplemental briefs, the arguments at
21 hearing and the circumstances in this matter, the court finds plaintiffs’ proposed statistical
22 sampling plan is a permissible approach in attempting to prove falsity as required to prevail on
23 their FCA and CFCA claims.
24 Under the FCA and CFCA, plaintiffs must establish the following elements: “(1) a
25 false statement or fraudulent course of conduct, (2) made with scienter, (3) that was material,
26 causing (4) the government to pay out money or forfeit moneys due.” U.S. ex rel. Hendow v.
27 Univ. of Phoenix, 461 F.3d 1166, 1174 (9th Cir. 2006); United States ex rel. Afionyan v.
28 Pedorthic Lab Specialist Custom Shoe Co., 781 F. App’x 671, 672 n.1 (9th Cir. 2019) (“The
1 CFCA is based on the FCA and its elements are the same for the claims alleged.” (citing Cal.
2 Gov’t Code § 12651(a)(1–2); Laraway v. Sutro & Co., 96 Cal. App. 4th 266 (2002)). Plaintiffs
3 intend to use “statistical sampling to prove the element of false statements or false conduct.” Pls.’
4 Supp. Resp. at 2. They plan to do so by “estimating the number of, percentage of, and total
5 payments associated with false claims made by Rite Aid to Medi-Cal for Code 1 drugs.” Id.
6 (quotation marks omitted). They “will then extrapolate the findings of false sample claims
7 (which also constitute overpayments).” Id. at 3. Thereafter, plaintiffs will use “universal”
8 evidence applicable to all false claims to prove the elements of materiality and scienter. Id. Rite
9 Aid argues plaintiffs’ “measurement of the FCA’s falsity element alone is not a proper use of
10 statistical sampling” because it lacks Ninth Circuit support, undermines the government’s
11 evidentiary burden under the FCA and is rife with measurement error. Def.’s Supp. Resp. at 2
12 (emphasis in original).
13 The use of statistical sampling in FCA cases is nothing new. See United States v.
14 Life Care Centers of Am., Inc. (Life Care I), 114 F. Supp. 3d 549, 560–65 (E.D. Tenn. 2014)
15 (surveying development of statistical sampling in FCA cases). Although some courts have been
16 reticent to fully embrace the practice of statistical sampling in the FCA context, id. at 560–62, the
17 Ninth Circuit has generally permitted the practice when evaluating Medi-Care claims, for some
18 time. See Ratanasen v. State of Cal., Dep’t of Health Servs., 11 F.3d 1467, 1471 (9th Cir. 1993)
19 (“We now join other circuits in approving the use of sampling and extrapolation as part of audits
20 in connection with Medicare and other similar programs, provided the aggrieved party has an
21 opportunity to rebut such evidence.”).
22 The purpose of sampling is to “provide a means of determining the likelihood that
23 a large sample shares characteristics of a smaller sample.” United States v. Rosin, 263 F. App’x
24 16, 29 (11th Cir. 2008) (citing Laurens Walker & John Monahan, Sampling Evidence at the
25 Crossroads, 80 S. Cal. L. Rev. 969, 973–74 (2007)). As long as a proposed sample meets the
26 reliability standards of Rule 702, then courts will “place[] the burden of evaluating the weight of a
27 statistical sample on the fact finder.” Life Care I, 114 F. Supp. 3d at 560. A sample’s proponent
28 does not use the sample as conclusive proof of the fact or element for which it is offered, but as a
1 body of statistical evidence from which the factfinder may draw an inference. Id. Naturally, the
2 most appropriate way for a party to undermine a sample’s legitimacy is “through cross-
3 examination of the proponent’s expert, presentation of its own expert, as well as other competing
4 witnesses and evidence.” Id. The factfinder may then consider the sample, and any risk of
5 uncertainty therein, and assess how much weight to afford it. Id. That sampling has been
6 acknowledged as a viable method of attempted proof does not relieve the proponent of sampling
7 from satisfying Rule 702’s reliability standards or the court from exercising its inherent
8 gatekeeping role. See In re Countrywide Fin. Corp. Mortg.-Backed Sec. Litig., 984 F. Supp. 2d
9 1021, 1031 (C.D. Cal. 2013) (court exercised gatekeeping role sua sponte by addressing
10 deficiency in expert report).
11 The district court in Life Care I addressed the use of statistical sampling to prove
12 falsity under the FCA. In that case, defendant advanced essentially the same argument Rite Aid
13 advances here: “that it would be inappropriate for the Government to prove liability for its FCA
14 claims through statistical sampling because the determination of whether therapy is medically
15 necessary for a particular patient requires an individual assessment of the patient’s clinical
16 condition.” Life Care I, 114 F. Supp. 3d at 566 (internal quotation marks omitted). In other
17 words, the defendant argued that, under the circumstances of that case, falsity required a “fact-
18 intensive, subjective determination” because it was influenced by multiple factors for which the
19 proposed statistical sampling method did not account. Id. at 565–66. The court concluded that
20 while “these factors exist and are likely unique to each patient[, that] does not necessarily
21 preclude the use of statistical sampling.” Id. at 566. The court observed that defendant’s
22 argument actually highlighted the value of statistical sampling in the case, given “that a smaller
23 portion of claims will be used to draw an inference about a larger, not entirely identical,
24 population of claims.” Id. “If all the claims were exactly the same in every respect, there would
25 be no need for statistical sampling and extrapolation . . . because each individual unit would be
26 identical.” Id. As the court in Life Care I recognized, a chief benefit of statistical sampling is
27 avoiding the “paralysis” that can be caused by wading through thousands of individual claims,
28 United States v. Rogan, 517 F.3d 449, 453 (7th Cir. 2008), by instead providing information from
1 which a factfinder, if persuaded, can draw a reasonable inference from smaller data sets in
2 considering a larger universe of claims.
3 In United States v. Robinson, a Kentucky district court applied reasoning similar to
4 that in Life Care I in rejecting the defendant’s contention that expert testimony based on a 30-
5 claim sample, drawn from a pool of over 25,000 claims, was insufficient to prove falsity. No. 13-
6 CV-27-GFVT, 2015 WL 1479396, at *6–7 (E.D. Ky. Mar. 31, 2015). Considering the issue at
7 the summary judgment stage, the court in Robinson drew a key distinction:
8 The question is not whether [the expert] opinion testimony about 30
examinations proves the lack of medical necessity beyond all doubt,
9 but whether it creates an issue of material fact that should be
submitted to a jury. Although [expert testimony] alone may not have
10 proved definitively that each of the over 25,000 claims at issue were
unreasonable or unnecessary, such proof is unnecessary at this stage
11 of litigation. The United States simply must present evidence of a
genuine issue of a material fact, and [expert] opinion testimony at the
12 very least creates a genuine dispute concerning the necessity of the
30 claims he reviewed, and also as to whether [defendant] acted with
13 reckless disregard to the truth in his billing practices. Such evidence
fulfills the government’s affirmative duty in this matter.
14
15 Id., at *6.
16 The takeaway from Life Care I, Robinson and Rogan is that statistical sampling is
17 a viable evidentiary tool for organizing voluminous information and then arguing for the drawing
18 of reasonable inferences with respect to that information. It is not, however, a tool that will
19 conclusively resolve questions of fact or alleviate a plaintiff’s burden of proof under the FCA.
20 Both Life Care I and Robinson were decided at the summary judgment stage; consequently, those
21 courts emphasized that while the sampling in those cases was permissible and sufficient to raise a
22 triable question of fact to survive summary judgment, defendants would still have the opportunity
23 to challenge the weight a factfinder should afford a given data sample as well as any extrapolation
24 from the sample urged by the sampling party. See Life Care I, 114 F. Supp. 3d at 567 (“If
25 Defendant wishes to challenge the weight that a fact finder may attribute to the extrapolation, it
26 can employ cross-examination and competing witnesses and testimony to highlight the disparity
27 between claims.”); Robinson, 2015 WL 1479396, at *6 (“[T]he issue raised by [the expert’s]
28 opinion evidence is one of credibility and of the weight that should be given to his opinion.”); see
1 also United States v. Long Grove Manor, Inc., No. 10 C 368, 2019 WL 2774149, at *5 (N.D. Ill.
2 July 2, 2019) (reviewing Life Care I, Robinson and Rogan and noting “[a]t most, these decisions
3 hold only that a relator need not demonstrate the falsity of every particular claim, not that it is
4 unnecessary to demonstrate the falsity of any particular claim” (emphases in original)).
5 Rite Aid cites U.S. ex rel. Michaels v. Agape Senior Cmty., Inc., No. CA 0:12-
6 3466-JFA, 2015 WL 3903675, at *8 (D.S.C. June 25, 2015) and United States v. Vista Hospice
7 Care, Inc. (“Vistacare”), No. 3:07-CV-00604-M, 2016 WL 3449833, at *11 (N.D. Tex. June 20,
8 2016), for the proposition that the use of statistical sampling is improper to establish liability
9 where potential measurement error exists and a falsity determination requires a subjective, claim-
10 specific evaluation. Def.’s Supp. Resp. at 3. In other words, Rite Aid argues that Agape and
11 Vistacare show “sampling is improper when reasonable disagreement exists as to what is or is not
12 a false claim.” Id.
13 These cases, however, are distinguishable. In Agape, the court reexamined the
14 propriety of using statistical sampling to establish damages where the government, an admitted
15 non-party to the FCA qui tam action, was attempting to foil a settlement agreement between
16 relator-plaintiffs and defendant because it believed the settlement represented a gross
17 undervaluation compared to the government’s damage projections. 2015 WL 3903675, at *2–3.
18 Having excluded the government’s proposed sampling and extrapolation model earlier in the
19 litigation, the court again found, in the context of the objection to settlement, that the proposed
20 sampling in that case was inappropriate because each claim “present[ed] the question of whether
21 certain services furnished to nursing home patients were medically necessary . . . [which] is [a]
22 highly fact-intensive inquiry involving medical testimony after a thorough review of the detailed
23 medical chart of each individual patient.” Id. at *8.
24 Similarly, Vistacare involved alleged false claims related to reimbursement for
25 Medicare Hospice Benefits (“MHB”). 2016 WL 3449833, at *1. In the case, eligibility for MHB
26 reimbursement was contingent upon an individual’s prognosis taking into account “diagnoses and
27 all other things that relate to a patient’s life expectancy.” Id. at *3. Because this evaluation was
28 “inherently subjective, patient-specific, and dependent on the judgment of involved physicians,”
1 the court excluded the expert’s testimony pertaining to claims not individually reviewed. 2016
2 WL 3449833, at *11.
3 The case at bar does not implicate the subjective factors animating the Agape and
4 Vistacare decisions. Plaintiffs characterize the Code 1 evaluation processes as relatively simple.
5 Regarding Rite Aid’s business prescription records, the sample plaintiffs will rely on examines
6 “whether or not [the records] notate that Rite Aid performed the requisite review, verification,
7 and documentation of the approved Code 1 diagnosis at the time of dispensing before Rite Aid
8 submitted the sample claim for payment by using override codes.” Pls.’ Supp. Resp. at 5. A
9 missing notation, plaintiffs argue, suggests Rite Aid failed to perform its Code 1 duties. Id. As to
10 prescriber records, plaintiffs characterize the evaluation as a simple question of whether “the
11 prescriber’s records indicate that the beneficiary had the approved Code 1 diagnosis at the time of
12 the subject prescription[.]” Id. “If the answer is ‘no,’ it can reasonably be inferred that the
13 beneficiary did not have the approved Code 1 diagnosis and Rite Aid therefore should not have
14 overridden the initial denial of the claim.” Id. Plaintiffs appear to fairly characterize their
15 sampling review process, based on the record currently before the court. Moreover, Rite Aid will
16 have ample opportunity as the case moves forward to dispositive motion practice and trial to
17 attack plaintiffs’ evaluations of the sample data, including plaintiffs’ position that their method is
18 simple and straightforward.
19 The court also notes the support of the United States, as a real party in interest, for
20 the use of statistical sampling in this matter. See U.S. Statement of Int. at 2–3 (collecting cases
21 utilizing statistical sampling and “ask[ing] th[e] Court to decline ruling against California’s use of
22 statistical sampling to demonstrate details regarding false claims”). The United States’ position
23 bolsters the court’s conclusion that plaintiffs’ proposed use of statistical sampling is appropriate
24 here. See Long Grove Manor, Inc., 2019 WL 2774149, at *5 (“[C]ourts have allowed the use of
25 statistical sampling in some FCA cases – particularly those involving very large numbers of
26 allegedly false claims.”); Chaves Cty. Home Health Serv., Inc. v. Sullivan, 931 F.2d 914, 919
27 (D.C. Cir. 1991) (“[C]ourts have routinely permitted the use of statistical sampling to determine
28
1 whether there has been a pattern of overpayments spanning a large number of claims where case-
2 by-case review would be too costly.”).
3 In sum, the court finds that plaintiffs’ proposed use of statistical sampling is
4 allowable in their effort to prove the falsity element of plaintiffs’ FCA and CFCA causes of
5 action.
6 B. Rite Aid’s Motion to Exclude Plaintiffs’ Sampling Methodology
7 In light of the court’s ruling that plaintiffs are permitted to use statistical sampling
8 in an effort to prove their case, the question then becomes whether they may rely on their
9 proposed specific methodology, stratified random sampling and total survey design. It is this
10 question that occupied much of the parties’ argument at hearing. While the use of sampling
11 generally and the precise methodology raise separate questions, the arguments and authority
12 overlap and the court draws on similar reasoning in resolving both. As explained below, the court
13 denies Rite Aid’s motion and approves plaintiffs’ use of stratified random sampling.
14 While Rite Aid makes a number of arguments addressed below, Rite Aid’s motion
15 to exclude plaintiffs’ sampling methodology is premised essentially on three general grounds: (1)
16 the methodology fails to account for measurement error; (2) it fails to accurately assess Code 1
17 violations and Michael Petron, plaintiff’s statistical expert, never examined how the methodology
18 was developed; and (3) Petron concedes the sample may need to be redesigned should the court’s
19 ultimate interpretation of California Code of Regulations, title 22, section 51476(c) undermine the
20 validity of plaintiffs’ methodology. Sampling Mot. at 13–20. In response, plaintiffs contend the
21 scope of Petron’s report was specifically limited to addressing the question of sampling
22 methodology given the early stages of discovery; thus, any argument related to data collection,
23 analysis and estimation are irrelevant to the validity of Petron’s expert opinion. Opp’n to
24 Sampling Mot. at 14–16. As to the sampling methodology itself, plaintiffs contend its validity is
25 based on its clearly stated objective and the statistically verified methods used in alignment with
26 that objective, including the use of three well-defined data universes, single sample units within
27 the overall sample objective, use of stratified random sampling, adequate sample sizes and
28
1 representative sample frames. Id. at 16–23. Taken together, plaintiffs argue their stratified
2 random sampling methodology is valid, reliable and admissible. Id. at 23.
3 Ensuring the reliability of testing methodologies, and expert testimony derived
4 therefrom, “is often the most challenging step in the Rule 702 inquiry.” In re Countrywide, 984
5 F.Supp. 2d at 1026. In performing this exercise, it is important to remember “[t]he Daubert
6 standard does not exist to ensure that only the most ideal scientific evidence is admissible in court
7 proceedings, but instead to ensure that expert testimony is ‘derived by the scientific method.’” Id.
8 at 1036 (quoting Daubert I, 509 U.S. at 590) (citing Deutsch v. Novartis Pharms. Corp., 768 F.
9 Supp. 2d 420, 431 (E.D.N.Y. 2011) (“Under Daubert, an expert need not base his opinion on the
10 best possible evidence, but upon ‘good grounds, based on what is known.’”)). The court must
11 also “keep in mind the Supreme Court’s admonition that ‘[v]igorous cross-examination,
12 presentation of contrary evidence, and careful instruction on the burden of proof are the
13 traditional and appropriate means of attacking shaky but admissible evidence.’” Massachusetts
14 Mut. Life Ins. Co. v. Residential Funding Co., LLC, 989 F. Supp. 2d 165, 171 (D. Mass. 2013)
15 (alteration in original) (quoting Daubert I, 509 U.S. at 596).
16 Rite Aid argues plaintiffs’ proposed methodology should be excluded because it
17 “fails to use the federally-endorsed methodology to assess patients’ claim histories: cluster
18 sampling.” Sampling Mot. at 1. If they had used this methodology, a patient’s medical and
19 prescription history would be considered in determining whether a claim was deemed false under
20 the Code 1 reimbursement process. Id. at 11. This failure is critical, Rite Aid argues, because it
21 “‘will likely exclude relevant information[,] lead to erroneous conclusions,’ and ultimately
22 ‘inflate an estimate of allegedly false claims in the universe.’” Id. at 12 (alteration in original)
23 (quoting Epstein Rep. ¶ 47). In response, plaintiffs argue that a patient’s history is irrelevant
24 considering the test objective, which is “to estimate the number of false Code 1 diagnosis-
25 restricted claims submitted by use of override codes without the proper review, verification, and
26 documentation.” Opp’n to Sampling Mot. at 20. In other words, each individual claim becomes
27 a sampling unit, and a patient’s medical history is inconsequential to whether a Rite Aid associate
28 performed the necessary review prior to deploying the override code. Id.
1 1. Has Plaintiffs’ Sampling Methodology Been Tested?
2 In fulfilling its gatekeeping function under Rule 702, and guided by the
3 admonition that under Daubert “the rejection of expert testimony is the exception rather than the
4 rule,” the court finds that plaintiffs’ proposed use of stratified random sampling is sufficiently
5 reliable at this stage in the litigation. Frye v. Warden, San Quentin State Prison, No. CIV S-99-
6 0628, 2010 WL 3210767, at *1 (E.D. Cal. Aug. 10, 2010) (quoting Fed. R. Evid. 702, advisory
7 committee note to 2000 amendment). Under the first Daubert factor, the court asks whether the
8 methodology can be and has been tested. 509 U.S. at 593. As plaintiffs argue, stratified random
9 sampling is an established and oft-used sampling technique. See U.S. ex rel. Martin v. Life Care
10 Centers of Am., Inc. (Life Care II), No. 1:08-CV-251, 2014 WL 4816006, at *14 (E.D. Tenn.
11 Sept. 29, 2014) (“a stratified random sample ‘is one obtained by separating the population
12 elements into nonoverlapping groups, called strata, and then selecting a simple random sample
13 from each stratum.’” (citing Richard L. Scheaffer, William Mendenhall, R. Lyman Ott & Kenneth
14 G.Gerow, Elementary Survey Sampling)). In reviewing plaintiffs’ methodology, Petron cited
15 numerous authoritative texts detailing the validity of the testing methods plaintiffs deployed. See,
16 e.g., Petron Rep. ¶ 9 n.4 (citing Särndal, Swensson & Wretman, Model Assisted Survey Sampling,
17 Springer-Verlag (1992)), ¶ 16 n.15 (citing Cochran & William, Sampling Techniques, John Wiley
18 & Sons, Inc. (3rd ed. 1977)), ¶ 17 n.16 (citing Arkin & Herbert, Handbook of Sampling for
19 Auditing and Accounting, Prentice Hall (3rd ed. 1984)). Rite Aid does not question the scientific
20 validity of stratified sampling, rather, it questions its use here under the conditions of the Code 1
21 reimbursement system because, it says, cluster sampling is a preferred method when a patient’s
22 claim history is required. Sampling Mot. at 11. This may be true, but plaintiffs have
23 continuously argued that, consistent with the sample objective, each individual claim is evaluated
24 in isolation. Petron Rep. ¶ 14; Opp’n to Sampling Mot. at 20. As explained above, the court has
25 accepted plaintiffs’ representation of this simplified determination process; in this process, cluster
26 sampling is unnecessary because a patient’s history will not help determine whether a Rite Aid
27 associate has fulfilled his or her responsibilities in using an override code to verify a Code 1
28 claim.
1 Rite Aid also contends that plaintiffs have violated the conditions of total survey
2 design, arguing plaintiffs have not specified either the objective of the methodology or the
3 information necessary to achieve the objective. Sampling Mot. at 9. Thus, Rite Aid argues,
4 because Petron was not actually involved in the development of the methodology itself, his report
5 is devoid of any reliable opinion regarding “how the relevant information for the sample was
6 selected or even what it precisely is.” Id. at 10 (quoting Epstein Rep. ¶ 13). Here too Rite Aid
7 overstates. Petron’s report explicitly lays out the process of total survey design, including its
8 requirements, survey specifications and survey operations, and walks through each element in his
9 examination of the plaintiffs’ design. Petron Rep. ¶¶ 9–25. Petron was tasked specifically with
10 evaluating whether the sample methodology developed by the government was statistically valid.
11 Id. ¶ 5. This determination was based on Petron’s professional expertise in total survey design,
12 not whether he himself participated in the development of the survey methodology, or even
13 whether he was fully apprised of the all the data the government considered in its stratification
14 process. Petron considered how the sample sets were developed through the stratification process
15 and the 95 percent confidence level used and recreated the sample sizes for each stratum to test
16 their viability. Id. ¶¶ 12–22. Through this process, Petron reached the conclusion that plaintiffs’
17 “methodology is statistically valid and should produce reliable estimates of the number of,
18 percentage of, and payments associated with false claims made by Rite Aid to Medi-Cal for Code
19 1 drugs.” Id. ¶ 23.
20 Even if, as Rite Aid contends, plaintiffs’ methodology presupposes what
21 constitutes a “false” Code 1 claim, either under the applicable regulatory provisions, see Cal.
22 Code. Regs, tit. 22, section 51476(c), or that is a key factual determination requiring resolution by
23 the factfinder. See Life Care I, 114 F. Supp. 3d at 560. If there are holes in plaintiffs’
24 methodology for failing to consider certain variables, then Rite Aid will have the chance to
25 vigorously expose those flaws through cross-examination, casting doubt on the weight to be given
26 to the sample. Id. At this stage in the litigation, however, the court’s role is to assess the
27 scientific reliability of plaintiffs’ chosen methodology, not determine whether that methodology
28
1 can or will satisfy the legal elements of plaintiffs’ case. The court finds plaintiffs methodology
2 satisfies the first Daubert regarding whether the methodology can be and has been tested.
3 2. Potential Error Rate of Plaintiffs’ Sampling Methodology
4 The next Daubert factor requires the court to consider the methodology’s potential
5 error rate. 509 U.S. at 594. Rite Aid argues plaintiffs’ methodology is invalid because it fails to
6 account for measurement error and therefore will not calculate valid confidence intervals.
7 Sampling Mot. at 14. In opposition, plaintiffs argue that within the concept of total survey
8 design, measurement error is calculated during the data collection and data processing stages, and
9 with those two stages still ongoing in the first phase of discovery and exacerbated by Rite Aid’s
10 delayed production of certain medical records, plaintiffs cannot yet account for measurement
11 error. Opp’n to Sampling Mot. at 22.
12 Plaintiffs are correct: plaintiffs’ methodology is valid because measurement error
13 will largely be calculated as data unfolds and plaintiffs cross-check sampling data against the
14 individual medical records associated with each claim. See id. at 11–12. Moreover, Rite Aid’s
15 claim that measurement error cannot be corrected on the “back end” pertains to necessary
16 adjustments if cluster sampling were used, not whether error rate can be determined using
17 plaintiffs’ stratified samples as currently formulated.7 See Reply to Sampling Mot. at 6 (“[A]
18 decision to utilize cluster sampling cannot be made on the “back end.” (emphasis added)).
19 Finally, to the degree the measurement error argument challenges the creation of the sampling
20 design, Petron’s report specifically considered the sample’s 95 percent confidence interval in
21 evaluating the overall validity of the design. Petron Rep. ¶ 18. Taking all of these factors into
22 consideration, the court does not find plaintiffs’ pending error rate measurement categorically
23 invalid. This Daubert factor weighs in plaintiffs’ favor.
24
25
26
27
7 At hearing plaintiffs confirmed that stratified random sampling will be their sampling
method of choice for the duration of this action and they will not later adjust sampling methods if
28 the stratified method yields unfavorable results. July 15 Hr’g Tr. at 8:20–9:7; id. at 29:9–13.
1 3. Other Daubert Factors
2 Finally, the remaining Daubert factors—whether “the theory or technique has been
3 subjected to peer review and publication,” the “existence and maintenance of standards
4 controlling” the methodology’s operation, and the methodology’s general acceptance in the
5 scientific community—are easily satisfied here. 509 U.S. at 593–94. As discussed, stratified
6 random sampling and total survey design are well-established, well-supported testing methods
7 typical of homogeneous data groups such as these, see also Fed. Jud. Ctr., Ref. Manual on Sci.
8 Evid. 299 (3rd ed. 2011), and Petron’s report is replete with citations to and reliance on reputable
9 texts in the field of statistical sampling, see generally Petron Rep. The validity of plaintiffs’
10 methods is confirmed by their vetting in the scientific community.
11 The court finds plaintiffs’ sampling methodology permissible. Rite Aid’s motion
12 to exclude plaintiffs’ sampling methodology is DENIED.
13 C. Viability of Plaintiffs’ Symmetry Sampling Frame
14 Having approved plaintiffs’ proposed sampling methodology, the court turns to the
15 second question it posed at hearing: should the court permit use of the Symmetry sampling frame
16 given Rite Aid’s claim it could not recreate the frame?
17 Rite Aid contends the Symmetry sample is incapable of replication and therefore
18 should be excluded for lack of reliability. Def.’s Supp. Resp. at 8–10. Thus, Rite Aid argues, the
19 claims universe from which the Symmetry sample was drawn must also be excluded. Id.
20 Excluding the Symmetry claims universe would not impact use of the Off-Formulary (“OF”) and
21 Diagnosis Related (“DR”) samples. Id. Plaintiffs argue they provided Rite Aid with ample
22 documentation as needed to recreate the Symmetry sample; were the court to adopt Rite Aid’s
23 reasoning for excluding the Symmetry sample, it would adversely affect future proponents of
24 probability sampling through use of software technology. Pls.’ Supp. Resp. at 8–10. The United
25 States similarly asserts Rite Aid has received “sufficient information to understand the relatively
26 simple function performed by the Symmetry program to create the sample frame.” U.S.
27 Statement of Int. at 4.
28
1 Rite Aid’s inability to replicate the Symmetry frame does not require its exclusion.
2 Given the court’s findings above regarding the admissibility of plaintiffs’ sampling methodology,
3 Rite Aid’s position is that the Symmetry program is incapable of reproducing data from the
4 timeframe used to produce the relevant sample. See U.S. Statement of Int. at 4 (fairly
5 characterizing Rite Aid’s argument as such). In other words, having found the methodology used
6 to produce the Symmetry sample acceptable, does the inability to independently replicate the
7 sample render the sample unreliable? It does not.
8 It is true, as Rite Aid asserts, that to ensure reliability under Daubert, “an expert’s
9 opinions ‘must be testable and someone else using the same data and methods must be able to
10 replicate the result.” Def.’s Supp. Resp. at 8 (alterations omitted) (quoting Zenith Electronics
11 Corp. v. WH-TV Broadcasting Corp., 395 F.3d 416, 419 (7th Cir. 2005)). However, where
12 scientifically accepted methods are used to produce data, and the techniques used in the process
13 are within a range of acceptable practice, the reliability of the data produced becomes a question
14 of evidentiary weight, not admissibility. City of Pomona v. SQM N. Am. Corp., 750 F.3d 1036,
15 1046 (9th Cir. 2014) (“The question is whether an expert’s methodology can be ‘challenged in
16 some objective sense, or whether it is instead simply a subjective, conclusory approach that
17 cannot reasonably be assessed for reliability.’” (quoting Fed. R. Evid. 702 advisory committee
18 note to 2000 amendment)).
19 In City of Pomona, for example, the Ninth Circuit overruled a district court’s
20 exclusion of expert testimony because other experts in the field had tested the same
21 methodologies from a Guidance Manual8 relied on by the expert, Dr. Sturchio, the procedures
22 were subject to retesting by another laboratory and the results obtained through the expert’s
23 techniques were best left to a factfinder’s consideration. 750 F.3d 1036, 1047 (9th Cir. 2014).
24 Specifically on the subject of retesting, respondents argued Dr. Sturchio’s testing could not be
25 replicated because he “failed to duplicate columns in collecting groundwater samples” and “failed
26
27
8 “Guidance Manual” in the case referred to the Guidance Manual for Forensic Analysis
of Perchlorate in Groundwater using Chlorine and Oxygen Isotopic Analyses. City of Pomona,
28 750 F.3d at 1042.
1 to take split samples in order to compare analytical results.” Id. The court rejected these
2 arguments because neither technique was “required” under the Guidance Manual. Id. Moreover,
3 the court noted that although “Dr. Sturchio failed independently to verify his test results with a
4 separate lab[,] [t]h[at] point . . . may serve to undermine or impeach the weight that should be
5 afforded to [his] testimony, but it does not refute the scientific reliability of his analysis.” Id.
6 Here, two points support allowing plaintiffs to rely on the Symmetry sample.
7 First, recommendations promulgated by the federal Centers for Medicare and Medicaid Services
8 (“CMS”) and the federal Office of Inspector General (“OIG”) regarding sampling recreation and
9 documentation advise that “[f]ailing to keep sufficient records to replicate a statistical sample”
10 will not “necessarily render a sample invalid, but it can make the resulting estimate more difficult
11 to defend.” See Pls.’ Supp. Resp. at 8 (citing U.S. Dep’t of Health and Human Servs. – Ofc. of
12 the Inspector Gen., Statistical Sampling: A Toolkit for MFCUs (Sept. 2018)9). Although not
13 binding on this court, practice guidance regarding statistical sampling from the U.S. Department
14 of Health and Human Services, an authoritative body and home to the CMS, suggests an inability
15 to replicate does not render a sample invalid per se; rather, replicability speaks to the degree of
16 reliance a factfinder may assign. City of Pomona, 750 F.3d at 1047 (“Daubert . . . does not forbid
17 admission of a report where the weight of the conclusions are subject to challenge.” (internal
18 quotations and citation omitted)).
19 Second, it appears duplication of the Symmetry sample is impracticable, not
20 necessarily impossible. See Petron Dep. at 246:3–19 (“[Replication] would have been extremely
21 expensive and time-consuming to have me to recreate a sample frame that, frankly, I don’t have
22 an opinion on even if I could recreate.”); Gonzales Decl. ¶ 19 n.4, ECF No. 204-15 (“Symmetry
23 can only go through the last 4 full calendar years of a claim database.”); Opp’n to Sampling Mot.
24 at 5 (noting “Rite Aid’s . . . inherent inability to replicate the Symmetry data pull”). The
25 infeasibility of data replication does not render the methodology used to produce that data
26
9 Available at https://oig.hhs.gov/fraud/medicaid-fraud-control-units-
27
mfcu/files/MFCU%20Sampling%20Guidance%20Final.pdf
28
1 unreliable. As both plaintiffs and the United States highlight, plaintiffs recorded and produced all
2 documentation related to the Symmetry data pull consistent with CMS and the OIG guidelines.
3 Pls.’ Supp. Resp. at 8 (citing Medicare Program Integrity Manual, ECF No. 204-16); U.S.
4 Statement of Int. at 4. Plaintiffs’ interrogatory responses detailed the Symmetry sample creation
5 and methodology, Interrog. Resp. No. 1, ECF No. 204-4; Petron’s report explained his degree of
6 involvement and reliance on the pre-formulated Symmetry pull and included sample frames as
7 exhibits to his report, Petron Rep. ¶ 15, Ex. 4.1–4.7; and Petron testified regarding the sample’s
8 validity, Petron Dep. at 246:13–16. Indeed, even Rite Aid’s expert Dr. Epstein testified that the
9 Symmetry sample is representative of the Symmetry frame. Epstein Dep., ECF No. 204-2, at
10 108:18–22. The record regarding Symmetry comports with guidance provided by the Medicare
11 Program Integrity Manual that “units shall document all steps taken in the random selection
12 process exactly as done to ensure that the necessary information is available for anyone
13 attempting to replicate the sample selection.” ECF No. 204-16, § 8.4.4.2 (emphasis added).
14 Although replication here is limited by constraints in the Symmetry software itself, thus making
15 replication efforts extremely expensive, the evidence suggests plaintiffs fundamentally complied
16 with their documentation duties. Any inability to replicate the data can be used in an effort to
17 undermine the weight a factfinder gives that data, but does not require its exclusion. The cases
18 cited by Rite Aid are distinguishable and thus the court is not persuaded by them. See, e.g., Wyatt
19 Tech. Corp. v. Malvern Instruments, Inc., No. CV 07-8298 ABC (MANx), 2010 WL 11505684,
20 at *7 (C.D. Cal. Jan. 25, 2010) (“[O]ffer[ing] no evidence from which [defendant]—or any other
21 scientist—could replicate this experiment.”), aff’d in part, 526 F. App’x 761 (9th Cir. 2013);
22 Zenith Elecs. Corp. v. WH-TV Broad. Corp., 395 F.3d 416, 418 (7th Cir. 2005) (expert opinion
23 based on “supposed ‘uniqueness’ of a market does not justify substituting a guess for careful
24 analysis”); Hutchinson v. Hamlet, No. C 02-974 JSW (PR), 2006 WL 1439784, at *4 (N.D. Cal.
25 May 23, 2006) (finding expert’s experiment did “not comport with proper scientific
26 methodology” and “clearly did not provide for any peer review or scrutiny”).
27 In light of these findings, the court need not reach the contention of plaintiffs and
28 the United States, that excluding the Symmetry sample could have adverse policy implications
1 regarding use of third-party software for probability sampling. See Pls.’ Supp. Resp. at 9–10;
2 U.S. Statement of Int. at 4–5.
3 The Symmetry frame is admissible.
4 D. Remaining Motions to Exclude
5 In light of the above discussion, given the court’s primary reliance on the Petron
6 report in weighing the Daubert factors and finding plaintiffs’ proposed methodology valid,
7 without relying on Dr. Epstein’s report, the court does not at this time reach plaintiffs’ motion to
8 exclude portions of Dr. Epstein’s report and testimony. ECF No. 206. The court also does not
9 reach Rite Aid’s motion to exclude and strike untimely evidence and expert opinion submitted in
10 support of plaintiffs’ opposition to the motion to exclude sampling methodology. ECF No. 208.
11 Rite Aid targets four declarations, those of Meixner, Lien, Gonzales and Yew, which are not
12 themselves the basis of any of the court’s substantive conclusions above.
13 These motions may be renewed in connection with summary judgment or as
14 motions in limine approaching trial, if the parties continue to believe in good faith that they
15 require resolution. Accordingly, the motions are DENIED without prejudice.
16 IV. CONCLUSION
17 For the reasons set forth above, Rite Aid’s motion to exclude plaintiffs’ proposed
18 sampling methodology, ECF No. 195, is DENIED. The court declines to rule on plaintiffs’
19 motion to exclude portions of Dr. Epstein’s report, ECF No. 206, and Rite Aid’s motion to
20 exclude and strike untimely evidence and expert opinion submitted in support of plaintiffs’
21 opposition to the motion to exclude sampling methodology, ECF No. 208. These motions are
22 DENIED without prejudice.
23 IT IS SO ORDERED.
24 DATED: July 13, 2020.
25
26
27
28