The opinion
1
2
3
4
5
6 UNITED STATES DISTRICT COURT
7 NORTHERN DISTRICT OF CALIFORNIA
8
9 IN RE GOOGLE PLAY STORE MDL Case No. 21-md-02981-JD
ANTITRUST LITIGATION
10 Member Case Nos. 20-cv-05761-JD,
21-cv-05227-JD
11
ORDER RE MERITS OPINIONS OF
12 DR. HAL J. SINGER
13
14
15 In this multidistrict antitrust litigation, several plaintiff groups have challenged Google’s
16 Play Store practices. The Play Store is a marketplace that offers millions of apps for devices that
17 use the Android operating system, such as phones and tablets made by Samsung and other original
18 equipment manufacturers. The apps in the Play Store are created and supplied by independent
19 developers, many of whom charge users a fee to acquire the app or in-app content. A central
20 theme in all of the constituent cases of the MDL action is that Google illegally monopolized the
21 Android app distribution market in violation of Section 2 of the Sherman Antitrust Act, which is
22 said to have caused overcharges to consumers and other injuries.
23 This order pertains primarily to the consumers case, In re Google Play Consumer Antitrust
24 Litigation, Case No. 20-cv-05761-JD. The consumers sued Google, LLC, Google Ireland
25 Limited, Google Commerce Limited, Google Asia Pacific Pte. Limited, and Google Payment
26 Corp. as defendants. In keeping with the parties’ practice in the MDL, defendants are referred to
27 collectively as “Google.”
1 The consumer plaintiffs have proffered the opinions of Dr. Hal J. Singer, an economist at
2 the consulting firm, Econ One, and the University of Utah, as an essential part of their case against
3 Google. Dr. Singer previously provided opinion testimony in support of the consumers’ motion to
4 certify a class. After a concurrent expert evidentiary proceeding (known informally as a “hot tub”)
5 in which Dr. Singer exchanged views on key topics with Google’s expert, Dr. Michelle Burtis, an
6 economist at Charles River Associates, the Court denied Google’s motion to exclude Dr. Singer’s
7 opinions, and certified a consumer class. See Dkt. Nos. 302 (Class Cert. Hot Tub Tr.), 383 (Class
8 Cert. Order).1 An appeal of the grant of certification is pending before the circuit court. See In re
9 Google Play Store Antitrust Litigation, Case No. 23-15285 (9th Cir.).
10 The consumer plaintiffs have also asked Dr. Singer to provide opinion testimony at trial on
11 the merits of their antitrust claims against Google. The Court has denied Google’s request to defer
12 or stay the November 6, 2023, jury trial, see Dkt. No. 499, and so proceedings have moved
13 forward to the consideration of motions by Google for partial summary judgment and to exclude
14 the merits opinions of certain experts on the plaintiffs’ side. See Dkt. Nos. 483, 484, 487.2 For the
15 experts, Google has asked to exclude under Rule 702 of the Federal Rules of Evidence (FRE) the
16 merits opinions of Dr. Singer, and of Dr. Marc Rysman, an economist at Boston University
17 retained by the State plaintiffs. See Dkt. Nos. 487 (Singer), 484 (Rysman).3
18 As is the Court’s practice for Rule 702 motions involving complex expert evidence, the
19 Court convened on August 1, 2023, a hot tub focused on the parties’ main disagreements about the
20 admissibility of the merits opinions of Drs. Singer and Rysman. See Dkt. No. 585 (Merits Hot
21
22
1 Unless otherwise noted, all docket number references are to the ECF docket for the MDL, Case
No. 21-md-02981-JD.
23
2 The Match Group plaintiffs have also filed a motion for partial summary judgment on Google’s
24
counterclaims. Dkt. No. 486.
25
3 The record is a bit fuzzy on whether the plaintiff States in State of Utah v. Google LLC, Case
No. 21-cv-05227-JD, intend to rely on Dr. Singer’s opinions at trial. Dr. Singer offers all of his
26
opinions on behalf of the consumer plaintiffs, and a subset on behalf of “the Consumer Plaintiffs
and Plaintiff States.” Dkt. No. 489-2 (Singer Merits Report) ¶ 1. Even so, the Court understands
27
that the States are relying primarily on the proposed testimony of Dr. Rysman, which is
1 Tub Tr.). This time, Google presented Dr. Gregory K. Leonard as its expert economist and not
2 Dr. Burtis, on whom Google had relied for the class certification proceedings. Dr. Leonard is an
3 economist at the consulting firm, Charles River Associates. After the hot tub, the Court posed
4 several questions to Dr. Singer and Dr. Leonard, Dkt. No. 570, which they answered under oath on
5 August 14, 2023. Dkt. Nos. 578, 580.
6 After consideration of the now fully developed record, the merits opinions of Dr. Singer
7 are excluded under FRE 702 and the familiar standards in Daubert v. Merrell Dow
8 Pharmaceuticals, Inc., 509 U.S. 579 (1993). The motion to exclude Dr. Rysman’s merits opinions
9 will be addressed in a separate order.
10 BACKGROUND
11 The Court provided an in-depth background for the litigation in the class certification and
12 expert admissibility order, see Dkt. No. 383 (Class Cert. Order), and will not replow that ground
13 here. The parties’ familiarity with the background is assumed.
14 I. DR. SINGER’S CLASS CERTIFICATION OPINIONS
15 The consumer plaintiffs initially presented Dr. Singer in the class certification proceedings
16 to opine on a proposed method of classwide proof of antitrust impact and damages.4 In an expert
17 report prepared with respect to certification, Dr. Singer identified and analyzed two proposed
18 relevant markets for the consumers’ claims: an Android App Distribution Market and an In-App
19 Aftermarket. See Class Cert. Order at 8. For the Android App Distribution Market, Dr. Singer
20 opined that Google’s “take rate,” meaning the share of revenue Google takes from developers for
21 each app sale, would have fallen from 30.1 percent in actual practice to 23.4 percent in a
22 competitive but-for world. This led Dr. Singer to conclude that Play Store users had paid an
23 average overcharge of $0.30 for each app they purchased, resulting in “aggregate damages of
24 $18.76 million” for the proposed class. Id. at 18. For the In-App Aftermarket, which involves
25
26
4 In his class certification report, Dkt. No. 254-4 (Singer Class Cert. Report), Dr. Singer offered
opinions on other elements of the consumer plaintiffs’ antitrust claims, e.g., that Google has
27
engaged in anticompetitive conduct in the Android App Distribution Market and In-App
1 purchases a user makes within an app after buying it, Dr. Singer opined that Google’s take rate for
2 in-app content would have fallen from 29.2 percent in actual practice to 14.8 percent in a
3 competitive but-for world, resulting in an “average $1.34 consumer savings per transaction and an
4 aggregate damage figure of $4.71 billion.” Id. Dr. Singer offered an alternative damages model
5 based on Google’s Play Points rewards program, and concluded that in a competitive but-for
6 world, the Play Points program would have “expanded to be worth an average of $0.77 per
7 transaction, or approximately 8.7 percent of consumer spend,” resulting in aggregate damages of
8 $2.71 billion. Id. at 22; Singer Class Cert. Report ¶ 255.
9 For certification purposes, the Court determined that the Rule 23 questions of commonality
10 and predominance could be answered for the class as a whole on the basis of Dr. Singer’s
11 overcharge models for the Android App Distribution Market and In-App Aftermarket, and so
12 deferred for another day consideration of the Play Points model. Class Cert. Order at 23. The
13 Court overruled Google’s primary objection that Dr. Singer’s overcharge models were
14 inadmissible under FRE 702 because they were based on a faulty “pass-through” formula that
15 Dr. Singer used to quantify how much of Google’s developer fees consumers would ultimately
16 end up paying. As the Court noted, the pass-through formula was a “critical element of
17 Dr. Singer’s overcharge analysis,” and “was an input for both the Rochet-Tirole model (which
18 Dr. Singer used for the Android App Distribution Market) and the Landes-Posner model (used for
19 the In-App Aftermarket).” Id. at 9, 17. The pass-through formula was essential because app
20 developers independently set the prices of the apps and in-app content they make available
21 through the Play Store. The purpose of the pass-through formula was to quantify the “portion of
22 the supracompetitive cost imposed on developers” by Google that was “passed through” to, or
23 more aptly paid by, consumers. Id. at 17. This is a critical part of the consumers’ claim that they
24 overpaid for apps and in-app content as a result of Google’s anticompetitive conduct, and so a
25 classwide method of determining the pass-through rate was vital to the certification motion.
26 Dr. Singer used a pass-through formula “derived from a logit model,” which Dr. Singer
27 believed would correctly model “the demand curve faced by the developers who sell apps and
1 through formula may ultimately be expressed as ‘one minus the share’ an app has in its self-
2 selected Play Store category.” Id. at 17-18. To unpack this at a high level, “category” refers to
3 Google’s own denomination of broad topical groupings such as “education,” “game,” “sports,”
4 and the like used to organize apps in the Play Store. If an app has, say, a 20% share of the sports
5 category, then for that app, Dr. Singer would estimate a pass-through rate of 1 - 20% = 80%.
6 Dr. Singer’s ultimate calculation of the overcharges paid by consumers entails several additional
7 steps, but this logit-based pass-through formula is an essential core element of his overall
8 approach.
9 As the proponents of Dr. Singer’s expert testimony, plaintiffs had the burden of
10 establishing its admissibility over Google’s objections. See Southland Sod Farms v. Stover Seed
11 Co., 108 F.3d 1134, 1141-42 (9th Cir. 1997) (plaintiff, as “proponent of scientific evidence” had
12 “burden of establishing that the evidence is scientifically valid,” but nevertheless concluding that
13 “[b]ecause Defendants have not demonstrated that Plaintiffs are unable to make such a showing as
14 a matter of law, we will not exclude [plaintiffs’ expert’s] testimony under Daubert.”). An
15 important aspect of the admissibility analysis at the class certification stage was a careful
16 consideration of the comments made by Google’s proffered expert, Dr. Michelle Burtis, in her
17 certification report and at the hot tub with Dr. Singer. The goal of the hot tub was to provide
18 Google with the opportunity, through its expert, to illuminate its concerns about Dr. Singer’s
19 work, and to give Dr. Singer an opportunity for a real-time response. At the Court’s direction,
20 Dr. Burtis and Dr. Singer jointly prepared a list of discussion topics for the hot tub, in descending
21 order of importance for the question of certification. See Dkt. No. 284, Ex. 1.
22 Critically, for certification purposes, Dr. Burtis did not say that Dr. Singer’s opinions,
23 including his pass-through analysis, were “junk science” that ought to be excluded. See Ellis v.
24 Costco Wholesale Corp., 657 F.3d 970, 982 (9th Cir. 2011). To the contrary, and with specific
25 respect to Dr. Singer’s pass-through model, Dr. Burtis stated: “As I said, the model exists in the
26 literature; and I’m not here to say that this is a model that nobody uses. I won’t say that about this
27 model. Whether it’s the right model, I don’t know, and I don’t have an opinion.” Class Cert. Hot
1 model: “Regarding this model, I would say, I don’t think this model itself is junk science. I
2 wouldn’t say that. All I’m saying here is that, you know, Dr. Singer, he didn’t try to adapt the
3 model, to really test the issue of common impact here. He didn’t do anything to test.” Id. at 26:1-
4 5.
5 Dr. Burtis’s expert report was equally benign about Dr. Singer’s pass-through formula as a
6 method of analysis. See Dkt. No. 254-5 (Burtis Report). Dr. Burtis devoted three short
7 paragraphs in a 125-page report to the question of whether a logit demand model could, as a
8 matter of sound economics, generate reliable pass-through rates in the Play Store market. Id. at
9 ¶¶ 306-08. She did not say that a credible economist would never use a logit model in the Play
10 Store context. Dr. Burtis agreed that the logit model was “frequently used in economics.” Id. ¶
11 306. Her main substantive criticism was that Dr. Singer was wrong to use Google’s app
12 categories for the logit analysis, and that he should have come up with his own customized
13 groupings of apps into “more appropriate categories” that would “ensure that substitutes are
14 properly grouped together.” Id. ¶ 311; see also id. ¶ 279 (“The ‘categories’ used by Dr. Singer,
15 which are integral to the results, are not based on any economic analysis or reasoning but are
16 simply the categories used in Google Play.”). Dr. Burtis also faulted Dr. Singer for not accounting
17 for variables such as developers’ marginal costs and pricing strategies to set prices that end in
18 $0.99 cents. See id. ¶¶ 303-04, 313.
19 Overall, Dr. Burtis did not challenge the fundamental soundness of Dr. Singer’s approach
20 in light of the economic literature, and instead offered criticisms that went to the weight of his
21 opinions, and not to admissibility. Consequently, after conducting an independent analysis of
22 Dr. Singer’s work and weighing Google’s objections, the Court determined that Dr. Singer’s
23 testimony was admissible for certification purposes. See Class Cert. Order. Google did not
24 challenge the expert qualifications of Dr. Singer, a well-credentialed economist who is active in
25 the antitrust field. See id. at 8. On the record as it then stood, plaintiffs met their burden of
26 establishing admissibility, and Google and Dr. Burtis did not state objections that demonstrated
27 that Dr. Singer’s opinions warranted exclusion as junk science under Rule 702 or Daubert, 509
1 II. DR. SINGER’S MERITS OPINIONS
2 The situation has developed at the merits stage. The consumer plaintiffs proffer Dr. Singer
3 again to provide expert testimony on the substance of their antitrust claims, over Google’s
4 objections. Google does not challenge Dr. Singer’s qualifications as an expert, the relevance of
5 his testimony, or all of his opinions. Its motion to exclude is directed only at the injury and
6 damages portions of Dr. Singer’s work, and it challenges his opinions on these topics as unreliable
7 under FRE 702 and Daubert. See Dkt. No. 487.
8 In substantial measure, Dr. Singer’s injury and damages opinions are the same in his class
9 certification and merits reports. The pass-through formula is the same, and Dr. Singer again uses
10 the Rochet-Tirole model for the Android App Distribution Market and the Landes-Posner model
11 for the In-App Aftermarket. Singer Merits Report ¶¶ 288, 326, 358. But this time, Dr. Singer
12 offers aggregate damages figures calculated six different ways: (1) aggregate overcharge damages
13 of $23.83 million for the Android App Distribution Market; (2) aggregate overcharge damages of
14 $7.00 billion for the In-App Aftermarket; (3) a “discount model” based on Google Play Points,
15 calculated for a combined Android App Distribution Market and In-App Aftermarket “where the
16 locus of competition is on the consumer subsidy,” producing $3.92 billion in damages; (4) a
17 “single take rate” damages calculation, “where competition occurs only with respect to the take
18 rate in a single, combined market,” resulting in $3.66 billion in damages; (5) an “Amazon
19 Discount Model,” using the “Amazon Appstore’s consumer discounts” as a “reasonable
20 benchmark for calculating aggregate damages,” producing $8.039 billion in damages; and (6) a
21 single-market “hybrid model,” in which competition occurs with respect to both the take rate and
22 buyer-side subsidy, producing $3.81 billion in aggregate damages. Id. ¶¶ 414-21, 441-45.
23 With respect to the pass-through formula, Dr. Singer again states that, “when demand is
24 logit, a developer’s pass-through rate can be estimated as one minus that developer’s category
25 share.” Id. ¶ 358. The pass-through formula continues to be an essential input in his calculation
26 of aggregate overcharge damages for the Android App Distribution Market, see id. at 141, Table
27 6, and the In-App Aftermarket, see id. at 155, Table 8. The pass-through rate is also an input for
1 Table A5. It is not an input for the “discount” model, see id. at 191, Table 16, or the Amazon
2 Discount model, see id. at 206, Table 21.
3 Google’s response to Dr. Singer has changed since class certification. Most notably,
4 Dr. Burtis has yielded the floor to a new expert witness, Dr. Leonard. See Dkt. No. 487.
5 Dr. Leonard took a fresh look at Dr. Singer’s opinions and proffered, as will be discussed, a
6 different response from Dr. Burtis. As the Court stated at the merits hot tub, it has some
7 misgivings about Google taking a second shot at Dr. Singer’s testimony with a new witness. Even
8 so, the path to a fair result often has some turns, particularly as the record develops in a complex
9 antitrust dispute such as this one. Consideration of Google’s revised FRE 702 presentation based
10 on a new expert witness serves “the end of ascertaining the truth and securing a just
11 determination” in this multidistrict litigation. Fed. R. Evid. 102.
12 DISCUSSION
13 I. LEGAL STANDARDS
14 As Federal Rule of Evidence 702 states, a “witness who is qualified as an expert by
15 knowledge, skill, experience, training, or education may testify in the form of an opinion or
16 otherwise if: (a) the expert’s scientific, technical, or other specialized knowledge will help the
17 trier of fact to understand the evidence or to determine a fact in issue; (b) the testimony is based on
18 sufficient facts or data; (c) the testimony is the product of reliable principles and methods; and
19 (d) the expert has reliably applied the principles and methods to the facts of the case.”
20 This rule is expected to be updated soon. By order of the United States Supreme Court
21 dated April 24, 2023, a proposed amendment to FRE 702 will take effect on December 1, 2023,
22 barring any contrary Congressional action. See https://www.supremecourt.gov/orders/
23 ordersofthecourt/22 (“4/24/23 Rules of Evidence”); 28 U.S.C. § 2074. The proposed amendment
24 clarifies that an expert witness’s opinion testimony is admissible under FRE 702 only “if the
25 proponent demonstrates to the court that it is more likely than not that” the proposed testimony
26 satisfies subsections (a) through (d) of the Rule. Subsection (d) will also be replaced in its entirety
27 to provide that the expert’s opinion must “reflect[] a reliable application of the principles and
1 proposed amendment is not a sea change but rather an amplification of existing FRE 702
2 standards. For present purposes, the Court is mindful of FRE 702 as it stands today and as it will
3 be imminently amended.
4 As the Court has observed in another case, the FRE 702 admissibility standard does not
5 change with the different stages of litigation or become more rigorous as a case progresses from
6 class certification to the merits stage. See In re Capacitors Antitrust Litigation, MDL Case
7 No. 17-md-02801-JD, 2020 WL 870927, at *2 (N.D. Cal. Feb. 21, 2020). At all stages, “Rule 702
8 of the Federal Rules of Evidence tasks a district court judge with ‘ensuring that an expert’s
9 testimony both rests on a reliable foundation and is relevant to the task at hand.’” Elosu v.
10 Middlefork Ranch Inc., 26 F.4th 1017, 1023 (9th Cir. 2022) (quoting Daubert, 509 U.S. at 597).
11 Reliability is the touchstone. “The test of reliability is flexible,” and “the trial court has
12 discretion to decide how to test an expert’s reliability as well as whether the testimony is reliable,
13 based on the particular circumstances of the particular case.” Primiano v. Cook, 598 F.3d 558,
14 564 (9th Cir. 2010) (cleaned up). As the amendment of FRE 702 emphasizes, the burden of
15 establishing the reliability of the proposed expert witness testimony rests with the proponent of the
16 expert evidence. See Southland Sod, 108 F.3d at 1141. The Court “must decide any preliminary
17 question about whether a witness is qualified, . . . , or evidence is admissible,” and “[i]n so
18 deciding, the court is not bound by evidence rules, except those on privilege.” Fed. R. Evid.
19 104(a). When “admissibility determinations . . . hinge on preliminary factual questions,” those
20 factual matters must be “established by a preponderance of proof”; application of the
21 “preponderance standard ensures that before admitting evidence, the court will have found it more
22 likely than not that the technical issues and policy concerns addressed by the Federal Rules of
23 Evidence have been afforded due consideration.” Bourjaily v. United States, 483 U.S. 171, 175
24 (1987).
25 II. THE PASS-THROUGH FORMULA
26 In his merits opinions, Dr. Singer used a pass-through formula “specific to logit” that was
27 developed by the economists Nathan Miller, Marc Remer, and Gloria Sheu, and he applied that
1 ¶¶ 358, 360. The Google Play Store has approximately 33 app categories for “Beauty,” “Dating,”
2 “Events,” “Health and Fitness,” “Productivity,” “Weather,” and similar categories, and app
3 developers self-select a category when positioning their apps in the Play Store. Id. ¶¶ 349-50 &
4 Table 13. In Dr. Singer’s view, “Miller et. al. demonstrate mathematically that, when firms are
5 subjected to an industrywide change in costs, the profit-maximizing change in the price of a
6 particular product i in response to a one dollar change in a firm’s marginal cost is equal to [M –
7 Qi]/M, where M is the size of the category -- inclusive of the outside good -- and Qi is the quantity
8 sold of product i. This means that, when demand is logit, a developer’s pass-through rate can be
9 estimated as one minus that developer’s category share, consistent with what has been shown
10 previously in the peer-reviewed economics literature.” Id. ¶ 358.
11 The reliability of this logit-based pass-through rate depends on whether Dr. Singer reliably
12 “estimate[d] logit demand systems for each of the categories used by Google.” Id. ¶ 354. “In a
13 logit demand system, each product within the system has its own (nonlinear) demand curve, given
14 by the following formula: ln(Sj / S0) = δj + αPj.” Id. ¶ 348. Dr. Singer explains, “Sj is the share of
15 product j, and S0 is the share of the outside good -- that is, the proportion of consumers that do not
16 purchase any of the products at issue. The term δj represents factors other than price that shift
17 demand (and thus share). These are modeled as fixed effects unique to a given App and purchase
18 type (Initial Downloads, In-App, and Subscription). The model also includes fixed effects by
19 state, and for sub-products within a given App (e.g., Pandora Plus versus Pandora Premium).” Id.
20 Dr. Singer states that “[e]conomists have frequently used logit to analyze a variety of economic
21 phenomena, including (but not limited to) potentially anticompetitive conduct in markets with
22 differentiated products.” Id. He also states that “[t]he standard logit model is widely used by
23 economists to estimate pass-through in a range of contexts,” and he acknowledges that the logit
24 demand system implies “that developers in a given category pass through cost savings according
25 to their dominance (or lack thereof) in the category, as measured by their market share within that
26 category.” Id. ¶¶ 351, 356.
27 In response to these and related propositions by Dr. Singer, Dr. Leonard presented several
1 “IIA” property. As Dr. Leonard stated in his report, the logit model “exhibits what is called the
2 ‘independence of irrelevant alternatives’ (IIA) property. The IIA property places strong
3 restrictions on substitution patterns between products (i.e., the own- and cross-price elasticities of
4 demand). Because of IIA’s restrictiveness regarding substitution patterns, from the early 1980s,
5 the economics literature has warned about the use of the logit model of demand.” Dkt. No. 489-3
6 (Leonard Report) at 60 n.76; see also id. ¶ 153. This was new information in that Dr. Burtis had
7 not specifically identified or highlighted the IIA property in a meaningful way. She did not use
8 that term in her report. See Dkt. No. 254-5. She and Dr. Singer did not identify the IIA restriction
9 as a topic for debate at the certification hot tub. See Dkt. No. 284, Ex. 1. During the hot tub
10 discussion, Dr. Burtis never expressly mentioned IIA and made only a passing mention of
11 substitution late in the proceeding. See Dkt. No. 302 at 88:22-91:7.
12 In significant contrast, Dr. Leonard put the IIA property of logit front and center in his
13 challenge to Dr. Singer’s analysis. Dr. Singer does not seriously dispute Dr. Leonard’s
14 observations about the IIA property itself. In the experts’ joint statement of topics for the merits
15 hot tub, Dr. Singer said that he “will address Google’s claim that he misapplied logit because the
16 property of ‘IIA’ or ‘proportional substitution’ -- when prices for one product increase, consumers
17 switch to substitutes in proportion to their relative shares -- is allegedly not satisfied.” Dkt.
18 No. 540-2 at 12. Dr. Singer added that he “will explain that it is reasonable to conclude that the
19 proportional substitution property is satisfied here, as evidenced by his regressions . . . .
20 Moreover, logit is routinely and reliably used as an approximation even when IIA is not strictly
21 satisfied . . . .” Id. Dr. Leonard, on his part, stated that “[o]ne feature of the logit model
22 Dr. Singer used is the ‘irrelevance of independent alternatives’ property, or IIA, which holds that
23 all goods in the market where demand is being studied are substitutes for one another in
24 proportion to their share of that market. There is an economic consensus that if real world
25 demands do not satisfy this property, then the model will yield unreliable results. . . . As applied
26 to demand for Android apps, the IIA principle means that all apps in a given app category must be
27 substitutes for each other, and must be substitutes in proportion to their share of that category.
1 However, Dr. Singer concedes that apps in each category fail this condition. This makes his entire
2 model unreliable.” Id. at 12-13.
3 The IIA issue was raised in the parties’ Rule 702 motion briefing, see Dkt. No. 487 at 6-10,
4 Dkt. No. 508 at 5-9, and was discussed in detail at the merits hot tub. In his opening comments
5 about Dr. Singer’s work, Dr. Leonard underscored that “the big problem with the logit model is
6 the so-called IIA assumption. . . . [S]ince probably 1977 or so there have been well-known tests
7 that test for the IIA assumption. And it’s also very well known you shouldn’t just assume logit
8 because it has these very restrictive assumptions on substitution patterns . . . basically a
9 proportional substitution.” Merits Hot Tub Tr. at 27:18-25. Dr. Singer did not take serious issue
10 with Dr. Leonard. When the Court asked, “what is the source of the proportionate substitution or
11 demand proposition, is that Miller?” Dr. Singer said, “Oh, I think it will be in Miller, but it will be
12 on any -- in any -- I don’t think that’s disputed. It’s proportional substitution. That’s what the --
13 that’s what the IIA property is about.” Id. at 52:8-14.
14 This discussion at the hot tub, and in the merits reports generally, put a much finer point
15 than at class certification on the question of whether Dr. Singer’s logit-based pass-through formula
16 was sufficiently valid and reliable to be admissible. The Court inquired further into the question
17 when it called for additional comments by the economists after the hot tub proceeding. Dkt.
18 No. 570. Among other inquiries, the Court asked: “(A) What economic literature states that a
19 regression analysis is a reliable way of (i) testing for the IIA assumption in the logit model, or
20 (ii) confirming that a logit model can be used to reliably measure the relevant demand curve
21 here?” And, “(B) To what extent can IIA be ‘not strictly satisfied’ before the use of logit model
22 becomes unreliable? How can the Court know that this limit has not been crossed here? How
23 close is the ‘approximation’ that Dr. Singer posits, and how can the Court have confidence that his
24 logit model has produced a sufficiently reliable approximation of pass-through here even if the
25 apps in each category are not proportional substitutes for one another?” Id. at 2.
26 Dr. Singer and Dr. Leonard filed sworn answers to the follow-up questions. Dkt. Nos. 578,
27 580. Dr. Leonard stated that the “defining characteristic of the logit model is the IIA assumption,
1 substitute among products in the marketplace being studied.” Dkt. No. 578 ¶ 6. Dr. Leonard also
2 stated that, in the “specific case of Android apps, given the category definitions that Dr. Singer
3 used, the IIA assumptions of the logit model that all apps are substitutes and substitution is
4 proportional to shares are clearly false,” because “[s]ome of the apps within a category are not
5 substitutes for each other at all, let alone in a manner proportional to their respective shares.” Id.
6 ¶ 19. To illustrate, Dr. Leonard gave the example of “Rosetta Stone,” “Duolingo,” and
7 “PictureThis - Plant Identifier,” which are “three apps in the Education category.” Id. Rosetta
8 Stone has less than a 5% category share; Duolingo has around 15%; and PictureThis - Plant
9 Identifier has around 20%. Id. Dr. Leonard observed that, “[w]ith entirely different functionality
10 than the language learning apps, there can be no serious argument that PictureThis - Plant
11 Identifier is any kind of substitute at all for Rosetta Stone,” and yet, “the logit model, with its IIA
12 assumption, assumes that if Rosetta Stone raised its price and some customers substituted away,
13 PictureThis - Plant Identifier would capture a larger percentage of these switching customers than
14 Duolingo . . . simply because PictureThis - Plant Identifier has a larger category share than
15 Duolingo.” Id. In Dr. Leonard’s view, “[t]his makes no economic sense at all.” Id.
16 Dr. Singer stated in his response to the follow-up questions that “IIA is a property of
17 logit,” and “[a]pplied here, IIA implies that consumers will tend to substitute among different
18 Apps within a given category in proportion to an Apps’ share in that category (‘proportional
19 substitution’ or ‘proportionate shifting’).” Dkt. No. 580 ¶ 13. Dr. Singer’s comments were
20 consistent with Dr. Leonard in terms of how the IIA assumption would be expected to play out in
21 the context of apps in the Play Store: “Suppose the price of App A increases. To avoid the price
22 hike, some consumers will switch to different Apps within the same category. Suppose further
23 that App B is very popular, with a category share of 50 percent, and that App C is less popular,
24 with a category share of just one percent. Under proportional substitution, these consumers are
25 more likely to switch to the (more popular) App B than they are to switch to the (less popular) App
26 C. Specifically, consumers are, on average, fifty times more likely to switch to App B than App C
27 under this assumption.” Id.
1 Critically, Dr. Singer did not explain why this assumption would still make economic
2 sense if App A and App C were more similar, like Duolingo and Rosetta Stone, and App B were
3 entirely different, such as PictureThis - Plant Identifier. As Dr. Leonard suggests, it is intuitively
4 obvious that users looking for an app to learn Italian will not try to avoid a price hike by switching
5 to an app that identifies the type of geranium in their kitchen. This intuition highlights a
6 fundamental problem that a jury would face if Dr. Singer’s opinions were presented at trial. It
7 may be possible for a jury to make reasonable decisions about the substitutability of certain apps
8 at a very high and general level, but Dr. Singer’s analysis does not provide usable guidance on
9 what to do with the myriad of differences and distinctions between apps within the Google Play
10 Store categories. He does not provide any boundaries on substitution in broad app categories that
11 contain many unlike products. This would create a serious risk of the jury simply guessing about
12 proportionate substitution and ultimately the pass-through of fees to consumers.
13 Dr. Singer’s position with respect to the IIA property of logit is further eroded by one of
14 the main authorities he cited in his follow-up response and attached in full as an exhibit: Kenneth
15 Train, Logit, in Discrete Choice Methods with Simulation 34 (Cambridge University Press 2009).
16 See Dkt. No. 580, Ex. 15. Professor Train’s chapter on logit deepens rather than alleviates the
17 Court’s concern that the logit model cannot be reliably used in the context of apps in the Google
18 Play Store in the way Dr. Singer has done in his analysis. Professor Train starts with the
19 observation that “[b]y far the easiest and most widely used discrete choice model is logit.” Id. at
20 34. He explains that “[i]ts popularity is due to the fact that the formula for the choice probabilities
21 takes a closed form and is readily interpretable.” Id.
22 From there, he sounds many cautionary notes about the appropriateness of its use. He
23 states, for example, that “[l]ogit models can capture taste variations, but only within limits. In
24 particular, tastes that vary systematically with respect to observed variables can be incorporated in
25 logit models, while tastes that vary with unobserved variables or purely randomly cannot be
26 handled.” Id. at 43. Also, “if taste variation is at least partly random, logit is a misspecification.
27 As an approximation, logit might be able capture the average tastes fairly well even when tastes
1 might therefore choose to use logit even when she knows that tastes have a random component,
2 for the sake of simplicity. However, there is no guarantee that a logit model will approximate the
3 average tastes. And even if it does, logit does not provide information on the distribution of tastes
4 around the average. This distribution can be important in many situations . . . .” Id. at 44.
5 Further, “[p]roportionate substitution can be realistic for some situations, in which case the logit
6 model is appropriate. In many settings, however, other patterns of substitution can be expected,
7 and imposing proportionate substitution through the logit model can lead to unrealistic forecasts.”
8 Id. at 48.
9 These comments support Dr. Leonard’s critiques and undercut the reliability of
10 Dr. Singer’s work. Dr. Singer endeavors to use the logit model in an overly simple way to
11 represent the demand curve for developers in the Play Store. In Dr. Singer’s model, when the
12 price of an app goes up, the consumer will necessarily switch to a different app in the same
13 category, based purely on the popularity of those other apps. As Dr. Singer acknowledges, this
14 approach works only if the apps within each category are proportional substitutes for one another.
15 This is an unproven assumption in Dr. Singer’s work. It cannot be squared with the economic
16 literature such as that of Professor Train, and it flies in the face of the huge diversity of apps
17 within the Play Store categories. As Dr. Leonard has noted, given the broad categories in the
18 Google Play Store, which developers self-select, the IIA’s assumption that “all apps are substitutes
19 and substitution is proportional to shares” is not factually supported in this context. Dkt. No. 578
20 ¶ 19.
21 Dr. Singer’s main defense is to say that “IIA is reliably established here” because he has
22 “confirmed using standard regression methods from the economic literature that the logit demand
23 curve is well-specified here.” Dkt. No. 580 at 8. The problem is that nothing validates the use of
24 regressions in this manner. Professor Train certainly did not identify this kind of regression
25 analysis as a way of validating a use of logit. He did say that the “independence assumption . . . in
26 fact can be interpreted as a natural outcome of a well-specified model,” and that “[i]n a deep
27 sense, the ultimate goal of the researcher is to represent utility so well that the only remaining
1 model is appropriate. Seen in this way, the logit model is the ideal rather than a restriction.” Id.,
2 Ex. 15 at 35-36. But this observation does not appear to fit Dr. Singer’s model. He has not
3 specified his observed variables so well that “the remaining, unobserved portion of utility is
4 essentially ‘white noise.’” Id. at 35. Rather, as Dr. Leonard notes, Dr. Singer’s model “includes
5 only the app price and a set of SKU-time-state indicator variables. This leaves plenty of room for
6 substantial correlation among the remaining unobserved portions of a consumer’s utilities for
7 apps. For example, consumers who like a given single-shooter game likely also like other single-
8 shooter games . . . . That is, such consumers will exhibit positive correlation among unobserved
9 parts of their utilities for single-shooter games. The unobserved portions of their utilities are not
10 just ‘white noise.’ The price and indicator variables included in Dr. Singer’s model would not
11 capture this correlation in consumers’ preferences over single-shooter games and therefore the
12 ‘ideal’ would not be met and the logit model would not apply.” Dkt. No. 578 ¶ 29.
13 Dr. Leonard has also pointed out that Dr. Singer did not compare the “fit” of the logit
14 model with “that of an alternative demand model.” Id. ¶ 14. And in Dr. Leonard’s view,
15 Dr. Singer’s claim that he “obtained the ‘right’ signs and statistical significance on the price
16 coefficients in his regression model as support for the logit model” is “a low bar,” because “all
17 demand models predict lower share (i.e., lower quantity) when price increases and vice versa.” Id.
18 at 8 n.9. Similarly, the States’ expert, Dr. Rysman, was asked in his deposition whether it would
19 be sufficient for him “to determine that a standard logit model was appropriate that there was a
20 negative correlation between price and demand,” and he responded, “Not by itself[,] that wouldn’t
21 tell me that the logit model was appropriate.” Dkt. No. 487-4 at 68:21-69:2. While plaintiffs have
22 pointed out that Dr. Rysman “had not read Dr. Singer’s report,” Dkt. No. 508 at 7 n.5, it is hard to
23 see why that would matter for purposes of the answer Dr. Rysman gave, which stands on its own
24 and bolsters Dr. Leonard’s critique of Dr. Singer’s work.
25 Overall, the record at the merits stage is substantially more developed than at class
26 certification, and establishes that Dr. Singer’s pass-through model is not within accepted economic
27 theory and literature, and is based on assumptions about the Play Store apps that are not supported
1 reasonable judgment about antitrust impact and damages in a product market that does not show
2 proportional substitution across alternatives, at least not on a Play Store category share basis as
3 Dr. Singer has modeled.
4 Because that pass-through model is the keystone of Dr. Singer’s overcharge analysis, his
5 opinions based on it must be excluded. The purpose of judicial gatekeeping under Rule 702 is “to
6 make certain that an expert . . . employs in the courtroom the same level of intellectual rigor that
7 characterizes the practice of an expert in the relevant field.” Kumho Tire Co., Ltd. v. Carmichael,
8 526 U.S. 137, 152 (1999). Dr. Singer’s use of a logit approach to model the demand curve faced
9 by app developers in the Play Store, ultimately producing the simple pass-through formula of one
10 minus the app’s share of its category, was a decision that “fell outside the range where experts
11 might reasonably differ, and where the jury must decide among the conflicting views of different
12 experts, even though the evidence is ‘shaky.’” Id. at 153 (quoting Daubert, 509 U.S. at 596).
13 Because the characteristics of a logit model and its IIA property are enough to find that
14 Dr. Singer’s pass-through formula here is not sufficiently reliable to be admitted under Rule 702,
15 the Court declines to reach Google’s other arguments that the pass-through formula suffers from
16 additional admissibility shortcomings.5 Since Dr. Singer’s pass-through formula is not reliable
17 enough to be admitted, his testimony about that formula, and his injury and damages opinions that
18 necessarily rely on it, are excluded.
19 III. THE CONSUMER SUBSIDY MODELS
20 As an alternative approach, Dr. Singer offered “consumer subsidy” models that did not use
21 the pass-through formula. Opinions with respect to these models are also excluded.
22 The main reason for exclusion is that the analysis behind the subsidy models is too anemic
23 to let them go to a jury. For the Play Points model, Dr. Singer relies on wholly speculative
24 assumptions that make his opinions ipse dixit unsuitable for admission at trial. For example, he
25 states, with no visible factual support, that “the structure of Play Points is a reasonable facsimile of
26
5 Google’s motion for leave to file a supplemental brief in support of its Rule 702 motion, Dkt.
27
No. 541, is granted. For the sake of deciding this issue on as complete a record as possible, the
1 what an expanded program might look like in a competitive but-for world,” Singer Merits Report
2 ¶ 373, and that “[c]onsumers would have enhanced economic incentives to enroll and participate
3 in a Play Points offering more valuable incentives in the but-for world, just as consumers have
4 more incentives to participate in a more generous credit card rewards program than a less generous
5 one.” Id. ¶ 381. Why any of this might be true is not said. Dr. Singer’s Play Points calculations
6 also rest on the assumption that, in the but-for world, Google “maintains a 60 percent market share
7 with an inelastic supply response from Google’s rivals.” Id. ¶ 386. Dr. Singer says that “[e]ven in
8 the presence of substantial competition, I assume conservatively that Google would have retained
9 a substantial market share of 60 percent,” because “this was approximately AT&T’s market share
10 in the long-distance market after competitive entry.” Id. ¶ 331. It is again not explained, and is
11 certainly not obvious, why the situation AT&T faced in the telecom market in the 1980s is a good
12 benchmark for Google’s app store practices today. As Dr. Leonard aptly commented, “[t]he
13 economics of long distance service in the 1980s and early 1990s differed substantially from the
14 but-for world for Android app stores in this case,” and “without an in-depth analysis,” there is an
15 insufficient basis “to think that the entry costs, requirements, and market opportunity for one or
16 more new firms to compete with the incumbent would be the same in the Android app store
17 marketplace as was the case in the 1980s and early 1990s long distance service marketplace.”
18 Dkt. No. 578 ¶¶ 43-44.
19 So too for Dr. Singer’s other consumer subsidy model. Dr. Singer devotes a paltry four
20 paragraphs to a purported Amazon Coins discount damages model. Singer Merits Report ¶¶ 417-
21 20. Not surprisingly, those four paragraphs do not adequately explain why or how the Amazon
22 Appstore might be a “reasonable approximation” of damages here. Id. ¶ 418. Dr. Singer again
23 simply asserts, with no real analysis or data, that “Amazon’s aggregate discount . . . on third-party
24 devices is a reasonable benchmark for estimating aggregate damages.” Id. ¶ 419.
25 “[N]othing in either Daubert or the Federal Rules of Evidence requires a district court to
26 admit opinion evidence that is connected to existing data only by the ipse dixit of the expert. A
27 court may conclude that there is simply too great an analytical gap between the data and the
1 opinion proffered.” General Electric Co. v. Joiner, 522 U.S. 136, 146 (1997). That is the case
2 || here for Dr. Singer’s consumer subsidy models.
3 CONCLUSION
4 Google’s motion to exclude the merits opinion testimony of Dr. Singer, Dkt. No. 487, is
5 || granted.
6 IT IS SO ORDERED.
7 Dated: August 28, 2023
8
9
JAME NATO
10 Unitedfftates District Judge
11
a 12
15
16
it
Z 18
19
20
21
22
23
24
25
26
27
28