# Establishing the Digital Opportunity Data Collection

> Briefs, arguments, decisions, and more.

URL: https://www.frixlaw.com/law-library/documents/fr%3A2022-06826

## Record

- **Collection:** Federal Register
- **Document type:** Rule
- **Published:** April 11, 2022
- **Citation:** 87 FR 21476

## Text

FEDERAL COMMUNICATIONS COMMISSION
47 CFR Part 1
[WC Docket No. 19-195, DA-22-241; FR ID 78895]
Establishing the Digital Opportunity Data Collection

AGENCY:

Federal Communications Commission.

ACTION:

Final rule.

SUMMARY:

In this document, the Wireless Telecommunications Bureau (WTB), the Office of Economics and Analytics (OEA), and the Office of Engineering and Technology (OET) (collectively, the Bureau and Offices) adopt technical requirements to implement the mobile challenge, verification, and crowdsourcing processes required by the Broadband DATA Act. The Bureau and Offices adopt the proposed processes and methodology set forth in the
Broadband Data Collection (BDC) Mobile Technical Requirements Proposed Rules
for collecting challenge process data and for determining when the threshold to create a cognizable challenge has been met. Additionally, the Bureau and Offices adopt detailed processes for mobile providers to respond to challenges, for the Federal Communications Commission (Commission or FCC) to initiate a verification request to a service provider, and for providers to respond to verification requests to confirm broadband coverage in areas they claim have service. The Bureau and Offices adopt the parameters and metrics that must be collected both for on-the-ground test data to support challenge submissions, rebuttals to cognizable challenges, and responses to verification requests, and for infrastructure information to support challenge rebuttals and responses to verification requests. Government entities and third parties are required to submit verified broadband data using the same data specifications required of mobile service providers. Finally, the Bureau and Offices find the Commission's speed test app to be a reliable and efficient method for entities to use in submitting crowdsourced mobile coverage data to the Commission and describe the methodology staff will use in determining when a “critical mass of” crowdsourced filings suggests that a provider has submitted inaccurate or incomplete data. The measures adopted in this document to implement the mobile challenge, verification, and crowdsourcing processes will enable the Commission, Congress, other Federal and state policy makers, Tribal entities, consumers, and other third parties to verify and supplement the data collected by the Commission on the status of mobile broadband availability throughout the United States.

DATES:

Effective May 11, 2022.

FOR FURTHER INFORMATION CONTACT:

William Holloway at
William.Holloway@fcc.gov,
Competition & Infrastructure Policy Division, (WTB), (202) 418-2334, Jonathan McCormack at
Jonathan.McCormack@fcc.gov
(OEA), (202) 418-1065, or Martin Doczkat at
Martin.Doczkat@fcc.gov
(OET), (202) 418-2435.

SUPPLEMENTARY INFORMATION:

This is a summary of the Commission's
Order,
DA 22-241, in WC Docket No. 19-195, adopted and released on March 9, 2022. The full text of this document, including the technical appendix is available for public inspection and can be downloaded at
https://www.fcc.gov/document/fcc-releases-bdc-mobile-technical-requirements-order.

People With Disabilities.
To request materials in accessible formats for people with disabilities (braille, large print, electronic files, audio format), send an email to
fcc504@fcc.gov
or call the Consumer & Government Affairs Bureau at 202-418-0530 (voice), 202-418-0432 (tty).

Paperwork Reduction Act.
This document does not contain new or modified information collection(s) subject to the Paperwork Reduction Act of 1995 (PRA), Public Law 104-13, as the requirements adopted in this document are statutorily exempted from the requirements of the PRA. As a result, the Order will not be submitted to the Office of Management and Budget (OMB) for review under Section 3507(d) of the PRA.

Congressional Review Act.
The Commission has determined, and the Administrator of the Office of Information and Regulatory Affairs, Office of Management and Budget, concurs, that these rules are “non-major” under the Congressional Review Act, 5 U.S.C. 804(2). The Commission will send a copy of the Order to Congress and the Government Accountability Office pursuant to 5 U.S.C. 801(a)(1)(A).

Synopsis

1. In this document, the Bureau and Offices adopt the technical requirements to implement the mobile challenge, verification, and crowdsourcing processes required by the Broadband DATA Act as part of the FCC's ongoing BDC effort to improve the Commission's broadband availability data.

I. Discussion

A. Mobile Service Challenge Process

2. In this document, the Bureau and Offices adopt the proposals for the mobile challenge process set forth in the
BDC Mobile Technical Requirements Proposed Rules
(86 FR 40398, July 28, 2021), with certain modifications described below.

3. The Broadband DATA Act requires that the Commission “establish a user-friendly challenge process through which consumers, [s]tate, local, and Tribal governmental entities, and other entities or individuals may submit coverage data to the Commission to challenge the accuracy of—(i) the coverage maps; (ii) any information submitted by a provider regarding the availability of broadband internet access service; or (iii) the information included in the [Broadband Serviceable Location] Fabric.” The general requirements and framework for the mobile challenge process predate the
BDC Mobile Technical Requirements Proposed Rules,
and were set forth in either the Broadband DATA Act or prior Commission orders. We note that, to the extent commenters ask the Bureau and Offices to eliminate, modify, or otherwise revisit particular requirements established in either the Broadband DATA Act or prior Commission-level orders, we lack the legal authority to do so. In the
Third Further Notice of Proposed Rulemaking (Third Further NPRM)
(85 FR 50911, Aug. 18, 2020), the Commission proposed a challenge process that “encourages participation to maximize the accuracy of the maps, while also accounting for the variable nature of wireless service.” In the
Third Order
(85 FR 18124, April 7, 2021), the Commission adopted its proposals from the
Second Order
(85 FR 50886, Aug. 18, 2020) and
Third Further NPRM,
and established a framework for consumers, state, local, and Tribal governments, and other entities to submit data to challenge the mobile broadband coverage maps.

4. The Commission determined that it should enable stakeholders to challenge mobile coverage data based on both a lack of service and poor service quality (such as slow delivered user speeds). Challenges must be based upon on-the-ground speed test data taken outdoors (
i.e.,
from an in-vehicle mobile or outdoor stationary environment). The Commission adopted a requirement that consumers use a speed test application (either developed by the FCC or a third-

party app approved by OET for use in the challenge process) that automatically collects information and metrics associated with each speed test and allows for submission of information directly to the Commission from a mobile device. Consumers will be required to submit certain identifying information to deter frivolous filings. Government and other third-party entity challengers (including competing mobile service providers) may use their own software or hardware to collect data for the challenge process so long as the data contain metrics that are substantially the same as those collected by approved speed test applications. Moreover, government and other entity challengers are required to conduct on-the-ground tests using a device advertised by the challenged provider as compatible with its network.

5. The Commission adopted a requirement for providers to either submit a rebuttal to the challenge or concede the challenge within 60 days of being notified of the challenge. Rebuttals must consist of either on-the-ground test data or infrastructure data. A challenge respondent may also submit supplemental data in support of its rebuttal, either voluntarily or in response to a request for additional information from OEA. The Commission directed OEA to develop a methodology and mechanism to determine if the data submitted by a provider constitute a successful rebuttal to all or some of the challenged service area and to establish procedures to notify challengers and providers of the results of a challenge. Further, the Commission adopted a requirement that providers that concede or lose a challenge file new coverage data within 30 days depicting the challenged area that has been shown to lack service.

6. The requirements that we adopt in this document will enable the Commission to collect sufficient measurements to ensure that the challenge process is statistically valid while remaining “user-friendly.” In particular, we establish a methodology for determining a threshold number of mobile speed tests and the geographic boundaries within a specified area. Based on this methodology, a challenge is created by associating the locations of validated speed tests within geographical hexagons defined by the accessible, open-source H3 geospatial indexing system and analyzing those speed tests. We also adopt the parameters and metrics that speed tests must meet to be validated and used to meet the challenge thresholds. Importantly, as the Commission specified in the
Third Order,
the challenge process will remain user-friendly because all of the information a consumer needs to create a challenge will be collected and submitted by the FCC Speed Test app and any third-party mobile speed test apps approved by OET. Governmental and other entity challengers may use these apps or their own software or hardware to collect data for the challenge process. Additionally, we implement the Commission's decision to aggregate speed tests to resolve challenges “in an efficient manner, mitigate the time and expense involved, and ensure that the mobile coverage maps are as reliable and useful as possible,” by adopting our proposal to combine speed tests conducted by consumers, governmental agencies, and other entities to determine whether the thresholds for a cognizable challenge have been met. These requirements strike the appropriate balance between ensuring that consumers, state, local, and Tribal governments, and other entities can participate in the challenge process, on the one hand, and protecting providers from being burdened by having to respond to challenges that do not meet the cognizable challenge standard, on the other hand.

1. Creating a Challenge/Cognizable Challenges

7.
On-the-Ground Speed Test Data Parameters and Metrics.
Challenges must be supported by on-the-ground test data. We have therefore established the required testing parameters and data metrics for speed test submissions. At the outset, we will require the FCC Speed Test app and approved third-party apps to collect the name and email address of the end user and mobile phone number of the device on which the speed test was conducted, to the extent technically feasible. As discussed in further detail below, Apple iOS devices will not automatically transmit the mobile phone number associated with the device that runs a speed test. We will therefore require testers submitting tests for use in the challenge process to manually submit, through the speed test app, the phone number associated with the device on which the speed test was conducted. The Commission's rules state that consumer challengers must include “name and contact information (
e.g.,
address, phone number, and/or email address) in their data submissions.” We amend these rules to require that app users also submit their email address so that the Commission can notify testers of the status of their speed test(s) and any resulting challenge(s), and we also amend the rules to require app users to submit the mobile phone number of the device on which the speed test was conducted so that we may, if necessary, share this information with mobile broadband providers for use when responding to challenges. We anticipate we will only share the phone number of the device on which the speed test was conducted with mobile broadband providers in situations where a challenged provider is unable to identify a subscriber by using the timestamp that test measurement data were transmitted to the app developer's servers, as well as the source IP address and port of the device, as measured by the server, which we will also require to be included in challenge data submitted by the app, as discussed below. We will not collect the address of an end user for use in the mobile challenge process at this time in order to minimize the amount of personally identifiable information we require from end users, and because a mobile user's physical address is not currently helpful either to the Bureau and Offices when considering challenges or to providers when responding to challenges. In addition to the testing metrics adopted by the Commission in the
Third Order,
we adopt the testing parameters and updated metrics for challenge speed test data proposed in the
BDC Mobile Technical Requirements Proposed Rules,
with the modifications described below. We will require the FCC Speed Test app and approved third-party apps to collect the consumer's name, email address, and phone number of the device on which the speed test was conducted to the extent technically feasible. With the exception of different considerations pertaining to the submission of speed test data taken on iOS devices and the submission of IP address, source port, and timestamp measured by an app developer's servers by government entities and service providers in some scenarios, these parameters and metrics will apply across all testing mechanisms, not only in the challenge process but also for on-the-ground data submitted in response to verification inquiries. The information we will use in the challenge process that can be collected from Android devices, but not iOS devices, includes the signal strength, signal quality, unique identifier, and other radiofrequency (RF) metrics of each serving cell, as well as the spectrum bands used for the test and other network characteristics (
e.g.,
whether the device was roaming, as well as the identity of the provider for the connected network). As discussed in greater detail below, we will allow

government and other third-party entities to alternatively submit the International Mobile Equipment Identity (IMEI) of the device used to conduct a speed test for use in the challenge process rather than provide the source IP address, source port, and timestamp measured by an app developer's servers. We will also not require a service provider to submit either the device IMEI or the combination of source IP address, source port, and timestamp measured by an app developer's servers when submitting speed tests either in response to a challenge or in response to a verification inquiry. Individual consumer challengers must collect on-the-ground speed test data using mobile devices running either a Commission-developed app (
e.g.,
the FCC Speed Test app) or another speed test app approved by OET for the submission of challenges. The Bureau and Offices will announce the process and procedures for third-party app providers to seek approval for a speed test app to be used in submitting data for use in the challenge process. Third-party and governmental entities may, as specified in the
Third Order,
collect data using either one of these speed test apps or their own software and hardware that collects broadband availability data, consistent with the parameters and metrics set forth herein. We include “hardware” to capture the professional tools such as laptops, hard drives, or other hardware devices, used to collect on-the-ground data. The
Third Order
provided that government and other entity challengers submit a complete description of the methodologies used to collect the data. The Bureau and Offices will issue a public notice announcing the process and procedures for such parties to submit the necessary documentation.

8. In the
Third Order,
the Commission required consumer challengers to use a speed test app approved by OET for use in the challenge process and provided the metrics that approved apps must collect for each speed test. The Commission directed OET, in consultation with OEA and WTB, to update the FCC Speed Test app as necessary or develop a new speed test app to collect the designated metrics, so that challengers may use it in the challenge process. For government and third-party entity challengers, the Commission did not require the use of a Commission-approved speed test app but instead set forth the information that all submitted government and third-party challenger speed test data must contain and directed OEA, WTB, and OET to adopt additional testing requirements if they determine it is necessary to do so. Our
BDC Mobile Technical Requirements Proposed Rules
proposed certain testing parameters and metrics to standardize the on-the-ground test data submitted in the challenge process and to assure more reliable challenges; a number of parties agree that such consistency among the apps used for challenges and rebuttals is important. This set of standardized parameters and metrics will also ensure that we can make a meaningful comparison of tests run by different entities using different methods (
e.g.,
tests run on a speed test app versus a government's own hardware and software), and will enable us to easily combine and evaluate speed test data used in the challenge process. Accordingly, we will require that such data meet the following testing parameters set forth in the
BDC Mobile Technical Requirements Proposed Rules:
(1) A minimum test length of 5 seconds and a maximum test length of 30 seconds; (2) test measurement results that have been averaged over the duration of the test (
i.e.,
total bits received divided by total test time); and (3) a restriction that tests must be conducted between the hours of 6:00 a.m. and 10:00 p.m. local time. To avoid requiring excessive data usage for tests on particularly fast networks (
e.g.,
5G-NR (New Radio) using high-band spectrum), we will relax the minimum test duration requirement once a download or upload test measurement has transferred at least 1,000 megabytes of data. Specifically, when a speed test transfers at least 1,000 megabytes of data, we will validate the test if it has a duration value of greater than 0 seconds and less than or equal to 30 seconds. Otherwise, a speed test must have a duration value of greater than or equal to 5 seconds and less than or equal to 30 seconds to be valid.

9. We clarify that the minimum and maximum test length parameters will apply individually to download speed, upload speed, and round-trip latency measurements, and will not include ramp up time. We disagree with the Competitive Carriers Association (CCA), Public Knowledge/New America, and Vermont Department of Public Service (Vermont DPS) that imposing a maximum test limit places an arbitrary or inferior limitation on testing. These timing requirements balance representative measurement over a stable Transmission Control Protocol (TCP) connection, on the one hand, versus data usage considerations, on the other hand—especially for consumers who may have limited data plans. The FCC Speed Test app, for example, first initiates a test server selection process, which typically takes two seconds (and a maximum of 10 seconds if servers fail to respond) then individually runs, including a warm-up time, a maximum of eight seconds for download and eight seconds for upload tests by establishing three concurrent TCP connections and summing the three resulting data rates for each test. In addition, the round-trip latency testing runs for a fixed five seconds to transmit up to 200 UDP (User Datagram Protocol) packets (
i.e.,
datagrams) to calculate the average latency of those datagrams. Hence, a typical test cycle takes approximately 23 seconds to complete, and a maximum of 31 seconds to complete.

10. We also decline to adopt CCA's request to exempt continuous network monitoring from the maximum test length. Continuous network monitoring software can monitor active users' speeds at the cell sites and other network parameters over extended periods of time. We are not persuaded that deviating from the uniform 30-second per test component maximum testing standard to accommodate continuous network monitoring will yield equal or more accurate test results. We found in the Mobility Fund Phase II challenge process that continuous network monitoring speed tests recorded significant variability within the same area and across a short time span, in some cases recording strong network performance well exceeding the minimum requirement interspersed with short seconds-long drops in performance that may have been the result of normal network conditions (
e.g.,
sector handover or network scheduling). The overall performance in these areas indicated that coverage was adequate (
i.e.,
with the average of tests in the same area over 15-20 seconds exceeding the minimum requirement), but because the test results were so variable, we are concerned that allowing the reporting of continuous speed tests could result in inaccurate results that do not reflect the typical on-the-ground customer experience, which as the results showed, may be adequate when averaged, but may not deliver consistent speeds to consumers. To the extent challengers choose to use continuous network monitoring to record challenge data, results of the speed tests should report the average speeds over a uniform time period consistent with the minimum and maximum test lengths we adopt above (
i.e.,
a minimum of 5 seconds and a maximum of 30 seconds).

11. We share Ookla's concern that averaging the number of bits received over the entire duration of a throughput

test may negatively affect the accuracy of any calculation, as that may not exclude an internet connection's known and expected “ramp-up time.” To account for this, we will apply the following formula: [(total bits received−ramp up bits) divided by (total test time−ramp up time)]. We consider “ramp up bits” to be the initial bits received during the initial warm-up time. We find that this approach will sufficiently account for ramp-up time and fully satisfy Ookla's concern, especially in light of the clarification above that the test time limits apply individually to tests' upload and download measurements.

12. We require on-the-ground speed test data to include a standardized set of metrics. Each on-the-ground speed test must include the following metrics that were previously adopted by the Commission as modified by the updates proposed in the
BDC Mobile Technical Requirements Proposed Rules:
(1) The timestamp and duration of each test metric; (2) geographic coordinates (
i.e.,
latitude/longitude) measured at the start and end of each test metric with typical Global Positioning System (GPS) Standard Positioning Service accuracy or better, along with the location accuracy (“location accuracy” refers to a metric that GPS-enabled smartphones are able to report on the horizontal accuracy of the geographic coordinates of the location reported); (3) the consumer-grade device type(s), brand/model, and operating system used for the test; (4) the name and identity of the service provider being tested; (5) location (
e.g.,
hostname or IP address) of the test server; (6) signal strength, signal quality, unique identifier, and other RF metrics of each serving cell, where available; (7) download speed; (8) upload speed; (9) round-trip latency; (10) for an in-vehicle test, the speed the vehicle was traveling when the test was taken, where available. All on-the-ground speed tests must also include the following metrics previously adopted by the Commission: (11) Whether the test was taken in an in-vehicle mobile or outdoor, pedestrian stationary environment (government and other third-party entities must also indicate whether an in-vehicle mobile test was conducted with the antenna outside of the vehicle); (12) an indication of whether the test failed to establish a connection with a mobile network at the time and location it was initiated; and (13) the network technology (
e.g.,
4G LTE (Long Term Evolution), 5G-NR) and spectrum bands used for the test. We adopt an additional metric that was proposed in the
BDC Mobile Technical Requirements Proposed Rules:
(14) The app name and version. We will also require all speed tests to include: (15) The timestamp that test measurement data were transmitted to the app developer's servers, as well as the source IP address and port of the device, as measured by the server. Given concerns that challengers may conduct tests after exceeding data limits, we will collect the timestamp that test measurement data were transmitted to the app developer's servers, as well as the source IP address and port of the device, as measured by the server, so that a service provider may determine if a challenger's device is subject to reduced speeds or otherwise lacks full network performance. The source port of the device is an available network port over which the device communicates with the server and is unique to a particular network connection or transmission. The IP address and source port associated with the device used in testing is attainable from devices using both iOS and Android devices. For the same reasons, we will allow government and other third-party entities to alternatively submit the IMEI of the device used to conduct the test rather than provide the source IP address, source port, and timestamp measured by an app developer's servers since such entities are allowed to use their own hardware or software to conduct speed tests. The purpose of collecting either type of data is to allow for the challenged provider to identify characteristics of the device or service plan used to conduct the test, such as whether the device was roaming or was subjected to slower service due to the subscriber's data plan. Accordingly, we will not require a service provider to submit either the device IMEI or the combination of source IP address, source port, and timestamp when submitting speed tests (either in response to a challenge or in response to a verification inquiry), as these fields are relevant only for data submitted by challengers.

13. Finally, we require on-the-ground challenge test data to include all other metrics required per the most recent specification for mobile test data adopted by OEA and WTB in accordance with 5 U.S.C. 553. Concurrent with release of this document, we are publishing the full technical and data specifications for mobile speed test data on the Commission's website at
https://www.fcc.gov/BroadbandData/resources.
The specification for speed test data includes additional fields derived from the high-level metrics defined herein, as well as other identifiers to facilitate management of the submission of such data. These fields include: a unique device installation ID; a unique test ID; the device Type Allocation Code (TAC); the Mobile Country Code (MCC) and Mobile Network Code (MNC) values measured from the network and from the device's SIM card; flags indicating whether the network is connected, is available, and/or is roaming; total bytes transferred and calculated bytes per second for download and upload tests; jitter and packets sent and received for latency tests; for each connected cell, the measured cell ID, Physical Cell Identity (PCI), cell connection status, Received Signal Strength Indication (RSSI), Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), Signal to Interference and Noise Ratio (SINR), Channel Quality Indicator (CQI), spectrum band and bandwidth, and Absolute Radio-Frequency Channel Number (ARFCN); and the horizontal accuracy of GPS coordinates and speed accuracy of measured velocity for each location measurement. Third-party app developers and government or other third parties that use their own hardware or software to conduct speed tests will be required to update their processes in accordance with such updates, including, as stated in the
BDC Mobile Technical Requirements Proposed Rules,
revised specifications for mobile test data adopted by the Bureau and Office in accordance with 5 U.S.C. 553. The modified set of parameters and metrics we adopt aligns more closely with those already required of government and third-party challengers. The Commission delegated authority to the Bureau and Offices to adopt additional testing requirements for government and third-party challengers. We therefore add certain metrics to those listed in paragraph 117 of the
Third Order
and § 1.7006(f) of the Commission's rules and make clear that all challengers must collect these metrics, with the exception that consumers need not indicate whether an in-vehicle mobile test was conducted with the antenna outside of the vehicle.

14. We recognize the concerns raised by Vermont DPS, Enablers, and Public Knowledge/New America about excessive data and burdens on consumers and governments and other third-party challengers to assure that their data aligns to these standards, but we believe that such parameters and metrics are necessary to provide the Commission with complete and reliable challenge data that accurately reflect on-the-ground conditions in the challenged

area and provide the additional context necessary to efficiently and fully adjudicate challenges and thereby assure that more accurate and reliable coverage maps are made available. These data metrics are also substantially similar to those adopted by the Commission in the
Third Order,
and therefore we do not anticipate that they will create any new burdens on consumers or governmental entities and third parties beyond those in place resulting from the previously adopted requirements. Further, the challenge process will remain user-friendly because any challenger can use a readily downloadable mobile app to collect and submit data (including the FCC Speed Test app, which the FCC makes available for download at no cost), and government and third-party entities have the flexibility also to use their own software or hardware. Therefore, government and other third parties will only need to modify their software once, to the extent necessary to conform to the required testing parameters and metrics we discuss above (and subject to our adopting any new metrics in the future). The Commission will also provide technical assistance to consumers and state, local, and Tribal governmental entities with respect to the challenge process, which will be a resource for government entities that do not understand some of our data collection requirements. The Bureau and Offices will ensure that the FCC Speed Test app and other apps approved for use in the challenge process collect this information, and government and other third-party challengers will be able to submit challenge data to the Commission through such apps under the procedures adopted for consumer challenges.

15. We understand that certain technical network information and RF metrics that we would otherwise require are not currently available on Apple iOS devices. The information we will use in the challenge process that can be collected from Android devices, but not iOS devices, includes the signal strength, signal quality, unique identifier, and other RF metrics of each serving cell, as well as the spectrum bands used for the test and other network characteristics (
e.g.,
whether the device was roaming, as well as the identity of the provider for the connected network). Therefore, until such time as such information and metrics are available on iOS devices, and the Bureau and Offices indicate they will collect such information from iOS devices, government and third-party entity challenges must use a device that is able to interface with drive test software and/or runs the Android operating system. The iOS operating system, which supports iPhone and iPad hardware devices, does not disclose certain technical network information and RF metrics that are essential to the Commission's challenge and crowdsource processes. This limits the conclusions that we can draw from on-the-ground tests conducted using such devices. OET will update its guidance if future iOS software versions are released that disclose this technical network information and/or RF metrics. To ensure that the challenge process remains user-friendly and encourage public participation, including by consumers who use a device running the iOS operating system, however, we will not extend this restriction to challenges submitted by consumers, and we will still consider speed test data submitted using an iOS device towards challenges. Although iOS software does not report the complete metrics we require in this document (
e.g.,
certain technical network information and RF metrics), the Bureau and Offices will nevertheless use the remaining on-the-ground data we receive from consumers using iOS software in the challenge process. Although we may receive limited data from tests run on iOS devices, we do not anticipate that such tests will significantly impede the creation of challenges because, as mentioned, the Commission will aggregate speed tests to create cognizable challenges. iOS speed tests will be considered in combination with other speed tests that fall within the same resolution 8 hexagon. We therefore anticipate that data submitted by government and other entities, as well as consumer tests run on Android devices, will help fill in any gaps in information about the on-the-ground quality and availability of broadband coverage that may result from the limited nature of the data we receive from speed tests run on iOS devices. Our approach preserves balance and flexibility for both types of challengers, while also ensuring that the Commission gathers adequate data to adjudicate challenges. On the one hand, government and other third-party entities who can be expected to submit large amounts of speed test data may not use iOS devices but have the flexibility to use their own hardware and software. On the other hand, consumers who use iOS devices and would face a prohibitive burden if required to use a non-iOS device to submit a challenge may submit speed tests conducted using an iOS device but do not have the same flexibility as government and other entities to use non-approved software.

16. Third-party app developers and government or other entities that use their own hardware or software to conduct speed tests will be required to update their processes in accordance with updates to the full technical and data specifications for mobile speed test data, including, as stated in the
BDC Mobile Technical Requirements Proposed Rules,
revised specifications for mobile test data adopted by the Bureau and Offices. The Rural Wireless Association (RWA) asserts that adopting the proposed data metrics and parameters, including “all other metrics required per the most-recent specification for mobile test data released by OEA and WTB” would be an improper incorporation by reference that violates the Office of the Federal Register (OFR) regulations and the Administrative Procedure Act (APA). We disagree with RWA that this is an improper incorporation by reference that violates OFR regulations and the APA. The metrics we require are substantially similar to those already adopted by the Commission in the
Third Order,
and have been adopted after notice and comment in accordance with the APA's rulemaking requirements. Furthermore, we note that certain changes to the specifications that apply to the submission of on-the-ground test data, including for example, changing the file type to be submitted, are not substantive changes, and may be adopted without notice and comment. The Bureau and Offices have been delegated authority to adopt such procedural changes pursuant to § 1.7010 of the Commission's rules. To the extent that we may wish to make any substantive changes to testing parameters or metrics, we clarify that we would make such changes in accordance with 5 U.S.C. 553. Any future changes we make to the testing parameters or metrics will also be consistent with the Commission's Orders implementing the Broadband DATA Act. Finally, the adoption of these rules will not result in an improper incorporation by reference because we will comply with the requirements of any applicable Federal statutes and regulations governing the publication of these test parameters and metrics in the
Federal Register
and the Code of Federal Regulations.

17.
Speed Test Applications.
Pursuant to the Commission's directive in the
Third Order,
OET is currently in the process of developing updates to the FCC Speed Test app to incorporate additional functionalities that will allow for its use in submitting speed test data

as part of the BDC mobile challenge and crowdsource processes. OET recently released a technical description of the metrics and methodologies used in the current version of the FCC Speed Test app. The revised technical description document includes updated technical standards and additional modifications. While this document does not illustrate future user experience design changes to the FCC Speed Test app that will be made to implement the challenge and crowdsource functionalities, we anticipate that the fundamental measurement methodologies reflected in the recently updated technical description document will not be affected by these design updates. We note that the description includes the following about the test system architecture: “The measurement servers, each supporting a 100 [gigabit per second] Gbps capacity, used for mobile broadband measurement are hosted by StackPath and are distributed nationally to enable a measurement client to select the host server with the least latency.” The technical description includes data dictionaries for both Android and iOS versions of the app, but these dictionaries define data fields and formats for the current version of the app (and not the updated version of the app). To provide third-party app developers and other stakeholders with information and guidance as early in the process as possible, the Bureau and Offices have made available, contemporaneous with the release of this document, a current draft of the data specification the FCC Speed Test app will use once updated to include challenge and crowdsource data functionalities. The updated data specification aligns with the test metrics adopted in this document. The updated FCC Speed Test app with those functionalities will be available on the FCC's website and in iOS and Android app stores prior to the opening of the challenge and crowdsource process.

18. We decline to provide a further opportunity for comment on the FCC Speed Test app. Although some parties request an opportunity for public comment on both the FCC Speed Test app and third-party apps before we allow them to be used in the challenge process, we note that the Commission already sought comment on the use of the FCC Speed Test app in the challenge process as part of this rulemaking proceeding. The Commission also provided other opportunities to comment on the FCC Speed Test app because (1) the app was initially developed in coordination with the major wireless providers and trade associations several years ago; and (2) information on the data collected by the app has been publicly available on the Commission's website and has been available for comment in a rulemaking docket for several years. Additionally, the Commission specified in the
Third Order
that the challenge process use an FCC app, and, unlike some newer third-party speed test apps, the FCC Speed Test app has been in use for several years and the updates that are underway will merely implement the data specifications and requirements proposed in the
BDC Mobile Technical Requirements Proposed Rules
and adopted by this document. For these reasons, we do not believe it is necessary to seek comment on the use of the FCC Speed Test app for challenge and verification purposes.

19. CCA and RWA assert that it is unclear how the FCC Speed Test app will operate when there is inadequate connectivity to upload data or record a test. The FCC Speed Test app is designed to record and store measurements conducted in areas without internet connectivity and then to automatically transmit such failed tests once the app is opened when the device next has broadband connectivity. Moreover, third-party apps will be required to function in a similar way to be granted approval for use in the challenge process. Several commenters likewise misunderstand how the FCC Speed Test App reports “failed” tests or tests where mobile service is unavailable. As set forth in the 2021 technical description of the FCC Speed Test app, “test[ ] results are transferred depending on the available connectivity at the conclusion of the test and can be stored and forwarded when connectivity is immediately unavailable.” Failed test results are therefore uploaded to the server and included in the relevant dataset when the app user reestablishes a broadband connection. The upload and download components of a failed test will be recorded as negative if they fail to meet the minimum speeds that the mobile service provider reports as available where the test occurred. For example, if a failed test records speeds of 0 megabits per second (Mbps) upload and 0 Mbps download, both components of the test will be recorded as negative.

20. At a later date, OET will release a public notice outlining the process for collecting, reviewing, and approving applications for third-party speed test apps. In their applications, app developers will be required to describe their performance-centric speed test methodologies and how their app complies with the data collection requirements set forth in this document. Applicants will not be required to disclose any proprietary and/or confidential information that is sensitive to public inspection, such as source code, to the Commission, and we therefore decline to adopt T-Mobile's request that we require developers to submit their source code for public review. The OET public notice also will describe procedures for interested parties to submit comments and replies in response to the proposals and will publish on the Commission's website a list of approved third-party apps and any available data specifications for third-party apps.

21. We agree with commenters who recommend holding the FCC Speed Test app and third-party apps to the same technical standards. Both the FCC Speed Test app and third-party apps, as well as software used by state and local governments and other third parties, must comply with the data collection requirements set forth in this document. We also agree with commenters who recommend requiring speed test apps to use multiple servers that are geographically diverse. As to this point, CCA asserts that Ookla's speed test app is more accurate than the FCC Speed Test app due in major part to its many geographically distributed servers (with 41 servers in the U.S. and 15,019 testing servers globally), which allow users to run a test against a server that is located physically close to them reducing the likelihood of inaccurate latency measurements or artificial increases in latency distorting the download and upload speeds. As described in the most recent technical description for the FCC Speed Test app, the app currently carries out measurements against 13 servers spread out across ten locations throughout the United States and initiates a test sequence by selecting a measurement server using a latency test to identify the optimal server that has the lowest round-trip latency for performing subsequent tests. We believe that the current distribution of FCC Speed Test app servers, combined with this measurement server selection process, provides sufficient diversity to meet this geographic-diversity criterion. We also note that the number of servers used by a speed test is of less concern than the ratio of the concurrent consumers conducting tests to the total capacity of the test server hosting those tests (
i.e.,
the server utilization rate). The FCC Speed Test app's test servers are overprovisioned based upon statistics of the utilization rate and usage pattern, which are automatically monitored for the highest system

availability, to maintain the optimal connectivity rate. A utilization rate of 80% or more is classified as a critical state, and triggers the provisioning of new servers to stabilize load across the platform. Accordingly, although not as geographically diverse as Ookla's speed test app, we believe that the geographic diversity offered by the FCC Speed Test app in the United States provides sufficient capacity to support its user base and that it is sufficiently diverse to meet the required needs that rely on the test system architecture. The test system architecture for multiple redundant and meshed servers to target maximum availability of the test platform also employs load balancing for traffic to failover to other servers in which each server provides a 100 Gbps connectivity capacity. In sum, the FCC Speed Test app provides sufficient capacity to support its users and has sufficient geographic diversity to meet the required needs of the test system architecture. We also observe that latency is the principal concern raised by commenters. In this regard, we note that Commission rules require measurement of round-trip latency. As adopted and implemented in the FCC Speed Test app, the variability of latency is not entirely a function of geographical distance to the test server but also is a function of the network congestion, and so, at a minimum, servers should be distributed nationally in consideration of user base, population density, and the server utilization rate for multiple servers to be examined before the test server selection and located reasonably close to internet eXchange Points (IXPs) to accurately reflect unbiased real-world conditions. We point out that the FCC Speed Test app sufficiently considers these effects to help reduce round-trip latency.

22.
Validating Speed Tests.
As proposed in the
BDC Mobile Technical Requirements Proposed Rules,
we will validate submitted speed tests and exclude those that: (i) Are outside the scope of the challenge process, (ii) do not conform to the data specifications, or (iii) do not otherwise present reliable evidence. We will accept as valid speed tests only those tests conducted between the hours of 6:00 a.m. and 10:00 p.m. local time. Commenters do not raise concerns with our adopting a window for purposes of validating speed tests. We will compare speed tests for a particular network technology (
e.g.,
3G, 4G LTE, or 5G-NR) to the coverage maps for the corresponding technology or higher-generation technology, to the extent the service provider claims coverage for the more than one technology in the tested location. We implement these changes so that testers are able to submit tests to be used to challenge a higher-generation technology map in situations when a mobile service provider claims multiple technologies at a location but the tester's device only connects to a lower-generation technology. We agree with Vermont DPS that our original proposal did not adequately address those situations where a device that is unable to connect to a network using a particular technology “falls back” to a lower-generation technology (
e.g.,
4G LTE to 3G), which could make it impossible to challenge the higher-generation technology. We will allow, therefore, a speed test conducted using a device capable of connecting to a higher-generation technology, but that only connects to a lower-generation technology, to count as a test for the higher-generation technology. To be a valid test for the higher-generation technology, the consumer submitting the challenge must also subscribe to a service plan that is capable of connecting to the provider's network using the higher-generation technology. To prevent gaming, and as discussed further below, we will allow challenged providers to invalidate challenger speed tests with specific evidence that the challenger's device was not capable of connecting using a higher-generation technology or that the service plan to which the challenger subscribes does not allow use of the higher-generation technology. For example, a test conducted with a 4G LTE-capable device in a location where the service provider claims 4G LTE but where the challenger can only connect via the 3G network could count as both a 3G test when compared to the provider's 3G coverage map as well as a negative 4G LTE test when compared to its 4G LTE coverage map if the test did not meet the 5/1 Mbps minimum speeds; alternatively, it could count as a positive 4G test if the test met or surpassed the 5/1 Mbps minimum speeds reported for the 4G LTE map. Note that, under this approach, the 3G test may count towards the 4G LTE coverage map regardless of whether the provider claims 3G coverage at the location. This modified approach would resolve Vermont DPS's hypothetical concern that, under the proposal set forth in the
BDC Mobile Technical Requirements Proposed Rules,
a test result that “fell back” to a lower-generation technology would not be “preserved.” As discussed, such tests will be preserved and used to challenge a higher technology's maps if a service provider offers a higher-generation service in that area and the tester subscribes to a service plan that is capable of connecting to the provider's network using the higher-generation technology.

23. Similarly, if a challenger conducts a test but fails to connect to any network, we will treat that as a failed test against the provider's coverage maps for each technology to which the device is capable of connecting. These small changes to our original proposal will help prevent the scenario raised by Vermont DPS and enable more meaningful challenges in areas with marginal coverage where a device “falls back” to a lower-generation technology. Our updated approach also accounts for situations in which a device could alternate between, or utilize both, 4G LTE and 5G-NR over the course of a single test. Verizon agrees with the Bureau and Offices' initial proposal to compare each speed test against the relevant coverage map, and argues that “only speed tests conducted on 3G networks should be used to challenge 3G coverage, only speed tests conducted on 4G LTE networks should be used to challenge 4G LTE coverage, and only speed tests conducted on 5G-NR networks should be used to challenge 5G-NR coverage.” However, we are persuaded that the proposal we sought comment on in the
BDC Mobile Technical Requirements Proposed Rules
could allow for a scenario in which a tester seeking to support a challenge to a provider's 5G coverage would be prevented from submitting evidence because their phone fell back to the 4G network. Under our original proposal, in areas where a provider claims coverage for multiple technologies, a lower-generation technology could have prevented the higher-generation technology from being challenged, which in turn could isolate higher-generation technologies from legitimate challenges.

24. We will also compare speed tests conducted in a particular environment—outdoor stationary or in-vehicle mobile—to where the provider's maps report coverage for the corresponding environment (
i.e.,
outdoor stationary or in-vehicle mobile), as discussed in greater detail below. Additionally, we will also treat as invalid and exclude from the challenge process any speed tests that fall outside the boundaries of the provider's most recent coverage data for all claimed technologies and environments. This differs from our original proposal in the
BDC Mobile Technical Requirements Proposed Rules
in that the system will preserve all tests in a geographic area

where a provider claims coverage by any technology. We believe our modified approach will result in more reliable evidence for challenges because tests that may otherwise have been excluded for falling outside a provider's coverage for a specific technology under the proposed methodology in the
BDC Mobile Technical Requirements Proposed Rules
may now be counted as challenge data. This change will allow for the scenarios discussed above, in which a test conducted using a lower-generation technology could be used to challenge a provider's map for a higher-generation technology if the provider claims both types of coverage (
e.g.,
4G LTE and 5G-NR), but a challenger's device is not connected to the higher-generation technology.

25. In response to Verizon's concerns that tests may be throttled, we will not validate, for purposes of the challenge process, speed tests conducted by customers of mobile virtual network operators (MVNOs) or tests conducted while roaming on another carrier's network, so as to avoid biasing the challenge process with speed tests that may not reflect typical network performance. MVNOs do not own any network facilities. Instead, they purchase mobile wireless service wholesale from facilities-based service providers and resell these services to consumers. Because the agreements between a facilities-based provider and MVNOs or roaming partners often include limitations on the technology and speed available to or the network prioritization of devices used by consumers of the MVNO or roaming partner, we conclude that speed tests from such devices are not reliable evidence about the performance of the facilities-based provider's network. While we anticipate that the majority of tests conducted by an MVNO subscriber or while roaming will fail our automated validations, there may be circumstances where the BDC system is unable to automatically identify these tests (
e.g.,
identifying whether an iOS device is roaming is not currently possible). We anticipate that a provider may identify whether a specific device(s) used in the testing was either roaming at the time, was an MVNO customer, or was subject to deprioritized or otherwise limited service because, as discussed, on-the-ground speed tests submitted in the challenge process will include the timestamp that test measurement data were transmitted to the app developer's servers, as well as the source IP address and port of the device, as measured by the server. We therefore do not agree with Vermont DPS's assertion that pre-paid tests in rural areas will be less accurate than speed tests run by subscribers of a typical service provider, due to the fact that pre-paid services exclude roaming in rural areas, because we will not validate any tests conducted while a subscriber is roaming. While we will allow a service provider's pre-paid customers to submit speed tests for use in the challenge process, a service provider will be able to use the timestamp that test measurement data were transmitted to the app developer's servers, as well as the source IP address, and port of the device, as measured by the server to determine if a specific speed test is run by a pre-paid subscriber that experienced limited service, and use that information when responding to a challenge. Given that these consumers may likely be subject to de-prioritization or otherwise limited service, and that the BDC system will be unable to detect whether or not a limitation in mobile service exists, we are unable to establish a reliable method for validating MVNO or roaming tests and, thus, these tests will be excluded from the challenge process. As discussed later, however, we may consider speed tests conducted by consumers of MVNOs and consumers roaming on other providers' networks when evaluating crowdsourced data.

26.
Aggregating Valid Speed Tests.
The Bureau and Offices will combine and collectively evaluate—according to the testing environment (
i.e.,
outdoor stationary or in-vehicle mobile) and technology type—valid speed tests submitted by consumer, governmental, and third-party challengers. Speed tests, including those collected through an approved speed test app and the data collected by government and other third-party entities through their own software and hardware, will be combined and collectively evaluated according to their tested environment and technology type. For example, as discussed in greater detail below, in-vehicle tests will generally be evaluated against a carrier's in-vehicle maps, and stationary tests will generally be compared against a carrier's stationary maps. We expect that in-vehicle and stationary tests will have substantially different results such that they would not provide an equal comparison and aggregating these tests would be problematic because there are fundamental characteristics of the two environments that are expected to cause noticeable signal losses for the in-vehicle mobile environment. As noted above, we do not expect iOS and Android devices to pose a similar problem. While we will receive a more complete set of datapoints from Android tests than iOS tests, we do not expect them to have substantially different results when, for example, tests using both types of devices are conducted in a pedestrian stationary environment, such that the tests would not have equal value and could not be compared and aggregated; the fact that iOS provides fewer datapoints than Android tests does not render a test run using iOS any less accurate than a test run using the Android operating system. Similarly, tests conducted with an external antenna will be considered in-vehicle, and while subtle differences between test results from those antenna placements may occur, overall those differences are considerably less significant than the differences between stationary vs. in-vehicle mobile more broadly.

27. We will combine such speed test evidence and apply a single methodology to determine whether the thresholds for a cognizable challenge (described in greater detail below) have been met and the boundaries of the challenged area. Several commenters express support for aggregating speed tests from multiple challengers, and we find that doing so will result in more accurate challenges and will further the Commission's goals of resolving challenges in an efficient manner, mitigating time and expense, and ensuring that maps are as reliable and useful as possible. We disagree with the California Public Utilities Commission's (CPUC) assertion that combining speed test data will not reduce costs or complexity in the challenge process. In fact, combining speed tests could ease the other potential burdens on an individual challenger of conducting multiple speed tests to meet the challenge thresholds. Our approach ensures that a smaller number of speed tests by one person or entity may nevertheless contribute to a challenge because the tests will be combined with other validated speed tests to meet the testing, temporal, and geographic thresholds. As a result, in many cases, no single challenger—whether a consumer, a government agency, or other entity—will be required to individually shoulder the burden of creating a challenge. While in places with low population density an individual challenger may be the only entity to submit speed tests to create a cognizable challenge, in many other cases, challengers will be able to combine efforts to submit speed tests in an area. Speed tests will be combined and used collectively—according to

testing environment (
i.e.,
outdoor stationary or in-vehicle mobile) and technology type—to meet the thresholds set forth below.

28. We will evaluate tests for a given technology against each provider map independently (one reporting stationary and one reporting in-vehicle mobile coverage) when determining whether to establish a cognizable challenge. Pursuant to the
Third Order,
tests taken on bicycles and motorcycles will be considered tests from in-vehicle mobile environments. We will consider in-motion tests taken in similar environments, such as on snowmobile or all-terrain vehicle, to be tests from in-vehicle mobile environments. By contrast, consistent with the
Third Order,
tests taken from stationary positions and tests taken at pedestrian walking speeds (such as on horseback) will be considered tests taken in outdoor pedestrian environments. We decline to exclude tests taken on other vehicles as T-Mobile requests. The Commission did not give the Bureau and Offices authority to change this accommodation; we anticipate that challengers may take speed tests on other vehicles than cars in areas with difficult or hard to reach terrain. Additionally, we will exclude stationary tests that occur outside a provider's stationary coverage map and in-vehicle mobile tests that occur outside a provider's in-vehicle mobile coverage map. Our approach differs from that which we proposed in the
BDC Mobile Technical Requirements Proposed Rules
in that we will no longer aggregate in-vehicle and stationary maps together. We find that the approach we adopt will result in more accurate challenges. To ensure that the challenge process also remains user-friendly, and because we expect performance to be better for stationary tests than for in-vehicle tests, stationary speed test results that create a cognizable challenge to an area on the stationary map will also create a cognizable challenge to the same area on the in-vehicle map if the area has overlapping coverage on both maps. On the other hand, the reverse situation will not be permitted, meaning, we will not permit a challenge to an area on the in-vehicle map to automatically create a challenge to the same area on the stationary map if the area has coverage on both maps. If, however, in an area that has coverage on both maps we find that large portions of a provider's in-vehicle mobile map have been successfully challenged, but there are very few speed tests conducted in a stationary environment, then we may use this as evidence upon which to form a credible basis for initiating a verification inquiry of a provider's stationary coverage in that area. Similarly, a provider refuting a challenge to a geographic area on the in-vehicle map would also refute a challenge to the same area on the stationary map if that challenge exists.

29. Several providers express concern about the proposal to aggregate in-vehicle mobile and outdoor stationary tests and compare them collectively against both coverage maps. As described above, we will not aggregate all stationary and in-vehicle mobile tests for comparison against both maps but will evaluate stationary tests against the stationary map and the in-vehicle mobile tests against the in-vehicle map. Rather than aggregating all tests, we will allow cognizable challenges to the stationary map to also create a challenge for the same area on the in-vehicle map and successful provider responses to the in-vehicle map to also refute a cognizable challenge of the same area on the stationary map. We find that this approach adequately addresses providers' concerns about comparing tests from different modeled environments, and promotes consistency between the maps. We thus decline to adopt the Vermont DPS's recommendation to allow challengers to submit in-motion tests to challenge stationary coverage, because we do not expect in-vehicle tests to achieve the same performance had the test been conducted in a stationary environment. If we did not allow for challenge or response comparison to both maps in the limited circumstances we adopt above, it would be easier for one map in an area to show a lack of coverage while the other map shows robust coverage—solely because of a lack of testing.

30. Data from speed tests taken after the “as-of” date of the initial BDC data collection will be considered as part of the challenge process upon confirmation that they meet the validation criteria set forth herein. Accordingly, once the Commission has generated maps of the data collected from providers, the BDC system will analyze all previously submitted tests to determine whether they were taken after the “as-of” date of the maps and to perform the data validations discussed further below, including whether they were taken within the published coverage area claimed by the applicable provider. Speed tests submitted as part of the challenge process that do not meet these qualifications will be considered crowdsourced data. Validated speed tests results will be reconsidered on a monthly basis, in conjunction with any newly validated speed test filings, to determine whether the data meet the geographic, temporal, and testing thresholds to create a cognizable challenge to an area. Such speed tests will be considered for up to one year to determine whether the data for a location subsequently meet the thresholds to be considered a cognizable challenge, and if so, the tests will be used collectively to challenge the maps that are published at that time.

31. Once the maps have been published, the BDC system will analyze all submitted tests to determine whether speed tests fall within the geographic area depicted in a provider's published coverage area. Speed tests submitted after the “as of” date but prior to publication of the map, as well as those submitted after the publication of the maps, will be used to challenge the maps that are published at that time, subject to the restriction that speed tests are considered valid evidence for one year from the date the test was taken. During the one-year period that they remain valid evidence, speed tests may initially be excluded from consideration in the challenge process because the speed tests fell outside of the provider's reported coverage maps but be included when the system reconsiders the challenge data every month, due to subsequent publication of maps reporting coverage in which such tests are located. For example, if a challenger submits otherwise valid speeds tests that were conducted in July in an area reported by the provider to not have coverage in its maps that are “as of” the previous December 31, such tests would be initially excluded. If the coverage maps submitted by the provider “as of” June 30 and published in September of that year do report such areas as covered however, the tests taken in July would be considered as valid evidence in favor of a challenge to the June 30 maps. Parties submitting speed tests to be used in the challenge process will be notified when their test has been submitted and that the test submitted may be used to create a challenge if such data meet the validation requirements. Thereafter, parties that have submitted speed tests to be used in the challenge process will be notified of the status of their submitted speed tests, which will include information on whether their speed test is used in the creation of a cognizable challenge.

32.
Maps That Can Be Challenged.
We clarify that speed test data will only be used to create challenges in areas where a provider reports that it has broadband service availability. We will, however, permit challenges to 3G, 4G LTE, and 5G-NR coverage maps. Some

commenters suggest that we defer consideration of challenges to 3G maps, but the Commission has classified 3G as a mobile broadband technology in previous BDC orders and has determined to allow challenges to the accuracy of mobile broadband coverage maps. Since the Commission did not delegate to the Bureau and Offices the authority to limit challenges to certain technologies, we lack the discretion to limit challenges to only 4G LTE and 5G-NR maps. Moreover, doing so could exclude certain consumers from the challenge process. For example, consumers rely on 3G in areas where 4G LTE and 5G-NR are not offered by the provider or are otherwise unreliable, and subscribers in rural areas continue to use 3G at higher concentrations than other parts of the country. We note that, when a provider retires a given mobile broadband technology such as 3G, that service would not be included on its updated coverage maps and therefore would not be available for challenges. However, until providers retire a particular broadband network technology, they will be obligated to respond to challenges to their claims of coverage for that technology.

33. Based on the record and the goals underlying the Broadband DATA Act, we adopt our proposal to exclude voice maps from the challenge process. The Broadband DATA Act requires the Commission to establish a process for challenging the accuracy of broadband coverage data, which, for mobile services, is defined as “the coverage maps” (
i.e.,
the broadband maps discussed in 47 U.S.C. 642 (c)(1)) and “any information submitted by a provider regarding the availability of broadband internet access service.” Additionally, the Commission has decided that the mobile challenge process applies only to broadband (and not voice) coverage maps. We also note that commenters raise concerns with using “speed test” data to verify voice coverage maps. Vermont DPS disagrees, proposing that the Bureau and Offices should set parameters for voice maps, including defining a threshold signal level of upload and download speeds that would indicate voice service is available in an area. We reject the Vermont DPS proposal. Vermont DPS was the only commenter to proffer minimum throughput parameters (
i.e.,
download and upload speeds) or signal strength values necessary to support a voice call, but these values did not receive any additional record support. Although Vermont DPS recommends that the Bureau and Offices determine threshold parameters that “would be indicative of no mobile service,” it does not propose specific parameters, noting only that zero would be indicative of no service and that 256 kilobits per second (kbps) download, 64 kbps upload, or a signal strength of less than −105 decibel-milliwatts (dBm) would indicate that service is likely insufficient. We therefore decline to include voice maps as part of the mobile challenge process at this time.

34. Additionally, we reject commenters' requests to allow challenges only to outdoor stationary coverage maps. CTIA—The Wireless Association, Verizon, T-Mobile, and AT&T argue that the Commission should focus first on challenges to outdoor stationary maps, and defer consideration of any challenges to in-vehicle maps until after it has ruled on CTIA's petition for reconsideration to eliminate in-vehicle coverage maps. The Commission's
Third Order
clearly directed that we collect both sets of maps, and we will not eliminate or delay the challenge process for in-vehicle maps given the importance in making the challenge process available for consumers and other entities that use mobile services in vehicles, unless the Commission determines that such maps are not necessary. CTIA, Verizon, T-Mobile, and AT&T also argue that in-vehicle maps should be excluded from the challenge process because the Commission has not established parameters for mapping in-vehicle coverage or evaluating in-vehicle challenges. Limiting the challenge process to outdoor stationary tests and maps could reduce the utility and accuracy of the challenge process, given that many consumers use mobile services in vehicles and in motion. We recognize that many states ban handset use while driving and many vehicle operators do not have passengers. We do not intend to contravene state bans on handset use while driving, nor do we advocate for consumers to run speed tests on a personal handset while operating a vehicle. It also would ignore a significant number of speed tests, especially on highways and in areas where it is not safe or convenient to conduct stationary speed tests. Moreover, the Commission has established sufficient parameters for mapping in-vehicle coverage and evaluating in-vehicle challenges. The Commission has allowed consumers to conduct speed tests in an in-vehicle mobile environment, but declined to adopt detailed testing requirements for in-vehicle consumer tests, whereas it required government and third-party challengers to submit more detailed information on tests run in in-vehicle mobile environments. We reiterate that all challengers must report whether the test was taken in an in-vehicle mobile or outdoor pedestrian environment; for in-vehicle tests, the speed the vehicle was traveling when in-vehicle tests were taken (where available); and, for government and other third-party challengers conducting in-vehicle tests, whether the test was conducted with an antenna located outside of the vehicle.

35. Finally, we decline to adopt Vermont DPS's request to change the thresholds for in-vehicle tests “to account for the slight difference in performance of stationary and mobile tests” because, as discussed, we will not use in-vehicle test data to form the basis of a challenge of stationary maps. Moreover, Vermont DPS has not given us any objective metric by which to adjust tests upward or downward for purposes of meeting the threshold when comparing the test against the other environment (
i.e.,
Vermont does not suggest any formula to accurately estimate actual performance (based upon,
e.g.,
signal strength) and thus, there is no way we could translate signal strength into actual speeds).

36. We also reject suggestions that we permit challenges only in rural areas. The Broadband DATA Act envisions a broad challenge process, and there is nothing in the Act that authorizes the Commission, or by extension, the Bureau and Offices, to limit the challenge process to rural areas.

37.
Grouping Valid Speed Tests by Location.
After excluding speed tests that fail our validations, we will associate the location of each valid speed test with a particular underlying hexagonal cell geography based on the H3 geospatial indexing system. The H3 system is designed with a nested structure wherein a lower resolution cell (the “parent” hexagon) contains approximately seven hexagonal cells at the next higher resolution (its “children” and each a “child” hexagon), which approximately fit within the “parent” hexagon. The lower the resolution, the larger the area of the hexagonal cell. Because of this nested structure, using the H3 system to group speed tests allows for challenges at multiple levels of granularity which, as discussed below, enables challengers in rural areas where broadband coverage may be more sporadic to contest larger areas if aggregated speed test data demonstrate a lack of coverage within a sufficient number of child hexagons. As proposed, the smallest cognizable challenge will be to a single resolution 8 hexagonal cell, which has an area of approximately 0.7 square kilometers.

38. Some commenters support the use of hexagons to evaluate challenges but recommend basing challenges on a different hexagonal cell size. While Vermont DPS generally supports the proposed use of H3 indexing, it argues that the system is not intuitive to use and asks the Commission to create and share geospatial indexing system (GIS) layers for the H3 hexagons at all resolutions it intends to employ in the coverage analysis, which we have already done. CTIA, T-Mobile, and AT&T urge us to use smaller resolution 10 hexagons instead of resolution 8, contending that hexagons at resolution 10 better match the 100-meter resolution providers must use when submitting their coverage maps. RWA and Vermont DPS, meanwhile, recommend allowing challenges to resolution 6 and 7 hexagons in rural areas, which RWA notes are often difficult to test because of a lack of accessible roads.

39. We find that resolution 8 strikes an appropriate balance as the smallest resolution for a cognizable challenge. Smaller areas (
e.g.,
resolution 9 or 10) could result in many disparate challenges that may require excessive testing by providers and, in the case of resolution 10 hexagons, may exceed the granularity of propagation maps that were not designed to provide such precision. Coverage maps must be submitted at a resolution of 100 meters (
i.e.,
0.1 km) or better. Therefore, allowing for challenges to an area as small as a resolution 10 hexagon cell, which is smaller than the 100 meter map resolution, may instead reflect inaccuracies due to the resolution at which the provider generated its maps. Larger areas (
e.g.,
resolution 6 or 7 hexagons), on the other hand, would require significantly more testing for challengers and make it difficult to verify coverage in distinct local areas. For example, a resolution 7 hexagon would require four to seven times as many tests as a resolution 8 hexagon to create a successful challenge. The Commission directed staff to determine the requisite number of tests and define the geographic boundaries of cognizable challenges while satisfying the goals of both “encourag[ing] consumers to participate in the challenge process and assuring that providers are not subject to the undue cost of responding to a large number of challenges to very small areas.” We are not persuaded that allowing challenges to areas smaller than the 100-meter resolution (
i.e.,
a resolution 10 hexagon) requirement would adequately meet these goals. Using areas smaller than a resolution 8 hexagon would additionally make it difficult for consumers to reach the threshold of cognizable challenges. A challenger would need to take many more tests in the smaller hexagons to achieve the statistical significance required. Use of particularly small areas also would likely make in-motion testing for both challengers and providers impossible. In the future, we may consider using hexagonal cells at a higher resolution if it becomes necessary to correct coverage errors at a more granular level.

40. RWA and Verizon assert that the use of the H3 geospatial indexing system would present implementation issues. RWA cautions that third-party network maps, which providers may use to supplement the data used to rebut challenges, may not be compatible with the H3 geospatial indexing system. Verizon also raises concerns that providers would need to develop new tools and systems for managing speed tests and evaluating data in an H3-based environment and notes that tracking and evaluation may be complicated because child cells will not nest precisely into their parent cell. These concerns do not warrant deviations from our proposal since parties seeking to rebut challenges do not need to conform their tools or data to the H3 indexing system. The BDC system itself will overlay submitted speed test points with the H3 hexagons; providers need only submit their speed test data and the BDC system will appropriately index them (so long as the data otherwise meet the specifications and test requirements to qualify as valid on-the-ground speed test data). Moreover, H3 is an open-source indexing system, and therefore we do not anticipate it being overly expensive or burdensome for providers to access. Finally, in response to Verizon's argument that the tracking and evaluation of speed test data will be complicated because child cells will not nest precisely into their parent cell, we note that speed tests will be evaluated based on the resolution 8 hexagon within which a test falls.

41. CPUC and Public Knowledge/New America assert that submitting speed test data under the H3 system using resolution 8 hexagons would be more burdensome and expensive, and would result in fewer challenges, because challengers would need to gather statewide measurements in each resolution 8 hexagon. We disagree. First, challengers will not need to submit speed tests in every resolution 8 hexagon in a state because challenge data cannot form the basis of a cognizable challenge in areas where a provider does not claim coverage. Challengers will be aware of the areas in which a provider does not claim coverage from the publicly available mobile broadband map and can avoid the burden and expense of conducting speed tests in those areas. Second, as discussed, we will combine, according to the tested environment, valid speed tests conducted by consumers, state, local, and Tribal governments, and other entities. This likely will reduce the number of speed tests that any one challenger needs to submit to create a challenge. The number of required tests needed to meet the thresholds reflect the
total
number of speed tests needed to create a cognizable challenge, not necessarily the number of speed tests that must be submitted by an individual consumer or entity. Third, CPUC's concerns ignore our decision to allow testers to challenge larger geographic areas, such as resolution 7 hexagons or resolution 6 hexagons, when at least four of the seven child hexagons of the parent hexagon are challenged. Testers will be able to see which areas have been challenged and if, for example, four or more of the seven child-resolution 8 hexagons in a resolution 7 hexagon are challenged, then the entire resolution 7 hexagon will be considered challenged. Finally, H3 indexing will not burden testers because it will serve as an “under the hood” way for the Commission to group and analyze speed tests submitted by testers at various times and places.

42. We will evaluate all valid challenger speed tests that present evidence about the service of a given technology and environment within each hexagon to determine whether to create a cognizable challenge to the coverage in that area. We did not receive any comments on this proposal. We also adopt the alternative approach proposed in the
BDC Mobile Technical Requirements Proposed Rules
to evaluate the download and upload components of each speed test individually rather than evaluating them jointly. Under this approach, each component will be categorized as either “positive” or “negative” based on whether the component is consistent with the provider's modeled coverage (
i.e.,
the coverage assumptions in providers' BDC propagation maps). A positive component is one that records speeds meeting or exceeding the minimum speeds that the mobile service provider reports as available where the test occurred. A negative component is one that records speeds that fail to meet the minimum speeds that the mobile service provider reports as available where the test occurred. For each speed test, the download component will be

either positive or negative, and the upload component will be either positive or negative. The coverage map will then be evaluated for all download tests and separately for all upload tests. If a resolution 8 hexagon meets the thresholds for either upload or download tests, a challenge would be triggered. In order to rebut a challenge, a provider would need to meet the thresholds for both the upload components and download components. Speed test apps typically measure download and upload components sequentially and not simultaneously, so evaluating these components independently will better account for geographic and/or temporal variability.

43. In the case where the starting and ending locations of a test are in different hexagons (
e.g.,
because the testing device was in motion), we will associate the test with the hexagon containing the midpoint of the reported start and end coordinates for each test component. We also will use the midpoint to determine whether the test component falls within the applicable provider's coverage map. Each test component will be point-hex dependent. Therefore, a download test could be associated with a different point-hex than an upload test, and in such cases, the two tests would be treated independently. We disagree with Ookla that we should use the start location as the single point value of a test rather than associating two locations for each data point. We also disagree with Vermont DPS that we should use a single set of geographic coordinates at the start of each on-the-ground sequence, but we do agree with its alternative recommendation and will capture the timestamp and duration of each test component, as well as the geographic coordinates measured at the start and end of each test component with typical GPS Standard Positioning Service accuracy or better. Having start and end coordinates for each test will facilitate our verification of stationary maps versus mobile maps because it will enable us to capture the precise locations of drive tests.

44. We decline Verizon's request to adopt additional device- and plan-specific requirements. We recognize that some devices have limitations (
e.g.,
an older device may not connect to all spectrum bands), but find that restricting the types of devices that can be used to conduct speed tests would make the challenge process less user-friendly and less accessible to consumers and non-consumers alike. At the same time, a challenger must disclose the manufacturer and model of its device so that providers will have this information when rebutting challenges and can seek to invalidate tests from devices that are not compatible with a specific network or band. We will also allow mobile service providers to respond to a challenge with infrastructure information in situations where a mobile device used in the testing accessed the network over a data plan that could result in slower service. Finally, the methodology we adopt for aggregating speed tests and requiring challenges to meet the thresholds described below will ensure that challenges are temporally and geographically diverse and therefore reflect a robust and representative sample of user experience, regardless of device type or subscriber plan.

45.
Challenges to Larger, Lower-Resolution Hexagons.
We adopt our proposal for a “parent” or “grandparent” hexagon (
i.e.,
a hexagon at resolution 7 or 6) to be considered challenged if at least four of its child hexagons are challenged. CCA supports this proposal, while T-Mobile and Verizon argue that it could allow for challenges to very large areas even though significant portions of them have not been tested. We disagree with T-Mobile and Verizon and find that this approach will allow for the effective challenge of larger areas where an abundance of geographically diverse tests indicate a pervasive problem. Under it, a resolution 7 or 6 “parent” hexagon will be considered challenged only if more than half (
i.e.,
at least four of seven) of its “child” hexagons are challenged. The threshold can therefore be met without testing each resolution 8 hexagon, including ones that may be practically inaccessible. But each “child” hexagon must still meet the geographic threshold described below, which means that any challenges to larger “parent” hexagons will reflect that negative tests are persistent throughout the geographic area. While we decline to set the minimum size of a cognizable challenge at either resolution 7 or resolution 6 hexagons as requested by RWA, we believe that the approach we adopt herein will allow for challenges covering a significant portion of otherwise inaccessible resolution 8 hexagons. So long as challengers submit tests meeting the thresholds in at least four of the seven resolution 8 hexagons for a “parent” resolution 7 hexagon, the remaining hexagons would be effectively covered by the challenge to the “parent,” even if these resolution 8 hexagons are inaccessible. We conclude that this strikes an appropriate balance between reducing the burden on challengers while ensuring that robust evidence of a problem exists before requiring a provider to respond.

46.
Required Thresholds.
A resolution 8 hexagon will, as proposed, be challenged when tests submitted within the hexagon meet three thresholds: Geographic, temporal, and testing. We adopt the proposed geographic threshold, modified to account for our approach to evaluate each test component (
i.e.,
download and upload) separately. If the tests for a given technology in a resolution 8 hexagon meet all three thresholds we will consider that map's coverage to be challenged in that area. To satisfy the geographic threshold for a challenge, in general, at least four child hexagons (
i.e.,
“point-hexes”) within the resolution 8 hexagon must contain two of the same test components (download or upload), one of which is a negative test, in each point-hex. The threshold must be met for one component entirely, meaning that a challenge may contain either two upload components per point-hex, one of which is negative, or two download components per point-hex, one of which is negative. Requiring at least four out of seven point-hexes to include two of the same test components and at least one negative test will ensure that more than half of the point-hexes within a resolution 8 hexagon show inadequate coverage. Requiring at least one negative test in multiple locations within the geographic area of a resolution 8 hexagon will demonstrate that negative tests are persistent throughout the hexagon.

47. Consistent with the Commission's direction to consider (among other factors) “whether the tests were conducted in urban or rural areas” when setting the methodology for aggregating speed test results, we will adjust the geographic thresholds to allow challenges that account for differences in areas. Specifically, we adopt a different geographic threshold depending on the road density of each resolution 8 hexagon. We will relax the geographic threshold to require tests in fewer than four point-hexes when fewer than four of the point-hexes of a resolution 8 hexagon are “accessible.” We define an “accessible” point-hex as one in which the provider reports coverage for at least 50% of the area of the point-hex in its reported coverage data and through which at least one road traverses. Using the most recent U.S. Census Bureau roadway data, a point-hex would contain a road if it overlaps any primary, secondary, or local road, which are defined as Master Address File/Topologically Integrated Geocoding and Referencing (MAF/

TIGER) Feature Class Codes S1100, S1200, or S1400, respectively. In order to account for road width, we will apply a small buffer around the U.S. Census Bureau road line data. No entities commented on this definition. We choose 50% of the area of the point-hex to be within the provider's reported coverage because we want challengers to have a high likelihood of being within the coverage map when they test. We note that challengers can still test within a point-hex that is not “accessible” so long as the test falls within the provider's reported coverage. We settle on this definition of “accessible” because without a road it becomes significantly more difficult for parties to run speed tests in a point-hex. We find that the existence of at least one road gives parties a way to access a hexagon and run speed tests. We anticipate that this approach will make it easier for challengers to establish a challenge in less densely populated areas because challengers will be permitted to show less geographic diversity among tests if there are fewer accessible point-hexes in a resolution 8 hexagon.

48. We decline to adopt Vermont DPS's proposal to eliminate the requirement that four of the seven point-hexes within a resolution 8 hexagon meet the geographic threshold. Requiring a challenge to meet the geographic threshold in four of seven point-hexes ensures geographic diversity of tests and will help identify potential coverage gaps over a sufficiently wide area. Vermont DPS does not propose any alternative geographic threshold, and the record supports our conclusion that the geographic threshold is necessary to minimize the chance of anomalous results. We also reject RWA's proposal to reduce the geographic threshold for inaccessible resolution 7 hexagons or allow for a resolution 7 hexagon with low road density to automatically trigger a challenge. We believe the two proposals we adopt—(1) to reduce the geographic threshold for resolution 8 hexagons with low road density, and (2) to allow a “parent” or “grandparent” hexagon (
i.e.,
a hexagon at resolution 7 or 6) to be challenged if at least four of its child hexagons are considered challenged—adequately address RWA's concerns. For example, a resolution 7 hexagon that does not contain any roads is comprised of seven resolution 8 hexagons that also do not contain roads. A challenger therefore would not need to meet the geographic threshold in any of the resolution 8 hexagons if none of the point-hexes contain roads. Moreover, if a challenger runs tests meeting the temporal and testing thresholds in four resolution 8 hexagons and such tests show inadequate coverage sufficient to create a challenge, then the entire resolution 7 hexagon will be considered challenged. Thus, while our proposal does require challengers to meet the temporal and testing thresholds in a resolution 8 hexagon that has no accessible point-hexes, the tests do not need to be geographically diverse within each resolution 8 hexagon. We believe such a trade-off is reasonable to challenge a large geographic area.

49. We also adopt a modified version of our proposed temporal threshold. To meet the temporal threshold under the approach we adopt, each resolution 8 hexagon cell must include a set of two negative components of the same type (upload or download) with a time-of-day at least four hours different from two other negative components of the same type as the first set, regardless of the date of the tests. In other words, if the negative tests within the hexagon were ordered chronologically, regardless of the day of the tests, the difference in time between the first two tests and the last two tests must be at least four hours. The temporal threshold is evaluated across all tests within the resolution 8 hexagon and need not be met for each point-hex within the hexagon. That is, the earliest two negative tests and the latest two negative tests can be recorded in different point-hexes and still meet the temporal threshold so long as the difference in time between the two pairs of tests is at least four hours. Accordingly, because the geographic threshold for a fully-accessible resolution 8 hexagon requires at least eight negative tests (
i.e.,
two each in four of the hexagon's point-hexes) whereas the temporal threshold could be met using only four of those tests (located in any of the point-hexes), the temporal threshold would not necessarily require the challenger(s) to conduct additional testing. This threshold is different from that which we proposed in the
BDC Mobile Technical Requirements Proposed Rules
in that we now require two sets of negative tests to be temporally diverse, rather than one negative test being temporally diverse from one other test. T-Mobile supports the adoption of the temporal threshold proposed in the
BDC Mobile Technical Requirements Proposed Rules,
and we believe our modified approach is consistent with the concepts for which T-Mobile expresses support. Verizon and AT&T generally support a temporal threshold, and agree with our determination that temporal diversity is important, but we decline to adopt their proposal to categorize tests into specific four-hour ranges. We disagree that categorizing tests into specific time ranges would ensure temporal diversity. For example, Verizon and AT&T's proposal could allow a challenger to satisfy the temporal threshold with tests that have been conducted within a very short timeframe. However, in light of Verizon's concerns with our initial proposal, we find that multiple tests separated by four hours, rather than one at each end of a minimum of a four hour period, are needed to show temporal diversity, and thus modify our approach to ensure temporal diversity across several tests.

50. We are also unpersuaded by Vermont DPS's argument that we should not adopt the temporal threshold because it would require a challenger to drive test a road twice, and by CPUC's argument that the temporal threshold would significantly increase costs on challengers. We believe that the effort required to achieve the temporal threshold is outweighed by the need to collect a representative sample of a mobile service provider's coverage, particularly since our decision to combine challenge data from consumers, governments, and other entities in a given area will help minimize burdens on challengers and limit the number of drive tests any one challenger will need to conduct. We conclude that our approach is a reasonable solution that will ensure challengers demonstrate persistent inadequate coverage while accounting for the temporal variability of mobile networks, such as variability due to cell loading.

51. Finally, we adopt a modified version of the proposed testing threshold to require that there must be at least five negative test components of the same type (upload or download) within the resolution 8 hexagon when 20 or fewer total challenge test components of that type have been submitted. Consistent with the approach originally proposed, when challengers have submitted more than 20 test components of the same type in a hexagon, we will require that a certain minimum percentage of the total number of test components of the same type in that hexagon be negative, ranging from at least 24% negative when challengers have submitted between 21 and 29 total tests, to at least 16% negative when challengers have submitted 100 or more tests. Once the percentage of negative test components of the same type submitted meets the

minimum negative percentage required (for example, for a sample of fewer than 21 tests, once there are at least five negative tests submitted), we will not require additional tests so long as both the geographic and temporal thresholds for a resolution 8 hexagon have been met. The failure rates we adopt were chosen to demonstrate that coverage does not reach a 90% probability threshold. We find that this 90% threshold is reasonable to use because most speed tests will be taken within the provider's cell (rather than solely at the edge of the cell) where the cell area probability should be greater than the modeled cell edge probability of 90%, and to simplify the process, we will use the 90% threshold for tests conducted anywhere in the cell. To avoid the risk that the testing threshold would be skewed by a disproportionate number of tests occurring in one location within a resolution 8 hexagon, however, we adopt a modified approach such that if the number of test components of the same type in a single point-hex represent more than 50% of the total test components in the resolution 8 hexagon (where there are four or more accessible point-hexes in the hexagon), the test components in that point-hex will count only toward meeting 50% of the testing threshold. In a resolution 8 hexagon where there are only three accessible point-hexes, if the number of test components in one point-hex represent more than 75% of the total test components in the hexagon where the geographic threshold is otherwise satisfied, the test components in that point-hex will count only toward 75% of the testing threshold. If fewer than three point-hexes are accessible, we will not apply a maximum percentage of total test components for a single point-hex as the risk that testing would be skewed by a disproportionate number of tests occurring in a single location is reduced. We believe that these changes mitigate the potential bias resulting from a disproportionate number of tests occurring in one point-hex, and that this revised testing threshold will result in greater variety of tests within each resolution 8 hexagon.

52. Verizon, CTIA, and T-Mobile generally support the adoption of a testing threshold. Verizon supports our evaluating challenges based on the percentage of tests in a cell that are below the relevant speed threshold, but expresses concern that the Commission's geographic threshold “would allow cognizable challenges even if substantially all of the negative tests are in a single point-hex.” The modified approach we adopt mitigates the potential problems Verizon raises because the Commission would adjust the testing threshold when a disproportionate number of tests occur in the same point-hex. T-Mobile contends that staff should adjudicate challenges based on a threshold number and percentage of “negative” tests, with a minimum of five tests for each resolution 10 hex cell and at least 50% of those negative. We decline to adopt T-Mobile's alternative proposal because, as discussed above, we believe resolution 10 hexagons are too small for the challenge process. We also find that T-Mobile's proposal to require that 50% of tests be negative, regardless of the number of tests run, would place a high burden on challengers, and could diminish legitimate indications that coverage is unavailable in particular areas. In contrast, the thresholds for the percentage of negative tests we adopt are based on the statistical significance necessary to demonstrate lack of coverage. We also decline to adopt Vermont DPS's proposal to allow a single test, or maximum of two tests to be used to show inadequate coverage at multiple locations within a resolution 8 hexagon. Vermont DPS's argument that the geographic and testing thresholds effectively prevent drive testing assumes that a challenger should be able to run all of the tests necessary to meet each threshold on a single drive through a resolution 8 hexagon, but if challengers find that they are having to drive at a slow pace to run an in-vehicle test in a resolution 9 hexagon, they may periodically stop to run tests in a stationary manner before moving on to the next resolution 8 hexagon. We anticipate that government and other third-party testers can use software that overlays the H3 indexing system and/or providers published maps on a drive test map and may therefore know whether they are keeping within a hex or moving into another one while doing a test. We note, however, that this may not be necessary since we will be combining challenges from consumers, governments, and other entities in a given area which would lessen the number of drive tests any one challenger will need to conduct. For this same reason, we disagree with the CPUC that the testing threshold will be extremely expensive and require complicated coordination of efforts. As discussed, we will aggregate challenges from multiple sources and no one entity will be required to conduct all tests needed to challenge a particular geographic area.

53.
User-Friendly Challenge Process.
AT&T concurs with our assessment that the challenge process we proposed is reasonable and user-friendly and supports the overall framework, including the use of the H3 geospatial indexing system. In addition, CTIA, T-Mobile, and AT&T agree that the proposal to combine test data from consumers, governments, and other entities is user-friendly and reduces burdens on challengers, who will not be required to collect and submit every drive test needed to sustain a challenge on their own. Although Public Knowledge/New America raise concerns about whether the challenge process is sufficiently user-friendly, they share our belief that the challenge process should be as streamlined and burden-free as possible for consumers and other entities; we note that our implementation of the consumer challenge process is consistent with the
Third Order'
s determination that challengers will collect and submit all speed test data needed to support a challenge, including the new speed test metrics and parameters we adopt, through the FCC Speed Test app or another app approved by OET to collect and submit challenge data to the Commission.

54. We disagree with commenters that argue that our challenge process is not “user-friendly.” RWA argues that the testing process is not “user-friendly” because consumers can test only the networks their handsets are authorized to use. It recommends requiring providers to allow tests by other networks' subscribers. The Commission has already determined that consumer challengers must submit certain identifying information, including that they are a subscriber or authorized user of the provider being challenged, to deter frivolous filings, and the Bureau and Offices were not delegated authority to change this requirement. Similarly, Vermont DPS recommends requiring providers to temporarily provide approved devices with post-paid service at no or reduced cost to governmental entities wishing to engage in a challenge. We decline to adopt Vermont DPS's request because we lack the authority to subsidize government challenges and believe it would be too burdensome to require providers to establish and bear the costs of such programs. Enablers argues (and Public Knowledge/New America agree) that “ `testing parameters that amount to an exceedingly high burden of proof for consumers and other parties' run `contrary to the Broadband DATA Act and [the Commission's] own policy goals.' ” Public Knowledge/New America accordingly encourage the Bureau and Offices to consider

“allow[ing] the option to use other trusted sources to challenge providers' claims.” The Precision Ag Connectivity & Accuracy Stakeholder Alliance (PAgCASA) similarly claims that the proposed challenge process “delineates a series of technical and non-technical steps [m]obile customers must initiate and successfully navigate when conducting their [c]hallenge process that . . . falls well short of being easy to use from a customer's perspective.” These commenters also raise many issues that were already decided in the
Third Order
(
e.g.,
subscriber certifications and testing methodology and metrics) and are not delegated to the Bureau and Offices, or urge the Bureau and Offices to ignore the instructions given by the Commission, and would have been more appropriately filed as a petition for reconsideration of the
Third Order.
We reject the arguments of these commenters as untimely because they should have been filed as petitions for reconsideration to the extent that they raise issues already decided by the full Commission. Under Section 405(a) of the Communications Act of 1934, as amended, any party in a proceeding may file a petition for reconsideration within thirty days of public notice of the decision. These commenters raise issues that were decided by the Commission in the
Third Order,
which was published in the
Federal Register
on April 7, 2021. This publication date means that deadline for filing a petition for reconsideration of the
Third Order
was May 7, 2021. Because these commenters did not file their comments until September 2021, the Bureau and Offices find that the arguments are untimely and would have been more appropriately filed as petitions for reconsideration.

55. In conclusion, while the challenge processes and methodologies we adopt are by necessity detailed and technical, so as to assure that accurate and rigorous measurements are supplied to challenge providers' claimed broadband coverage, the Commission and Bureau and Offices have minimized the burdens placed on challengers by providing a user-friendly means for challengers to run speed tests using their mobile devices and submit all data via either the FCC Speed Test app or another OET-approved third-party app. As discussed, the Bureau and Offices were instructed to implement a number of complex and complicated tasks, among them, developing thresholds for determining when a cognizable challenge has been met, a procedure for resolving challenges, and adopting additional testing requirements if necessary. These obligations were delegated by the Commission within the context of the Broadband DATA Act, which requires the Commission to consider user-friendly challenge submission formats, reducing the time and expense burdens on consumers submitting challenges and providers responding to them, while at the same time considering lessons learned from the challenge process established under Mobility Fund Phase II, and the costs to consumers and providers resulting from a misallocation of funds because of a reliance on outdated and inaccurate maps. Indeed, financial assistance for underserved areas may, in the future, be based on updated Commission maps. Therefore, we find that the processes we adopt strike an appropriate balance, within the authority delegated to us by the Commission, to ensure the challenge process is easy to use and accessible for consumers and government and other entities and also results in high-quality challenges that will accurately correct any errors associated with providers' reported coverage maps.

2. Challenge Responses

56.
Notification of Challenges.
We adopt the
BDC Mobile Technical Requirements Proposed Rules'
proposed procedures for notifying service providers of cognizable challenges filed against them and for notifying challengers and providers of results of challenges. The
BDC Mobile Technical Requirements Proposed Rules
proposed that challenged mobile service providers would be notified via the online portal at the end of each calendar month of the hexagons that are subject to cognizable challenges. CTIA and T-Mobile express support for our proposal. We find this approach will help create a manageable process for providers by providing them with a standard set of deadlines rather than an erratic and potentially unpredictable set of innumerable deadlines for rebuttals that begin as soon as any given discrete area becomes challenged. We also adopt our proposal for mobile service providers and challengers to be notified monthly of the status of challenged areas, and parties will be able to see a map of the challenged area, and a notification about whether or not a challenge has been successfully rebutted, whether a challenge was successful, and if a challenged area was restored based on insufficient evidence to sustain a challenge. In the
Third Order,
the Commission directed that challenge and crowdsource data other than the location that is the subject of the challenge, the name of the provider, and details concerning the basis for the challenge must be kept private to protect challengers' privacy interests. Accordingly, before a service provider receives access to crowdsourced or challenge data, it will be required, within the BDC system, to acknowledge that it will use personally identifiable information that it receives for the sole purpose of responding to the challenge and that it will protect and keep private all such personally identifiable information. Such personally identifiable information may include challenger contact information, device information, and network information, as well as other personally identifiable information included in addition to evidence that a challenger submits.

57.
Timeframe for Responding to Challenges.
In the
Third Order,
the Commission determined that providers must either submit a rebuttal to a challenge or concede a challenge within 60 days of being notified of the challenge. Consistent with the
Third Order,
if the challenged provider concedes or fails to submit data sufficient to overturn the challenge within 60 days of notification, it must revise its coverage maps to reflect the lack of coverage in the successfully challenged areas.

58. In comments on the
BDC Mobile Technical Requirements Proposed Rules,
CCA argues that the Bureau and Offices should allow providers to seek a waiver of the 60-day deadline if the provider needs additional time to submit on-the-ground data due to unforeseen events or weather. Verizon contends that providers should be able to choose to seek either: (1) A waiver of rules that limit the permitted uses of infrastructure data or transmitter monitoring software in lieu of speed tests; or (2) a waiver of the 60-day deadline if the provider will rebut with speed test data. The Commission adopted the requirement that providers submit a rebuttal or concede a challenge in the
Third Order
based on its determination that permitting 60 days to respond to a challenge would make the challenge process more manageable for providers, while also providing for speedy resolution of challenges consistent with the requirements of the Broadband DATA Act. The Bureau and Offices do not have authority to change the required timeframe for provider responses. To the extent that a provider may wish to seek a waiver of the 60-day deadline for responding to a challenge in any individual case, it may do so under the Commission's generally applicable waiver rules.

59.
Future Challenges in Successfully Rebutted Areas.
We adopt our proposal

to make any areas where a provider has demonstrated sufficient coverage in a challenged area ineligible for subsequent challenge until the next biannual broadband availability data filing at least six months after the later of either the end of the 60-day response period or the resolution of the challenge. This ineligibility applies only with respect to the particular network technology and modeled environment for which the provider has demonstrated sufficient coverage. We deny Verizon and AT&T's request that the Bureau and Offices make successfully rebutted areas exempt from future challenges for a period of three years. We find that preventing future subsequent challenges for a period as long as three years could result in less accurate maps due to changes over time in technology and coverage. We find instead that limiting subsequent challenges for at least six months after the resolution of the challenge strikes an appropriate balance between avoiding a requirement that providers repeatedly confirm the same areas while ensuring that challengers have the opportunity to submit data regarding changed conditions. Although commenters assert that it is unlikely that coverage will be reduced in an area that was subject to challenge, an area that is subject to repeated cognizable challenges may highlight that significant technical issues continue to affect the availability of broadband service in that area. Permitting a subsequent challenge in these areas will help ensure that the Commission receives the most accurate and up-to-date coverage data reflecting consumers' on-the-ground experience. In any area in which a provider does not overturn the challenge but which is otherwise no longer considered challenged (
e.g.,
where, as a result of data submitted by the provider there is no longer sufficient evidence to sustain the challenge to that area but the provider's data fall short of confirming coverage in the area), the coverage area will be restored to its pre-challenge status and will be eligible for future challenges against it.

a. Rebutting Challenges With On-the-Ground Data

60. We adopt our proposal from the
BDC Mobile Technical Requirements Proposed Rules
that, when a challenged mobile service provider submits on-the-ground speed test data to rebut a challenge, the provider will be required to meet analogous thresholds to those required of challengers, adjusted to reflect the burden on providers to demonstrate that sufficient coverage exists at least 90% of the time in the challenged hexagon(s). Consistent with our proposal, the on-the-ground test data that providers submit must meet the same three thresholds required of challenger tests for both the upload and download components: (1) A geographic threshold; (2) a temporal threshold; and (3) a testing threshold, albeit with different values (
i.e.,
the number of tests and percentages) for test data for each threshold.

61. For the geographic threshold, the provider will need to meet the same geographic threshold required of challengers, but with positive test components rather than negative test components. At least four point-hexes of a resolution 8 hexagon must include two download test components taken within them, at least one of which must be positive, and at least four point-hexes of a resolution 8 hexagon must include two upload test components taken within them, at least one of which must be positive to demonstrate that adequate coverage occurs at multiple locations within the resolution 8 hexagon. We adopt a modified version of our proposed temporal threshold. To meet the temporal threshold under the approach we adopt, each resolution 8 hexagon will need to include a set of five positive download components with a time-of-day difference of at least four hours from another set of five positive download components, regardless of the date of the test and a set of five positive upload components with a time-of-day difference of at least four hours from another set of five positive upload components, regardless of the date of the test. We modify the threshold proposed in the
BDC Mobile Technical Requirements Proposed Rules
because we find that requiring more tests to be separated in time will help ensure that there is more consistent temporal diversity across several tests. For the testing threshold, we adopt our proposal that challenged providers must demonstrate statistically significant evidence that coverage is adequate to overturn a challenge using on-the-ground speed tests, based on the same statistical significance analysis used for determining challenges for both upload and download components. Specifically, in order for the testing threshold for a resolution 8 hexagon to be met, we require that at least 17 positive test components of the same type have been taken in the hexagon when the provider has submitted 20 or fewer test components of that type. When the provider has submitted more than 20 test components of the same type, we require that a certain minimum percentage of the total number of test components of that type in the hexagon must be positive, ranging from at least 82% positive, when providers have submitted between 21 and 34 total test components of the same type, to at least 88% positive, when providers have submitted 100 or more test components of the same type. The positive test rates we adopt were chosen to demonstrate that coverage does reach a 90% probability threshold, as opposed to the requirement that challengers demonstrate coverage does not reach a 90% probability threshold. Additionally, in line with the modification we adopt for challengers, if more than 50% of the test components of the same type are within a single point-hex where four or more point-hexes in the resolution 8 hexagon are accessible, the test components in that point-hex will be down-weighted to only account for 50% of the total test components when evaluating the testing threshold. If more than 75% of the tests are within one point-hex where there are three accessible hexes in the resolution 8 hexagon, the tests in that point-hex will be reduced to only account for 75% of the total tests when evaluating the testing threshold. By limiting the percentage of test components within any one point-hex that may contribute to a challenge response, this requirement will help ensure that there is sufficient diversity in the test data that a challenged provider submits. A provider may also demonstrate sufficient coverage in a resolution 8 hexagon that was not challenged in order to rebut a challenge to a lower-resolution hexagon containing the non-challenged resolution 8 hexagon (
i.e.,
the “parent” resolution 7 hexagon or “grandparent” resolution 6 hexagon). As discussed more fully in Section 3.2.4 of Appendix A—Technical Appendix (available at
https://www.fcc.gov/document/fcc-releases-bdc-mobile-technical-requirements-order
), for challenged hexagons at resolution 7 or 6, if the provider submits response data sufficient to demonstrate coverage in the hexagon's child hexagons such that fewer than four child hexagons would still be challenged, then the resolution 7 or 6 hexagon would no longer be challenged even if suffici

[Text truncated at 120,000 characters. The full text is on the page linked above.]

---

Source: Frix Law Library, https://www.frixlaw.com/law-library/documents/fr%3A2022-06826. Public record. Not legal advice.