Oral argument — Pavement Coatings Technology v. United States Geological Surv

Case number 20-5035, Pavement Coatings Technology Council, Abellants v. United States Geological Survey. Mr. Ebner for the Abellant, Mr. Koppel for the Apellate. Good afternoon. May it please the court. I'm Lawrence Ebner and I represent the Pavement Coatings Technology Council. USGS is asking you to expand FOIA Exemption 5 in a way that would insulate all federal government scientists from external criticism of their published research. According to USGS, a government scientist can hide behind the deliberative process privilege simply by labeling essential study-related data as exploratory analysis. If USGS is correct that the deliberative process privilege applies to scientists thinking, virtually all federal government research is going to be covered by an exemption that is supposed to be construed narrowly. We believe that concealing scientific data so that a study cannot be discredited is not the purpose of the deliberative process privilege. For example, I just want to be clear, you have the data itself, correct? The only data that we have are the raw laboratory data. That's exactly what I'm referring to. You have the raw data. What you do not have is two people who have analyzed this data. What we do not have and what we have is the four out of 200 options they have here. That's correct. We do have, and government council will tell you, USGS has produced tens of thousands of pages of raw laboratory data. Those data are primarily late sediment chemical analyses and measurements. They are useless to us without the computer modeling inputs and outputs that we seek. In this case, many state and local governments have relied on the urban lake study publication to ban the use of refined tar sealant. The published study does disclose the computer modeling input and output data only from what the study authors describe as their four best modeling scenarios. Those are computer... All of the raw data, and you have these four models. Why on earth couldn't your own scientists with all the raw, they know the question, they have all the raw data, they know what models were used, they could run it, and then they could tweak and redesign the models to test just like these that's what scientists do. Why can't you do your own model runs with the same data? Judge Millett, I think that one of the study authors, Dr. Van Meter answers your question in his declaration where he explains that there were so many possible combinations of inputs for a given model that the possible outcomes are virtually limitless. And in fact... They give you the other model runs. So I'm trying to understand why, if your concern seems to be the FOIA interest here is knowing what other models were tried, what changes, what tweaks or adjustments were made to these four models that they didn't use, they didn't treat as their four best models. And any scientist can come up with their own tweaking to models, try their own changes, they can take all those variables and make all those changes. So you don't need, you may want it, but you don't have a need for that information to understand what the government did. You can test the government's theory because you have the data. With respect, your honor, all we can do with the data that has been provided is to try to replicate what the study authors describe as their four best modeling scenarios. Modeling runs... Your scientists aren't capable of designing their own model runs? There were so many possible combinations that what we want to do, your honor, is take a look at the other 200 modeling runs that the study authors chose not to use in their published study and not to disclose. They're saying in their published study, trust us, those other 200 modeling runs are going to confirm the four modeling runs that are discussed in the published study. In order for us to replicate the other 200 modeling runs, we need not just the raw data, but we need to have the input and output data so that we can run those other 200 models and see, for example, whether they report... You have the data that they put in and you have the raw data that came out. We do not, your honor. I thought you had all the raw data. Are you saying there's raw data that wasn't disclosed that they input into their model runs? There's a distinction, your honor, between the raw data and the computer modeling input and output data that we seek. USGS, on page 12 of its brief, explains that modeling data are critical, that's their word, critical to scientists' ability to analyze the raw data. You have the raw data and you have their four runs, right? What we want to do is duplicate... You want to do the 186, whatever. Approximately 200 modeling runs that they ran, but won't tell us what those modeling runs are. In order for us to test the study's conclusions based on their four best modeling scenarios, we need to know what all the parameters were, not just the raw data, but the parameters and the combinations of parameters, which Dr. Brand-Meter says are virtually limitless, in order to see how those other two modeling runs come out and see whether they really do support what's in the published study or whether they undermine it. And I'd like to point out... In other words, you want to be able to see whether in their deliberations they deliberated and reached the best conclusion or not, right? Well, I wouldn't call it deliberations, but we would like to know whether their study, for example, suffers from confirmation bias. Yeah, I mean, my point, sir, is that to say that you aren't really seeking to understand the deliberative process of these scientists seems to be like saying that there's no gambling going on in Casablanca. I mean, the whole purpose of this request is that you want to undermine their deliberative process because you think that it has been infected by bias or incompetence or whatever, right? Well, yes, Judge Wilkins, but the scientific deliberative process, which is really the scientific method, is not the same as the deliberative process that's contemplated by FOIA Exemption 5. Okay, so then let's get down to brass tacks. You do want to invade their deliberative process, but you believe that scientific deliberative process is not covered because that's not policy deliberative process, right? It's not only not policy, but it falls within the distinction that the Supreme Court and the Supreme Court have established between purely factual information and drafts and so forth that go into the formation of policies and recommendations. We are not seeking, for example, any notes or memos or emails that these scientists may have generated that analyze or evaluate or discuss the modeling runs. We're not asking for drafts of the proposed study. We're not asking for internal USGS comments on the proposed study. What we are asking for is pure scientific data, which are contained on spreadsheets, charts, and graphs, and I might point out that USGS's own internal manual defines data as non-interpretative, and I'd like to refer the court to the American Radio Relay case because in that case, the court held that a federal agency cannot invoke FOIA Exemption 5 in order to hide parts of a study that are inextricably bound to the study as a whole that may contain contrary evidence or may help to eliminate the strengths and weaknesses of the study. So, you have all the data, you have the final study with the four runs, and you want the other 196, which I gather from your statement, you're saying the agency still has. They still have. They refuse to produce them. There's nothing inherently deliberative about them. If these kinds of data, input and output data, were inherently deliberative, they wouldn't have disclosed the inputs and outputs for what they call their four best modeling runs. So, and to use the word that USGS uses in its brief, what we seek, the input and output data for the other 200 modeling runs are critical to the ability to analyze the raw data that they produced. They basically dumped tens of thousands of pages of raw data on us and said, we should go, you know, figure out by an infinite number of permutations what their other 200 modeling runs were, because they won't tell us. And we don't think that's what's contemplated by FOIA Exemption 5. Let's be clear about something. You already have the data. It's just not labeled input and output? Or are you saying there's raw data you don't yet have? Well, I'm using the term raw data like the study authors do, which is the lake sediment measurements, which is one of the inputs into the computer model. You have all the data. It just doesn't come with a label input and output. No, Your Honor, we do not have the input and output data. Another term for that is the modeling runs. Okay, that's different than let's let's just call it model runs versus data. You've got the raw data. Let's not call model runs data, because I think that's a question in its own right. So let's just call it model runs instead of saying you don't have certain data. Model runs represent the inputs and the outputs. There were more than 200 of them, and USGS has refused to produce them. And we cannot test the Urban Lake Studies conclusions without those other 200 model runs. We cannot do it just with tens of thousands of pages of raw laboratory data. We cannot do any... Articles were peer reviewed before publication in the journal. Does the peer review process require disclosure of all 200 model runs? We don't know whether those 200 model runs were disclosed, but in our brief we cite, for example, nature publications policy, which does require scientists to make all the data available. And this is where we keep... The language here is troubling to me, because they've made all the data public. And then the article went through a scientific peer review process. In fact, it's been... They have a number of publications, all of which have gone... My assumption is, so I might have been wrong, scientific peer review process. And there's no argument by you that they disclosed... They lost this privilege because they disclosed all these model runs as part of the scientific review process. Your Honor, there's nothing in the record indicating that the 200 model runs, which they refused to disclose to us, were disclosed to the peer reviewers in the two publications that ultimately published what they call their four best modeling scenario. You have FOIA requests that says all of your model runs that have been disclosed to members of the public already. Well, we originally filed a very broad FOIA request, which would have encompassed that. And I have no reason to believe that the withheld model runs, which consist of input and output data that they have not provided, were provided to the ask one question, Judge Rogers. But even if they had been provided to peer reviewers for the purpose of peer review, that wouldn't mean that they were public in the sense of they still... They would be outside the protection of the deliberative process privilege in FOIA, right? The peer reviewers would have been reviewing them on the same terms that someone reviews a draft article for publication with the requisite confidentiality protections that flow with that, right? Well, the peer reviewers substantive and would be covered by the privilege. We are not seeking that. We are just seeking the data. And my understanding is that when a scientist in academia or the private sector, for example, publishes a study, they must make all study-related data available, not just to peer reviewers, but to anyone upon request. No, but you have been making certain assumptions throughout your briefing here. Let's hear from counsel for APALE, and then we'll give you a couple of minutes. Good morning, your honors. Good morning. May it please the court. I'm Josh Kopp on behalf of the government. The preliminary model runs are deliberative because they reflect agency scientists evolving thinking about how to interpret and make sense of the raw data. The counsel, the pavement coatings technology council argues that the preliminary model runs are facts and that disclosure would not provide any insight into the deliberative process. That is wrong. As your honors have been discussing the council, sorry, the geological survey has already disclosed all of the facts, the raw data. Specifically, PCTC has received the chemical analyses of the lake sediment samples and the PAH fingerprints of the potential sources of contamination. Their own scientists could use that raw data to create any model runs that they wish. What the geological survey has withheld are selective compilations and analyses of that raw data that reflect the process by which agency scientists interpreted and analyzed the data. Were those model runs that they want disclosed to peer reviewers? Is that part of the scientific review process? Some of them were. Some of these preliminary model runs were encompassed in preliminary drafts of the study, which were sent to reviewers within the geological survey and at the journal where the geological survey sought to publish the study. Those reviewers gave feedback. The scientists went and refined their model. So some of them were disclosed to the reviewers. Were disclosed to, I don't care about inside the U.S. geological, talking about outside the geological survey, journal. These journals must select the people that they want to do these peer reviews. And some of those folks did get some of these model runs that they aren't getting. I believe that they did get some of them. Those peer reviewers are within the umbrella for the purpose of the deliberative process privilege. This court held. I think it's a very open question after Clayman. Pardon? I think that's a very open question under Clayman. I believe that in Formaldehyde Institute versus HHS, this court held that a letter from those peer reviewers. It was long before the Supreme Court decision in Clayman. That's true. Yes. The district court considered the consultant corollary PCTC made an argument on that basis in the district court, and they have chosen not to appeal that. They have not argued that any information was, that any privilege was waived by disclosure to the peer reviewers. Can I ask you what, the relevant agency decision at issue here to which this model run selection process applies, and you've described that as the decision to publish, and not just the up or down decision to publish, but what to publish. That's correct. Can you tell me who the agency decision maker was in making that comprehensive publication decision? Was it just these two scientists? Is there someone else within USGS that has to authorize, had to authorize them to publish in their official capacity with their official position noted in a private journal? Who's the decision maker here? Yes, I believe there are multiple levels of review, including the scientist supervisors. There is an approving official that takes a look at all publications, or multiple approving officials that take a look at all publications that the geological survey puts out. And were all these model runs disclosed to them when they made these decisions about their model runs? I don't believe that all of them were disclosed to those approving officials. The approving officials may have seen some in the preliminary drafts. They certainly knew that these preliminary model runs existed, and they were relying on the feedback that the scientist peers at the geological survey had given to the authors. The approving officials were relying on that feedback to ensure the publication was... It's not just to be a deliberative process. It's not just that somebody deliberates inside an agency. The point of the deliberation has to be to inform the ultimate decision makers decision. So what I'm trying to find is where there's anything in this record that explains how when these two scientists were running their models in their laboratory or office or wherever they did it, and making these decisions, they were doing that to inform the ultimate decision makers determination about whether to publish or not, as opposed to they were making it for their own scientific judgments. Right. Well, they were using these deliberations to ensure that the draft that they present to the approving official would be the best possible, and that the models that they are seeking to publish are sufficiently reliable, and they are ensuring that through this deliberative process with their peers at USGS, they are ensuring the quality of their research, and the approving official relies on that process in determining whether to approve. It relies upon them transmitting this information about how they chose the models that they did and the ones that they didn't. What I'm afraid of here is that this deliberative process might have been these scientists' deliberative process to inform themselves about running of the experiments. But I haven't seen a connection to how when they just say, when they reject model runs three, four, five, six, seven, eight, and nine, they did that to inform that person way down the line years later, decision whether to publish or not. And you have to make that connection. Their deliberative decisions have to been to inform that person months or years later about whether to publish or not. I don't believe that there is any case that has held that the ultimate decision maker has to have seen the material for it to be covered by the deliberative process privilege. Deliberative process is to allow the agency to inform the agency decision makers decision. That's what the deliberative process is. I mean, I worked in government and we would sit around and shoot the breeze debating issues all the time. They weren't to inform any ultimate decision made by the agency. They were deliberative between us, but they weren't informing an agency decision makers final agency action. And I'm having a lot of trouble getting from, and this may just be my ignorance of scientific process, getting from their deliberations about their scientific study as to those decisions were being made to inform the decision maker way down the road who decided whether to publish. Well, their deliberations were certainly made in order to ensure that what they presented to the ultimate decision maker would be reliable and would gain his or her approval. You know, if the scientists are considering a number of potential models. They don't share with that ultimate decision maker, 196 no's. Then how did they make that? How did those decisions inform the ultimate decision maker? Well, they can assure the ultimate decision maker that the scientists have done their due diligence and have looked at a number of options to ensure that they are presenting the most reliable draft for publication. They would have decision to tweak the model here, to not take that approach, to adjust the data here, to include this input and to exclude that input. That never gets passed down the line to the ultimate decision maker. There were certainly some preliminary model runs that could be discarded out of hand, or some of these preliminary model runs were not intended as actual reliable models, but were intended to test or break the model. So for instance, scientists would use a pristine mountain lake and give the model only urban sources of PAH. And the goal was to see if the model would work in returning essentially the result that there is no result. And so some of those models were certainly not presented, didn't need to be presented to the approving official. They were absolutely essential to the scientist's process of reaching the most reliable model results that they could then put in this draft publication, present to the approving official. You guys picked what the relevant agency decision here was. And you didn't pick the scientific determination by the USGS as to causal relationship between these sealants and these increased levels of this toxic chemical in the water. You have picked the decision to publish. And that's what I'm questioning is I don't see in the affidavits to record here anything that says when these decision makers were doing, these scientists were doing their study, that the model runs you're not disclosing were part of, were deliberations that were undertaken to inform the ultimate decision makers final agency action of issue. Yeah, our position is that the agency decision at issue is whether and in what form to publish. And that in what form certainly includes what results to publish. So which model runs, what the ultimate conclusions are of, what are the main contributors of PAH. All of these 196 model runs with the ultimate decision. Did the ultimate decision maker sort of see the whole package of everything they've done? I don't believe that he saw all of the preliminary model runs. I do believe that he saw the input that the authors received from their peers, some of whom had seen these preliminary model runs. I believe that the approving official took a look at the process that the agency scientists had been through, including their formulation of some of these model runs. But we don't have any of that in the record, right? That is correct. I don't believe that what the approving officials look at is in the record. But again, I don't think it is necessary because these agency officials are acting on behalf of the agency. And the deliberate process privileges, you know, covers the process of the agency, including the lower level scientists. You have the burden of establishing the privilege, right? The appellant doesn't have that burden. Plaintiffs don't have that burden. That's correct, but I don't think it's relevant what the approving officials saw or didn't see. What's relevant is that the agency officials, the agency scientists, as part of their decision-making process, were using these preliminary model runs to test out their hypotheses and to test out different approaches for analyzing the raw data. And what's in the record is that, and what you've said in your brief, is that this is basically kind of how you calibrate a model, right? This is standard. This is something that's done in any instance where you're trying to develop a model to replicate or to, you know, try to replicate and answer a question, right? Running preliminary model runs, I believe, is quite standard. If that's the case, then where is there evidence in the record that if we disclose these model runs, we order their disclosure, that it will discourage them from being done in the future or that it will somehow harm the quality of future agency decision-making? So which models the scientists run is an extremely subjective, you know, it is an exercise of subjective, quantitative, and qualitative judgment. And we can certainly, I believe that if the agency scientists knew that their preliminary model runs were going to be disclosed, they would certainly be less candid, you know, for fear of being ridiculed. That's not in any affidavit. That's not in any of the declarations. I believe that Dr. Van Meter and Dr. Van Moller, I believe that they both say that they would be discouraged from running some of these preliminary models I did not see that. I'd like for you to find that language because I looked for it and I didn't see it. Yes, I can take a quick look, Judge. I think that the best that they say, if you look at JA-27, Dr. Van Meter's is that it will give outsiders an opportunity to confuse the public and discredit our work. And so they say that they don't, you know, they think that's something bad that will happen, but they never says that in the future, then I'll do preliminary model runs differently or I'll be inclined not to do them. And there's similar language in the declaration from Dr. Moller, but I don't see anything that says that disclosure of this is going to kind of change the way that we do our business. I'm not sure if they explicitly say that it would change the way they do their business. They do say, as you mentioned, that it would confuse the public. And I believe, first of all, that is its own interest protected by the deliberative process privilege with public confusion. But I think that we can certainly imagine, if we took another context, for example, climate change, if agency officials knew that their climate change research, all their preliminary models would be disclosed to the public, agency officials might pressure lower level scientists to only run certain models that are consistent with what those agency officials want to ultimately publish. Or the scientists themselves may feel inhibited from running certain models if they are afraid that those models will show something other than what they think their ultimate conclusion will be. Those models are not done for publishing articles in a journal. Those are done for deciding what the final agency position or agency action will be in a regulation or a policy of some sort. The difference here is it seems to me you've picked an unusual agency action, and that is to publish an article in official capacity. And what I'm having trouble with is that you seem to be conflating the decision to come to a reliable scientific result through this trial runs, the scientists' own decision and concern about doing that with the decision of someone else who you said there are many layers of review, so I don't know how many layers this is, but apparently there's somebody at the end who signs off and says, yes, we will publish this in the name of the U.S. Geological Survey, or at least they can use their official titles. Does that mean, I guess I should clarify one thing, does that mean it's a United States Geological Survey position when scientists publish in a journal in their official capacity? Yes, I believe that's correct. We believe that's correct, but we don't even know that, right? So I'm not even sure if that's what's going on. But I will assume that for now, because they get to use their official titles. But that just seems to be an awful lot that's happened in between, and I'm just not convinced every time they're standing there a year or so before and they run a model run and they go, that's crazy. But it was part of the decision whether to tell someone way down the line, this person, whether she should authorize them to publish in their official capacity. And that's the connection I think we need for deliberative process. I think this is no different than those draft agency histories that were at issue in Dudman and Russell. In those cases, the court wasn't concerned whether the ultimate decision maker had viewed the preliminary draft. If there was a preliminary draft that got changed somewhere along the process, somewhere on its way up to the official in the Air Force that was going to ultimately approve whether these draft histories were made public, that was enough. You know, because it exposes the deliberative process of the primary author or really of anyone inside of the agency that's helping to formulate the history at issue. And so, you know, those cases also, of course, involve publications rather than policies. Maybe I just don't know how these folks work. When these scientists started this process, was their goal to write an article or was their goal to make a scientific determination about the relationship between this pavement coating and toxins in water? My guess is that it, I think that it was both. They expected to make a determination of the relationship and to publish those findings for the, you know, for the benefit of the scientific community. Because the geological survey doesn't make policy or... I don't have that record either. We don't have them saying what the point... Their deliberations, as they describe them, seem to be to get this science right, to figure out this question. That's what they're deliberating about. That's right. But also to publish it for the benefit of the scientific community and the benefit of the policy makers. Do the declarants here, the two scientists that did this study, do they state in their declarations, because I didn't see this either, that we were making this deliberate decision, deliberative decisions to inform the ultimate decision maker on whether or not to publish this in the name of the USGS? I'm sorry, can you repeat the question? In their declarations, they say when we were making our decisions about these 196 model runs that we rejected or the four that we thought were best and why we thought they were best. They were making that... Because we wanted to be able to inform the ultimate decision maker about whether to publish. They don't say exactly that. They say that these preliminary model runs were an essential part of them performing their ultimate conclusion, which was then approved by the approving official to be published. Pardon? The scientist's ultimate conclusion was a scientific relationship. That's right. Yeah. This is no different, I believe, than a professor workshopping a paper at a university. He receives feedback from his or her peers and then goes back and refines the paper. That's exactly what is going on here. The scientists... When they went to these outside journals, this wasn't a government journal that was published. It was an outside journal. Then did the government pick the peer reviewers or did the outside journal? The outside journal. Outside journal. And did the outside journal pick those peer reviewers for the purpose of telling the government what the best science is, whether these meet the scientific standard or for telling the journal whether these meet proper scientific standards to be published? I believe the intent of the... I can't speak to the intent of the journal, but I believe that it is to inform the journal whether the paper is worthy of publication, but also to help the study authors get the publication into shape. Because it's not a... The reviewers don't give a yes or no, up or down. They give feedback to the authors who then go and modify the paper, edit, run new models. They modify it because it's not getting published unless they clean something up. But I assume the point of it is to have comments so that the journal knows whether this is something that meets the journal standards for publication. I think that is one of the purposes. But again, I do believe that part of the... The journal is also... Doesn't select the reviewers only for its own benefit of giving a yes or no, but also to contribute to the scientific discussion and to aid the authors in improving their research. Unless there are further questions I have. You're describing this process, just so let me be clear. It's not two separate parts, but rather it's just one continuous effort until a decision is ultimately made to publish. I'm sorry, I'm not sure which two parts you're referring to. I'm just trying to understand. You have these two scientists doing all of this work and then they decide the best evidence they have, their view among the models are the four that they disclose. And then they discuss what they found. So now here, the appellant is seeking not simply the raw data, but it wants the worked up... It wants to understand the background to the workup of the document that was ultimately published. And it wants to understand that as I understand it, anything that went into that, that the parties, the scientists, the head of the survey didn't agree to publish is simply unavailable. You have the raw data, you have the published study with the four. And in that sense, there's a deliberation going on as far as the scientists are concerned. That's right. What PCTC seeks are the approaches or the approaches to or the interpretations of data that the scientists ultimately decided were not those that were the most reliable and worthy of publication. So using the raw data, PCTC scientists could run these exact same models. What they wouldn't know is whether that model is one that the USGS scientists considered before settling on their four final models that were worthy of publication. So what they seek is insight into the deliberative process. I mean, exactly. They have the raw data, they have the results. What they don't know is what approach the scientists use to interpret that data on the path from getting from raw data to ultimate publication. Unless there are further questions, we ask that your honors affirm the judgment of the district court. All right, counsel for appellant. Yes, thank you. Well, again, we're not seeking interpretations. We are only seeking the 200 modeling runs and we heard government counsel just explain that with the tens of thousands of pages of raw laboratory data that they have provided, we can try to somehow do our own modeling runs with all these limitless possible combinations of input variables and try to guess what their 200 modeling runs were. I'd like to refer the court to the declaration of USGS FOIA officer Brian May. He says in this declaration, first of all, that the withheld model runs were not used in the final decision. What page of the FOIA are you on, please? That would be in the appendix on page 49. He says that the withheld modeling data were not used in the final publication, were not used in the quote final decision. He also says on page 53 of the appendix that it was the USGS approving officials. Those are the people who ultimately approve publication. It is the USGS approving officials quote internal discussions and quote candid feedback to the study authors that quote comprised the USGS deliberative process. We're not asking for any documents reflecting those internal discussions. We're not asking for any documents reflecting the candid feedback that the approving officials gave to the study authors. We're not asking for any deliberative documents. We just want the modeling run, which we've described as the input and output data. And the other point I'd like to make. My question then is, I know you characterize it that way, but is that an accurate way to characterize it? It is. So the scientist takes the raw data and does something with it, right? Yes, I think the answer to your question, Judge Rogers, is set forth in the declaration of Dr. Van Meter, who's one of the two study authors. He explains in detail not only how they selected a particular computer model, but also how they formulated a modeling run. He explains that the modeling run includes the raw laboratory data, the sediment readings and the PAH profiles. There's more than a dozen different types of PAHs. Then he talks about the modeling parameters. And he explained, it's kind of like, you know, turning the knobs on a stereo hi-fi. There's all different ways to do it. And he says there are so many possible combinations of inputs and modeling parameters the possible outputs for a given model are virtually limitless. And the other point I'd like to make in rebuttal. Well, let's finish up on that. He's describing the intellectual element that goes in between the raw data and I'll call a draft and then it goes through some refining processes, but it's still the scientist's input. That's correct, Judge Rogers. And what you have just described is the scientific method, a method that Mr. Koppel said was standard running these computer modeling runs. And I think the question for the court is whether the court would like to take the unprecedented step of stretching Exemption 5 to encompass the entire scientific method. And if the court were to do that, in this case, it would set a precedent for federal government scientists in all agencies that employ federal government scientists and any federal government scientist just by tagging modeling data as exploratory analysis and saying, well, it was my scientific deliberations would be able to hide the ball, which is you're saying that we would be taking an unprecedented step, but I don't recall seeing you cite anything that says precedent-wise that says that the scientific method is outside of the deliberative process privilege. Well, I think one case we've cited, for example, is the American Radio Relay League case. I'm familiar with that case, as you know, and that's a different case. There was a study and the agency had to release all of it. Now, we're between the raw data and the ultimate study. American Radio is at the study. Now, you're asking for something that happened in between. And the question is whether or not that comes within Exemption 5 or not. No, our request refers to the Urban Lakes study, just like there was a study involved in the American Radio Relay case. And my understanding from reading that case is what the FCC was trying to do in that case was to hide, and that's the term, that's the word used in the decision, hide parts of the study. I'm asking you, is there something between the raw data and the published report? Yes. What is it? Modeling runs. But that's not a study, is what I'm trying to get at. It's an inherent, it's an intrinsic part of the study. The word study, the word study can be the scientific method or it could be the publication of the study. And what we have here, is a scientific process that the study authors conducted running more than 200 modeling runs. That culminated in a proposed publication, which according to their words, only disclosed the modeling runs for what they considered the four best modeling scenarios. Modeling scenarios that say basically that refined tar sealant is the primary source of PAHs in urban lakes. And they refuse to disclose the other models, as your honor has pointed out. And what Council for the Government is doing, respectively, is conflating a scientist's deliberative process, where he began his argument by referring to scientists thinking, that's also the term used in their brief, conflating that with the agency's deliberative process on whether to publish a study that is presented to them by agency scientists. And again, FOIA Officer May says that the modeling runs that were withheld were not used in that final agency decision. He says, what was the USGS deliberative process were the internal discussions among the USGS approving officials and the candid feedback they gave to the study authors? We don't know what that feedback was. We're not asking for that in this appeal. We just want the modeling runs, respectively. And one last point, if I may, very quickly. This is a very influential study. As we point out in our brief, it has, along with the HAUSDA study, which we haven't had time to discuss here, but it's discussed in our brief, have led many state and local governments to ban this product. And although USGS doesn't set policy, even a casual look at their refined tar sealant web page indicates that they rely on these studies to essentially recommend that state and local governments ban the product. So we have the agency promoting a published study and hiding the ball, the ball being what we need in order to test the study's conclusions. And I think... I want to push you just half a minute. The two scientists say we ran over a hundred models. We picked the best four. Those are the four that we think best support our conclusions. Yes. That's what they're saying. And we wonder whether what they consider their best scenarios are the scenarios that essentially condemn this product refined tar sealant. They're not worthy of any credence. Well, I'm not going to say that they're not worthy of credence, but it's... I'm sorry, Your Honor. They say these are the best and the argument is maybe they're not the best. Maybe these scientists are no good. Well, we're certainly not going to say that, Your Honor, but what they... I just want to understand what it is that we're after. That's all I... What we are after, Your Honor, is the ability to see how those other... how the outcomes of those other 200 modeling runs differ from the outcomes of these four scenarios. And so as Judge Millett asked you at the beginning, your own scientists could do all this and presumably that most scientists who would approach this question would approach it in similar ways. We can only do that, Your Honor, with the 200 modeling runs. No, that's one way you could do it. Pardon me, Your Honor? That's one way you could do it. I believe it's the only way. No, it's not the only way. You could do a thousand models. I mean, I realize we're talking about money here. All right. And setting up a whole chemistry lab. Yes, I mean, yes. I want to understand what it is we think that in a normal situation, the deliberative process would cover. You're saying all the work you do, and you may be right, all the work that is done in reaching your conclusion means that if you're on the government payroll, you've got to turn it over. And if you're if you work for a corporation or for a university, there'd be no question that you would have to make these 200 modeling runs available. But these, but USGS wants its scientists to have a privilege, if you will, that no other scientists have. And again, what do we do with of raw laboratory data? We can, yes, you're right, Your Honor. We could come up with some modeling runs, but we don't know whether they would be the 200 modeling runs that the study authors ran. So let's suppose that the agency that we were dealing with was an agency like the EPA. And what the scientists had done was developed a model that was used to ultimately promulgate a regulation by the EPA to regulate or restrict or ban in some way these pavement coatings. And you submitted a FOIA request. So there's kind of would the model runs be covered by the deliberative process privilege there? No, they would not. Because the model runs are simply factual data. What are the inputs? What are the outputs? And so it doesn't turn on whether the agency is a policy making agency or not. You're just saying that this is science. And so deliberative process of these scientists is not covered by the privilege. Or you're saying that the model runs themselves don't reveal the deliberative process. I'm trying to understand. I'm saying the former, Your Honor, that scientists thinking is not covered by the deliberative process privilege. And to use the words, again, we've cited a number of cases, but I think American Radio Relay is close. It says, quote, purely factual investigative matters, close quote, are not protected. The privilege is designed to protect processes by which agencies policies are formulated and recommended. Now, USGS doesn't establish policies, but it's done the next best thing. Well, well, I'm I, you know, we were talking about my EPA hypothetical. And and I mean, doesn't the doesn't your own briefing? I mean, part of your own briefing is, is that the data and is not helpful because we need to know kind of like which data they selected in in in in which models they ran. So disclosing that discloses their thinking. This isn't just data. It's disclosing choices that those scientists made, right? That is exactly correct. We submit that that type of scientific thinking is not the type of deliberation that is intended to be covered by a narrow privilege that goes to the formulation and recommendation of agency policies. And and the result of the rule or the interpretation that the government is advocating is to envelop the entire scientific method, all that scientific thinking at EPA and every other. What is your best case or cases for that proposition? American Radio Relay. I think that's that's one very good case, yes. National Home Builders is another one. All right, thank you. Thank you. We'll take the case under advisement.