State v. Adams

No. 456 May 29, 2025 661

IN THE COURT OF APPEALS OF THE

STATE OF OREGON

STATE OF OREGON,

Plaintiff-Respondent,

v.

ODELL TONY ADAMS,

Defendant-Appellant.

Multnomah County Circuit Court

18CR73436; A177071

Jerry B. Hodson, Judge.

Argued and submitted October 11, 2023.

Neil F. Byl, Deputy Public Defender, argued the cause

for appellant. Also on the briefs was Ernest G. Lannet,

Chief Defender, Criminal Appellate Section, Office of Public

Defense Services.

Peenesh Shah, Assistant Attorney General, argued

the cause for respondent. Also on the brief were Ellen F.

Rosenblum, Attorney General, and Benjamin Gutman,

Solicitor General.

Janis C. Puracal filed the brief amicus curiae for Forensic

Justice Project.

Before Aoyagi, Presiding Judge, Egan, Judge, and Joyce,

Judge.*

EGAN, J.

Reversed and remanded.

______________

* Egan, Judge vice Jacquot, Judge.

662 State v. Adams

Cite as 340 Or App 661 (2025) 663

EGAN, J.

Defendant appeals convictions for unlawful use of

a weapon with a firearm and second-degree criminal mis-

chief, raising two assignments of error. First, he argues

that the trial court erred by admitting as scientific evi-

dence the testimony and report of an expert who, using the

Association of Firearm and Toolmark Examiners method

(the AFTE method), “identified” cartridge cases found at the

scene of a shooting as having been fired from a gun found

in defendant’s home. Defendant contends that the state

failed to establish that the AFTE method is scientifically

valid and thus that evidence based on it is admissible under

State v. O’Key, 321 Or 285, 899 P2d 663 (1995), and State v.

Brown, 297 Or 404, 687 P2d 751 (1984). Second, he argues

that the trial court erred in denying his motion to contro-

vert and suppress evidence obtained pursuant to warrants.

As explained below, we agree with defendant that the state

failed to show that the AFTE method is scientifically valid.

However, we disagree that the court erred in denying defen-

dant’s motion to controvert and suppress. We reverse and

remand for further proceedings.

I. PROCEDURAL HISTORY

The charges against defendant arose out of a shoot-

ing at the Speakeasy Lounge in 2018. Directly after the

shooting, police recovered several .40 caliber shell cases

from the ground and found bullet holes in two cars parked

in the nightclub’s lot. The police eventually applied for a

warrant to search a residence where officers believed defen-

dant was residing. During the course of the search, officers

seized, among other things, a Taurus handgun. In prepar-

ing the state’s case against defendant, forensic examiner

Todd used the AFTE method to analyze the Taurus and

the cartridge cases found at the scene.1 Todd concluded that

the Taurus seized from defendant’s residence had fired the

1

As we will discuss below, before defendant’s trial in state court, he was tried

on a federal charge of felon in possession of a firearm, 18 USC § 922(g)(1). In that

case, the government sought to present evidence from a different AFTE exam-

iner who had analyzed the Taurus and the cartridge cases, but the district court

excluded that evidence, concluding that the AFTE methodology is “quasi-scientific”

and depends on subjective determinations of the examiner. United States v. Adams,

444 F Supp 3d 1248 (D Or 2020). Defendant was acquitted of the federal charge.

664 State v. Adams

10 cartridge cases found at the crime scene. After Todd

concluded her analysis, another AFTE examiner, Alessio,

examined the Taurus and cartridge cases and agreed with

Todd’s conclusion.

Defendant moved to controvert the warrant and

moved to exclude, among other things, evidence from Todd

and Alessio. He argued that, under Brown and O’Key, the

state could not meet its burden to show that the AFTE

method, on which Todd’s and Alessio’s expert opinions were

based, was scientifically valid. After holding a hearing pur-

suant to OEC 104, at which it heard testimony from Todd,

Alessio, and defendant’s expert, the trial court denied the

motion to exclude the AFTE identification evidence. The

court also denied defendant’s motion to controvert the war-

rant. At trial, the state presented testimony and a report

from Todd and testimony from Alessio, as well as evidence

discovered during execution of the warrant. The jury found

defendant guilty of two counts of unlawful use of a weapon

with a firearm and two counts of second-degree criminal

mischief; it acquitted him of attempted second-degree mur-

der and attempted first-degree assault.

Defendant appeals, assigning error to the denial

of the two motions. We begin by considering the motion to

exclude the evidence based on the AFTE method.

II. SCIENTIFIC VALIDITY OF THE AFTE METHOD

A. Standards for Admission of Scientific Evidence

On appeal, the parties agree, as they did below, that

Todd’s and Alessio’s evidence is scientific evidence, that is,

that it “draws its convincing force from some principle of sci-

ence, mathematics and the like.” Brown, 297 Or at 407. At

trial, Todd testified that she had been employed by the Oregon

State Police Crime Lab as a forensic scientist; she had used

“the AFTE methodology and tool mark analysis” to analyze

the Taurus and the cartridge cases that were found at the

scene; and her analysis “identified” the Taurus as having fired

the cartridge cases. Alessio testified similarly. The state also

presented Todd’s “Analytical Report,” which stated, “Based on

microscopic comparisons, [the Taurus] was IDENTIFIED as

having fired [the cartridge cases found at the scene].” (Boldface

Cite as 340 Or App 661 (2025) 665

and capitalization in original; internal footnote omitted.) In a

footnote after “IDENTIFIED,” the report stated, in a smaller

font, the AFTE definition of identification: “Identification:

Agreement of all discernible class characteristics and suffi-

cient agreement of a combination of individual characteristics

where the extent of agreement exceeds that which can occur

in the comparison of toolmarks made by different tools and

is consistent with the agreement demonstrated by toolmarks

known to have been produced by the same tool.”

Evidence that jurors perceive to be based on sci-

ence is particularly persuasive. O’Key, 321 Or at 291. As a

result, before allowing scientific evidence to be presented

to the jury, the court, performing a gatekeeping function,

must “ensure that the persuasive appeal is legitimate.” Id.

Specifically, “admissibility of scientific evidence requires a

showing that it is based on scientifically valid principles.”

Id. at 301-02.2

The proponent of the scientific evidence must make

that showing at an OEC 104 hearing by a preponderance of

the evidence.3 Id. at 307 n 29. On appeal, we review for legal

error the trial court’s determination that the proponent made

the necessary showing. Id. at 320 (“ ‘Notwithstanding the

usual deference to trial court discretion, we as an appellate

court retain our role to determine the admissibility of scien-

tific evidence under the Oregon Evidence Code.’ ” (Quoting

Brown, 297 Or at 442; internal footnote omitted.)); id. at 320

n 45 (“[a]lthough this court typically is deferential to a trial

court’s findings of preliminary facts under OEC 104(1), good

2

We have explained that it is possible for expert testimony based on a “prac-

tical” expert’s training and experience—that is not presented as and will not be

perceived as scientific—to be relevant and helpful to the jury and, thus, admissible

as nonscientific expert evidence: “[E]xpert testimony that is based on an expert’s

training and experience is not scientific evidence if it is, and will be recognized

by the trier of fact as, merely the ‘practical observations and assessments’ of a

person who has had ‘unusual specialized experiences.’ ” State v. Beltran-Chavez,

286 Or App 590, 605, 400 P3d 927 (2017) (quoting William Strong, Language and

Logic in Expert Testimony: Limiting Expert Testimony by Restrictions of Function,

Reliability, and Form, 71 Or L Rev 349, 369 (1992) (cited in O’Key, 321 Or at 291,

292)). This case concerns only scientific expert evidence, not nonscientific expert

evidence based on training and experience.

3

We decline the state’s invitation to rule that scientific evidence based on the

AFTE method is presumptively valid, because it does not present “a clear case, a

case for judicial notice, or a case of prima facie legislative recognition.” O’Key, 321

Or at 293 (internal footnote omitted).

666 State v. Adams

reasons exist to modify this approach in the context of sci-

entific evidence”; explaining those reasons (internal citation

omitted)).

In Brown and O’Key, the Supreme Court set out

a variety of factors that may assist courts in determining

whether the proponent of scientific evidence has met their

burden of showing that its underlying principles are scien-

tifically valid. Brown was decided before the United States

Supreme Court adopted a similar approach in Daubert v.

Merrell Dow Pharmaceuticals, Inc., 509 US 579, 113 S Ct

2786, 125 L Ed 2d 469 (1993). In O’Key, the Oregon Supreme

Court expressed agreement with the United States Supreme

Court’s approach in Daubert and adopted its factors—as well

as retaining the factors it had articulated in Brown—under

Oregon law, noting that Oregon courts should treat Daubert

as “instructive.”4 321 Or at 306-07.

The court applies the factors to guide its ultimate

determination regarding scientific validity: “[T]he ‘over-

reaching subject’ of this multifactor, ‘flexible’ inquiry ‘is the

scientific validity—and thus the evidentiary relevance and

reliability—of the principles that underlie a proposed sub-

mission.’ ”5 Id. at 305 (quoting Daubert, 509 US at 594-95).

Although the O’Key court used the term “valid-

ity” to describe the overarching inquiry, it recognized that

it includes consideration of both the ability of the method

to measure what it purports to measure (validity) and the

4

The factors, as articulated in O’Key, include the factors articulated in

Brown—the technique’s general acceptance in the field; the expert’s qualifications

and stature; the use which has been made of the technique; the potential rate of

error; the existence of specialized literature; the novelty of the invention; and the

extent to which the technique relies on the subjective interpretation of the expert—

and the factors articulated in Daubert—whether the theory or technique in ques-

tion can be (and has been) tested; whether the theory or technique has been subject

to peer review and publication; the known or potential rate of error; and the degree

of acceptance in the relevant scientific community. O’Key, 321 Or at 299, 303-04.

5

Although the court in O’Key recognized that the court’s determination of

scientific validity of a particular methodology might be appropriate for judicial

notice in future cases, 321 Or at 293 n 8, it also recognized that understandings

of what is scientifically valid may change over time and on different records: “[N]

o particular reason of logic or good sense exists to immunize particular areas or

principles simply on the basis of longevity or the fact that their introduction ante-

dated imposition of the new standard. Supposedly valid science has not infre-

quently been unmasked.” Id. at 293 n 9 (internal quotation marks omitted).

Cite as 340 Or App 661 (2025) 667

trustworthiness of its conclusions (reliability): “Whereas

validity describes how well the scientific method reasons to

its conclusions, reliability describes the ability of the scien-

tific method to produce consistent results when replicated.”

Id. at 301 n 19. Evidence that is based on a method that is

both scientifically valid—it is “capable of measuring what [it]

purport[s] to measure”—and reliable—it “produce[s] consis-

tent results when replicated”—may be relevant under OEC

401 and sufficiently helpful to the jury to be admissible under

OEC 702. In that case, “it will be excluded only if its probative

value is substantially outweighed by one or more of the coun-

tervailing factors set forth in OEC 403.” Id. at 299. In apply-

ing all of those evidentiary rules, “the court must identify

and evaluate the probative value of the proffered scientific

evidence, consider how that evidence might impair rather

than help the trier of fact, and decide whether truthfinding

is better served by admission or exclusion.” Id. (internal foot-

note omitted).

B. Analysis

As we will explain, in this case, the state did not

meet its burden to show that the AFTE method is scien-

tifically valid, that is, that it is capable of measuring what

it purports to measure and is able to produce consistent

results when replicated. That is so because the method does

not actually measure the degree of correspondence between

shell cases or bullets; rather, the practitioner’s decision on

whether the degree of correspondence indicates a match

ultimately depends entirely on subjective, unarticulated

standards and criteria arrived at through the training and

individualized experience of the practitioner. For a similar

reason, the state did not show that the method is replicable

and therefore reliable: The method does not produce consis-

tent results when replicated because it cannot be replicated.

Multiple practitioners may analyze the same items and

reach the same result, but each practitioner reaches that

result based on application of their own subjective and unar-

ticulated standards, not application of the same standards.

AFTE identification evidence is presented to jurors

as relevant to and probative of whether a particular bullet

or cartridge case was fired from a particular firearm. In the

668 State v. Adams

abstract, evidence of that type may be capable of helping

jurors; as we will explain, many practitioners of the AFTE

method succeed in matching bullets and cartridge case to the

firearms from which they were fired more often than would

be expected based on chance. However, when presented as

scientific evidence, AFTE identification evidence—an “iden-

tification” purportedly derived from application of forensic

science—impairs, rather than helps, the truthfinding pro-

cess because it presents as scientific a conclusion that, in

reality, is a subjective judgment of the examiner based only

on the examiner’s training and experience and not on any

objective standards or criteria. AFTE identification evi-

dence is thus presented to jurors cloaked with the “aura of

reliability” of science even though it is not actually derived

through science. O’Key, 321 Or at 302 n 20. We conclude that

the AFTE method is not scientifically valid under O’Key.

1. The AFTE theory

We begin by describing the AFTE theory. The initial

premise is that unique toolmarks are left inside every fire-

arm during the manufacturing process. Alessio testified that

the tools used to manufacture firearms leave marks on the

internal parts of those firearms. As those tools cut the metal

of the firearms they are manufacturing, the tools wear and

change, and, as a result, the marks that they leave inside the

firearms “will change from one firearm to the next.”

The next premise of the AFTE theory is that bullets

and cartridge cases that have been fired from a given fire-

arm have microscopic features created by the toolmarks left

inside the firearm, as well as other attributes of the firearm,

for example, environmental exposure after manufacture.

Because the manufacturing tools and other features create

marks that are unique to each firearm, the theory goes, the

microscopic features on bullets and cartridge cases fired

from a given firearm will also be unique. The AFTE the-

ory refers to firearms as “tools” and the microscopic features

left on bullets or cartridge cases by the toolmarks inside the

firearm as “toolmarks.”6

6

In other words, somewhat confusingly, there are two sets of tools at issue. First

are the tools used to manufacture firearms, which leave marks inside the firearms.

Second, when examining bullets or cartridge cases, examiners look at “toolmarks”

Cite as 340 Or App 661 (2025) 669

The final premise of the theory is, as Alessio

explained, that the individual marks left on each firearm

and, correspondingly, the resulting marks on each bul-

let or cartridge case fired from each firearm, are different

“enough that a trained examiner can differentiate between”

a bullet or cartridge case fired from one firearm and a bullet

or cartridge case fired by the next firearm manufactured by

the same tool—and, by extension, a bullet or cartridge case

fired by any other firearm.

The AFTE has defined the theory of identification

as follows:

“1. The theory of identification as it pertains to the

comparison of toolmarks enables opinions of common ori-

gin to be made when the unique surface of two toolmarks

are in ‘sufficient agreement.’

“2. This ‘sufficient agreement’ is related to the signif-

icant duplication of random toolmarks as evidenced by the

correspondence of a pattern or combination of patterns of

surface contours. Significance is determined by the com-

parative examination of two or more sets of surface contour

patterns comprised of individual peaks, ridges and furrows.

Specifically, the relative height or depth, width, curvature

and spatial relationship of the individual peaks, ridges and

furrows within one set of surface contours are defined and

compare to the corresponding features in the second set of

surface contours. Agreement is significant when the agree-

ment in individual characteristics exceeds the best agree-

ment demonstrated between toolmarks known to have been

produced by different tools and is consistent with agreement

demonstrated by toolmarks known to have been produced

by the same tool. The statement that ‘sufficient agreement’

exists between two toolmarks means that the agreement of

individual characteristics is of a quantity and quality that

the likelihood another tool could have made the mark is so

remote as to be considered a practical impossibility.

“3. Currently the interpretation of individualization/

identification is subjective in nature, founded on scien-

tific principles and based on the examiner’s training and

experience.”

on the bullets or cartridge cases, which are the marks left by the “tool”—i.e. fire-

arm—that fired the item. The “toolmarks” on the fired item are made, in part, by

the toolmarks left inside the firearm by the tools used to manufacture the firearm.

670 State v. Adams

Association of Firearm and Tool Mark Examiners, Theory of

Identification as it Relates to Tool Marks: Revised, 43 AFTE

Journal 287 (2011) (AFTE Revised Theory of Identification).

Several concepts are helpful to an effort to under-

stand that definition. First is the best known non-match.

Each trained examiner has their own best known non-

match, which is, as Todd explained, “the most agreement

that you’ve ever seen in two [bullets or cartridge cases]

that you know came from different [firearms].” The best

known non-match is based on the examiner’s “personal

identification criteria,” which each examiner develops sub-

jectively through their training: As Alessio explained, any

fully trained examiner has “seen enough matching and

non-matching marks to develop their own personal identi-

fication criteria.” Those criteria exist in the mind of each

examiner, and the AFTE method does not require the exam-

iner to articulate or apply them in any objective way. In the

definition quoted above, “the best agreement demonstrated

between toolmarks known to have been produced by differ-

ent tools” refers to the examiner’s best known non-match.

The next useful term is best known match. Unlike

the best known non-match, which the examiner relies on in

every case, the best known match is case specific, although it

still requires application of the examiner’s personal identifi-

cation criteria. The AFTE method requires the examiner to

test-fire the firearm to obtain “knowns,” which are bullets or

cartridge cases that the examiner knows were fired from the

firearm in question. As Todd testified, the examiner must

“identify what the pattern is for that particular firearm.” To

do that, the examiner puts “two of the known standards on

the comparison microscope, one on each side,” and they “com-

pare the individual characteristics, or when they’re oriented

in the same orientation, we’re going to find the pattern, and

we’re going to basically do what’s known as a pattern match.”

The best known match is the examiner’s assessment of the

amount of agreement that the examiner finds between the

patterns on two knowns test fired from the firearm in ques-

tion. The examiner documents the parts of the pattern that

they find significant for the best known match analysis, but,

at the OEC 104 hearing, neither Todd nor Alessio testified

Cite as 340 Or App 661 (2025) 671

to any objective criteria for measuring the amount of simi-

larity between two knowns. Rather, the examiner relies on

their subjective personal identification criteria to identify

the degree of agreement that they find between the knowns

so that they can mentally compare that degree of agreement

to the degree of agreement they see between the knowns and

the unknowns. In the AFTE theory definition quoted above,

“agreement demonstrated by toolmarks known to have been

produced by the same tool” refers to the best known match.

The concepts of the best known non-match and the

best known match suggest a meaning for the concept of “suf-

ficient agreement,” which is central to the AFTE theory but

whose meaning is not fully articulated by the definition set out

above. The definition expressly states the consequences of the

examiner finding sufficient agreement: “Sufficient agreement”

on “the unique surface of two toolmarks” “enables opinions of

common origin to be made.” That is, when the marks on two

items are in sufficient agreement, an examiner may opine that

the marks were created by the same firearm and, thus, that

that firearm fired the unknown bullets or cartridge cases. The

definition further explains that “[t]he statement that ‘suffi-

cient agreement’ exists between two toolmarks means that the

agreement of individual characteristics is of a quantity and

quality that the likelihood another tool could have made the

mark is so remote as to be considered a practical impossibility.”

In other words, the reason that the examiner can opine on the

common origin of the marks is because, when the examiner

sees sufficient agreement, they consider it practically impossi-

ble that two different tools made the marks.

Although sufficient agreement is the linchpin of

the AFTE theory—it is the standard by which an examiner

reaches their conclusion—the AFTE’s theory does not indicate

how an examiner is to decide whether sufficient agreement

exists.7 However, separately, in its glossary, the AFTE has

7

Confoundingly, the AFTE theory’s standard for identification is “sufficient

agreement,” but the theory discusses the degree of correspondence that an exam-

iner finds between toolmarks in terms of significance rather than sufficiency:

“Agreement is significant when the agreement in individual characteristics

exceeds the best agreement demonstrated between toolmarks known to have

been produced by different tools and is consistent with agreement demon-

strated by toolmarks known to have been produced by the same tool.”

672 State v. Adams

defined “identification” in a way that indicates that agreement

is “sufficient” when it exceeds the examiner’s best known

non-match and is consistent with the best known match:

“Identification” means “[a]greement of all discernible class

characteristics and sufficient agreement of a combination

of individual characteristics where the extent of agreement

exceeds that which can occur in the comparison of toolmarks

made by different tools and is consistent with the agreement

demonstrated by toolmarks known to have been produced

by the same tool.” Association of Firearm and Tool Mark

Examiners, AFTE Glossary, 94 (6th ed 2017), available at

https://afte.org/wp-content‌/uploads‌/2024/11‌/AFTE‌_Glossary‌_

Version‌_6.091922‌_ FINAL‌_COPYRIGHT‌.pdf (accessed

May 5, 2025) (defining “Range of Conclusions Possible When

Comparing Toolmarks”). In this case, both Alessio and Todd

testified that sufficient agreement occurs when the marks on

a bullet or shell case exceed the best known non-match and

are consistent with the best known match.

2. The AFTE method

With that background, we turn to the AFTE method,

which implements the AFTE theory by providing steps that

examiners follow to decide, based on the markings on bul-

lets and cartridge cases, whether they were fired from a par-

ticular firearm (referred to as a “match” or “identification”

with the firearm) or not fired from the particular firearm

(referred to as a “non-match” or “exclusion”). In addition to

a match or a non-match, the examiner may reach a result

of “inconclusive” or, based on the initial examination of the

item, which may be damaged, “unsuitable for comparison.”

Todd testified to the steps of the AFTE method. The

examiner begins by analyzing the unknowns, that is, the

bullets or shell cases that the examiner wants to identify

as having been fired from a particular firearm or not. The

examiner documents the condition of the items and their

class characteristics, which include the caliber of the bul-

lets and the types of marks that will be left on all bullets or

cases fired from some particular type or brand of firearm.8

AFTE Revised Theory of Identification, 43 AFTE Journal at 287 (emphasis added).

8

The method also recognizes subclass characteristics, which are “common

to a group of guns within a certain make or model, such as those manufactured

Cite as 340 Or App 661 (2025) 673

Then the examiner creates the knowns by test firing the

firearm in question. Once the knowns are created, the

examiner notes their class characteristics. If they do not

match the class characteristics of the unknowns, the result

is an exclusion.

Todd explained that, if all of the discernible class

characteristics are in agreement, the examiner moves on

to a microscopic comparison. The examiner begins by com-

paring the two test fires under a comparison microscope to

establish the best known match, which, as noted above, is

case specific. The examiner places two knowns on the micro-

scope, which allows the examiner to view them side by side.

Todd explained the process of pattern matching:

“So a collection of features would be a pattern, if we’re

talking about those striated marks that kind of look like a

bar code. Then you have a series of lines; they’re going to

have high spots and low spots which translate visually as

bright or dark from where the light is shining and catching

on that metal.

“And then you’re also going to have different widths.

There might even be some movement to the line, so instead

of being perfectly straight they might have some—some

wiggle to them, some curve to them. And so that’s what

we’re searching for is a pattern, and then we’re trying to

identify that same pattern in the same location on the

other known.

“Once we have that pattern matched up, then you would

continue comparing the rest of that surface area to estab-

lish how much more agreement or if there’s additional pat-

terns that are present that would be unique to that fire-

arm. And that’s basically establishing your best known

match for this particular firearm.”

Once the examiner has established the best known

match, they swap one of the knowns with an unknown, ori-

ent the items the same way, and consider whether the pat-

tern observed on the knowns is present on the unknowns.

at a particular time and place. An example would include imperfections on a

rifling tool that imparts similar toolmarks on a number of barrels before being

modified either through use or refinishing.” Abruquah v. State, 483 Md 637, 658-

59, 296 A3d 961, 974 (2023) (internal quotation marks and citations omitted).

Distinguishing subclass characteristics from individual ones can be difficult.

674 State v. Adams

The next step, which occurs at the same time as

the comparison step, is for the examiner to evaluate the

agreement between the knowns and the unknown. Todd

explained that she considers whether the amount of agree-

ment she sees is consistent with the best known match and

exceeds her own personal best known non-match:

“[W]hat I’m looking for, for the AFTE theory of identifi-

cation, I am—in order to say that it’s a common source, I

have to have agreement that was consistent with what I

established when I was looking at my knowns. And I have

to have agreement that exceeds what’s known as the best

known non-match. And that is something that’s estab-

lished through training and experience as well as all the

research that’s been done in the field. So as I’m agreeing—

as I’m evaluating these things and establishing how much

agreement I see, then I’m trying to meet those two things,

basically answer those two questions: Does it exceed your

best known non-match and does it—is it consistent with

your best known match?”

Using that process, the examiner reaches a conclu-

sion about whether there is sufficient agreement between

the knowns and the unknown. Consistent with the defini-

tion of the AFTE theory, which explains that “[c]urrently

the interpretation of individualization/identification is sub-

jective in nature, founded on scientific principles and based

on the examiner’s training and experience,” AFTE Revised

Theory of Identification, 43 AFTE Journal at 287, Alessio

testified at the hearing that, although the marks that the

examiner is comparing are objectively observable, the ulti-

mate determination—that is, the comparison of the degree

to which the marks match—is subjective:

“[PROSECUTOR:] How much of the AFTE technique

involves documenting objectively observable marks?

“[ALESSIO:] Most of it. It’s * * * like I talked about—

those class characteristics, those measurable features.

Then we get to those individual marks. But those individ-

ual marks are also measurable features that are reproduc-

ible in the fact that they occur in the same place, in the

same frequency, the same position, and can be repeated by

creating extra test fires to show that those measurable fea-

tures are there and being reproduced over and over and

Cite as 340 Or App 661 (2025) 675

over again. So I think it’s extremely objective until you get

to the last part where a human examiner has to render a

decision.”

Thus, the subjective part is the examiner’s evaluation of

and decision on whether the level of agreement between

the knowns and the unknown exceeds the examiner’s best

known non-match and is consistent with the best known

match for the firearm in question.

The final step of the AFTE method is verification. A

second examiner repeats the process and reaches a conclu-

sion. If it is the same as the first examiner’s conclusion, the

result is ready for reporting.

In addition to providing testimony, both parties

also cited scientific literature on the AFTE theory and

method. The state introduced several studies and papers,

most published in the AFTE Journal, concluding that the

AFTE method is “an accurate and precise method for deter-

mining a common source for bullets and cartridge case[s] for

firearms collected from casework.” Erich Smith, Cartridge

Case and Bullet Comparison Validation Study with Firearms

Submitted in Casework, 36 AFTE Journal, 130, 132 (2004);

see also, e.g., Angela Stroman, Empirically Determined

Frequency of Error in Cartridge Case Examinations Using a

Declared Double-Blind Format, 46 AFTE Journal, 157, 160

(2014).

Defendant cited a 2009 report from the National

Research Council of the National Academy of Sciences (NRC)

and a 2016 report from the President’s Council of Advisors on

Science and Technology (PCAST). See NRC, Strengthening

Forensic Science in the United States: A Path Forward (2009)

(NRC Report), available at https://‌www‌.ojp‌.gov‌/pdffiles1‌/nij‌/

grants‌/228091‌.pdf (accessed May 5, 2025); PCAST, Report to

the President, Forensic Science in Criminal Courts: Ensuring

Scientific Validity of Feature-Comparison Methods (2016)

(PCAST Report), available at https://‌obamawhitehouse‌.

archives‌. gov‌/sites‌/default‌/ files‌/ microsites‌/ostp‌/ PCAST‌/

pcast‌_forensic‌_science‌_report‌_final‌.pdf (accessed Apr 30,

2025). The NRC Report concluded that “[s]ufficient studies

have not been done to understand the reliability and repeat-

ability of the methods” of toolmark identification, NRC

676 State v. Adams

Report at 154—that is, as the PCAST Report paraphrased,

the NRC concluded that “the foundational validity of the

field had not been established.” PCAST Report at 105. The

NRC Report noted that the AFTE Theory “is the best guid-

ance available for the field of toolmark identification, [but]

does not even consider, let alone address, questions regard-

ing variability, reliability, repeatability, or the number of

correlations needed to achieve a given degree of confidence.”

NRC Report at 155; see also id. at 153-54 (under the AFTE

theory, “the decision of the toolmark examiner remains a

subjective decision based on unarticulated standards and

no statistical foundation for estimation of error rates”).

Similarly, the PCAST Report provided a view from

the scientific community on the topic of a court’s ultimate

determination under Daubert and O’Key, which is “ ‘the sci-

entific validity—and thus the evidentiary relevance and

reliability—of the principles that underlie a proposed sub-

mission.’ ” O’Key, 321 Or at 305 (quoting Daubert, 509 US

at 594-95); accord PCAST Report at 42 (same). The PCAST

Report explained that, in light of the Daubert inquiry, “[i]t

is the proper province of the scientific community to provide

guidance concerning scientific standards for scientific valid-

ity.” Ultimately, the report concluded that,

“To establish foundational validity for a forensic fea-

ture-comparison method, the following elements are

required:

“(a) a reproducible and consistent procedure for (i)

identifying features in evidence samples; (ii) comparing

the features in two samples; and (iii) determining, based on

the similarity between the features in two sets of features,

whether the samples should be declared to be likely to come

from the same source (‘matching rule’); and

“(b) empirical estimates, from appropriately designed

studies from multiple groups, that establish (i) the meth-

od’s false positive rate—that is, the probability it declares

a proposed identification between samples that actually

come from different sources and (ii) the method’s sensitiv-

ity—that is, the probability it declares a[n] * * * identifica-

tion between samples that actually come from the same

source.”

Cite as 340 Or App 661 (2025) 677

Id. at 65.

The PCAST Report distinguished between objec-

tive and subjective feature-matching methods. It took the

view that subjective methods can theoretically be valid for

Daubert purposes, but that they require particularly rigor-

ous empirical validation to avoid presenting subjective con-

clusions based on methods that have not been empirically

verified—and thus that have unknown and unknowable

probative value—as being based on science:

“For subjective methods, procedures must still be care-

fully defined—but they involve substantial human judg-

ment. For example, different examiners may recognize or

focus on different features, may attach different impor-

tance to the same features, and may have different criteria

for declaring proposed identifications. Because the proce-

dures for feature identification, the matching rule, and fre-

quency determinations about features are not objectively

specified, the overall procedure must be treated as a kind

of ‘black box’ inside the examiner’s head.

“Subjective methods require careful scrutiny, more gen-

erally, their heavy reliance on human judgment means

that they are especially vulnerable to human error, incon-

sistency across examiners, and cognitive bias. In the foren-

sic feature-comparison disciplines, cognitive bias includes

the phenomena that, in certain settings, humans (1) may

tend naturally to focus on similarities between samples

and discount differences and (2) may also be influenced

by extraneous information and external pressures about a

case.”

PCAST Report at 49.9

9

The report explained that, while some types of subjective judgments may

be scientific absent empirical validation—for example, “[b]ased on experience,

a surgeon might be scientifically qualified to offer a judgment about whether

another doctor acted appropriately in the operating theatre”—opinions based on

measurement and comparison of physical qualities (like AFTE identifications)

require empirical validation in order to have any probative value at all:

“ ‘[E]xperience’ or ‘judgment’ cannot be used to establish the scientific valid-

ity and reliability of a metrological method, such as a forensic feature-com-

parison method. The frequency with which a particular pattern or set of fea-

tures will be observed in different samples, which is an essential element in

drawing conclusions, is not a matter of ‘judgment.’ It is an empirical matter

for which only empirical evidence is relevant.”

Id. at 55.

678 State v. Adams

The PCAST Report summarized the requirements

for scientific validity and reliability: “For a metrological

method [like the AFTE method] to be scientifically valid

and reliable, the procedures that comprise it must be shown,

based on empirical studies, to be repeatable, reproducible,

and accurate, at levels that have been measured and are

appropriate to the intended application.” Id. at 47. It defined

the relevant terms:

“By ‘repeatable,’ we mean that, with known probabil-

ity, an examiner obtains the same result, when analyz-

ing samples from the same sources. By ‘reproducible,’ we

mean that, with known probability, different examiners

obtain the same result, when analyzing the same samples.

By ‘accurate,’ we mean that, with known probabilities, an

examiner obtains correct results both (1) for samples from

the same source (true positives) and (2) for samples from

different sources (true negatives). By ‘reliability,’ we mean

repeatability, reproducibility, and accuracy.”

Id.

Within that framework, the PCAST Report further

developed the NRC Report’s critique of firearms identifica-

tion, the AFTE theory, and the research that purported to

support it. With respect to the existing research studies, it

explained,

“Although firearms analysis has been used for

many decades, only relatively recently has its validity been

subjected to meaningful empirical testing. Over the past

15 years, the field has undertaken a number of studies that

have sought to estimate the accuracy of examiners’ con-

clusions. While the results demonstrate that examiners

can under some circumstances identify the source of fired

ammunition, many of the studies were not appropriate for

assessing scientific validity and estimating the reliabil-

ity because they employed artificial designs that differ in

important ways from the problems faced in casework.”

PCAST Report at 106.10 After discussing the research in

more detail and explaining the problems with the design of

10

The report raised several issues regarding the design of existing studies,

but the most pervasive was that, other than the Ames Laboratory study, which

we describe below, the existing studies were not “black box” studies because they

did not require examiners to make multiple independent identification decisions.

Cite as 340 Or App 661 (2025) 679

all but one of the existing studies, the report ultimately con-

cluded that, with respect to the Daubert standard, “firearms

analysis currently falls short of the criteria for foundational

validity.” Id. at 112. It elaborated:

“The early studies indicate that examiners can, under

some circumstances, associate ammunition with the gun

from which it was fired. However, as described above, most

of these studies involved designs that are not appropriate

for assessing the scientific validity or estimating the reli-

ability of the method as practiced. Indeed, comparison of

the studies suggests that, because of their design, many

frequently cited studies seriously underestimate the false

positive rate.

“At present, there is only a single study that was appro-

priately designed to test foundational validity and esti-

mate reliability (Ames Laboratory study). Importantly, the

study was conducted by an independent group, unaffiliated

with a crime laboratory. Although the report is available

on the web, it has not yet been subjected to peer review and

publication.

“The scientific criteria for foundational validity require

appropriately designed studies by more than one group

to ensure reproducibility. Because there has been only a

single appropriately designed study, the current evidence

falls short of the scientific criteria for foundational validity.

There is thus a need for additional, appropriately designed

black-box studies to provide estimates of reliability.”

PCAST Report at 111 (emphasis in original; internal foot-

note omitted).

As noted in the foregoing quote, the PCAST Report

identified and described a 2014 “black box” study conducted

by the Ames Laboratory at Iowa State University. David P.

Baldwin, Stanley J. Bajic, Max Morris, and Daniel S. Zamzow,

A Study of False-Positive and False-Negative Error Rates in

Cartridge Case Comparisons, Ames Laboratory-USDOE

PCAST Report at 106. Studies of the AFTE method commonly use closed or par-

tially open sets of items, requiring examiners to conduct a matching exercise in

which each match or exclusion that they identify narrows the pool of other pos-

sible matches and exclusions. In other words, as the PCAST Report noted, those

studies work similarly to a game of Sudoku, “where initial answers can be used

to help fill in subsequent answers.” Id. That structure makes those studies very

dissimilar to real-world case work.

680 State v. Adams

Technical Report #IS-5207 (2014) (Ames I), available at

https://‌www.osti.gov‌/servlets‌/purl‌/1987636 (accessed Apr 30,

2025). The PCAST Report generally approved of the design of

Ames I, although it noted some ways in which the study con-

ditions still were dissimilar to case work. It also noted that,

although all the participating examiners were members of

the AFTE who had been suitably trained and employed in

law enforcement, the results suggested that “the false pos-

itive rate is highly heterogeneous across the examiners.”

PCAST Report at 110.11

In 2020, a report on a second Ames Laboratory

black-box validation study of the AFTE method conducted

in response to the PCAST Report was released. Stanley J.

Bajic, L. Scott Chumbley, Max Morris, and Daniel Zamzow,

Ames Laboratory-USDOE Technical Report #ISTR-5220,

Report: Validation Study of the Accuracy, Repeatability, and

Reproducibility of Firearm Comparisons (2020) (Ames II).12

In that study, 173 examiners conducted 20,130 compari-

sons, some of bullets and some of cartridge cases. Id. at 2. In

keeping with the AFTE Range of Conclusions, the examin-

ers were asked to classify bullets and cartridge cases using

four conclusions: “Identification, Inconclusive, Elimination,

and Unsuitable, with three possibilities for qualifying an

Inconclusive decision.” Id. at 13. The researchers divided the

comparisons, allocating them “to assess accuracy (8,640),

repeatability (5,700), and reproducibility (5,790) of the eval-

uations made by participating examiners.” L.S. Chumbley,

M.D. Morris, S.J. Bajic, D. Zamzow, E. Smith, K. Monson,

11

The report summarized the results of the Ames I study:

“Among the 2178 different-source comparisons, there were 1421 elimina-

tions, 735 inconclusives and 22 false positives. The inconclusive rate was 33.7

percent and the false positive rate among conclusive examinations was 1.5

percent (upper 95 percent confidence interval 2.2 percent). The false positive

rate corresponds to an estimated rate of 1 error in 66 cases, with upper bound

being 1 in 46. (It should be noted that 20 of the 22 false positives were made

by just 5 of the 218 examiners—strongly suggesting that the false positive

rate is highly heterogeneous across the examiners.)”

See also Ames I at 11 (noting that “each examiner has his or her own false identi-

fication probability, and * * * these probabilities vary substantially”).

12

Like the Ames I study, it does not appear that the Ames II study has ever

been published or peer reviewed. Defendant’s expert testified that Ames II was

available on the Ames Laboratory Department of Energy website for some period

of time but is no longer available there.

Cite as 340 Or App 661 (2025) 681

G. Peters, Accuracy, Repeatability, and Reproducibility of

Firearm Comparisons Part 1: Accuracy 1 (2021) (Accuracy

Report). As to accuracy, the study had rates of false posi-

tives around 2 percent and sensitivity (the probability that

examiners identify as a match a pair that comes from the

same source) of around 75 percent.13 Alan H. Dorfman &

Richard Valliant, Inconclusives, Errors, and Error Rates in

Forensic Firearms Analysis: Three Statistical Perspectives,

5 Forensic Sci Int’l: Synergy, 100273, Table 3 (2022), avail-

able at https://www.sciencedirect.com/science/article/pii/

S2589871X22000584?via%3Dihub (accessed May 14, 2025);

see Ames II at 34, Table V (providing data); Accuracy Report

at 19-20 (discussing sensitivity). The rate of false negatives

was somewhat higher, 3.6 percent for bullets and 2.3 per-

cent for cartridge cases. Dorfman & Valliant, Inconclusives,

Errors, and Error Rates in Forensic Firearms Analysis:

Three Statistical Perspectives, 5 Forensic Sci Int’l: Synergy,

100273, at Table 3.

13

The Ames II study report included the inconclusive results in the denom-

inator of its calculation of the false positive rates rather than calculating the

rates based only on the conclusive results, as the PCAST Report explained is

necessary. PCAST Report at 51-52 (“When reporting a false positive rate to a

jury, it is scientifically important to calculate the rate based on the proportion

of conclusive examinations, rather than just the proportion of all examinations.

This is appropriate because evidence used against a defendant will typically be

based on conclusive, rather than inconclusive, examinations. To illustrate the

point, consider an extreme case in which a method had been tested 1000 times

and found to yield 990 inconclusive results, 10 false positives, and no correct

results. It would be misleading to report that the false positive rate was 1 percent

(10/1000 examinations). Rather, one should report that 100 percent of the conclu-

sive results were false positives (10/10 examinations).”)

We agree with PCAST that the better method of calculation is to disregard

inconclusives rather than including them in the denominator for false positive

rates. All of the information necessary to recalculate the rate of false positives

and negatives compared to the number of conclusive results is in the Ames II

study report; thus, we rely on the method identified as scientifically appropri-

ate by PCAST. See Alan H. Dorfman & Richard Valliant, Inconclusives, Errors,

and Error Rates in Forensic Firearms Analysis: Three Statistical Perspectives, 5

Forensic Sci Int’l: Synergy, 100273, Table 3 (2022), available at https://‌w ww.sci-

encedirect.com‌/science‌/article‌/pii‌/S2589871X22000584?via%3Dihub (accessed

May 14, 2025) (recalculating Ames II rates of error in a variety of ways; when

treating inconclusives as irrelevant, Ames II data demonstrates “different

source,” i.e. false positive, error rates of 2.04 percent for bullets and 1.86 per-

cent for cartridge cases and “same source,” i.e. false negative, error rates as 3.67

percent for bullets and 2.31 percent for cartridge cases). However, we are unable

to determine the appropriate confidence interval. See id. at 2.2.1 (noting, in any

event, that the Ames II confidence intervals “despite their sophistication, are not

well-grounded”).

682 State v. Adams

In the accuracy section of the Ames II study, as in

Ames I, examiners’ rates of error varied widely:

“[H]ard errors were made by 34 of the 173 examiners when

examining bullets; 36 of 173 for cartridge cases. Three par-

ticipants made both kinds of errors. Analysis of the data

collected in the first round of the study showed that the

six most error-prone examiners account for 33 of the 112

errors—29%[—]while 13 examiners account for almost half

of all the hard errors (54 of 112 errors).”

Accuracy Report at 11-12.

The Ames II investigators also assessed how often

examiners agree with other examiners when they repeat the

same comparison without knowledge of the first examiner’s

conclusion (reproducibility) and how often they agree with

themselves when asked to perform the same comparison a

second time (repeatability). The results indicated that exam-

iners disagreed with other examiners between 22 percent

and 69 percent of the time (depending on whether the items

were matches or non-matches; how the three types of incon-

clusives were grouped; and whether the examined items were

bullets or cartridge cases), id. at 45-52, and examiners dis-

agreed with themselves between 14 percent and 38 percent of

the time, depending on the same variables. Id. at 40-45.

3. United States v. Adams

Before the pretrial motions in this case, defen-

dant was tried on a federal charge arising from the same

incident at issue here. See United States v. Adams, 444 F

Supp 3d 1248, 1251 (D Or 2020). In the federal case, the

district court, Judge Mosman, granted defendant’s motion

in limine to exclude scientific testimony of an AFTE exam-

iner, Gover, who examined the same firearm and cartridge

cases at issue here. The record before the federal district

court—primarily testimony from Gover—was somewhat

different from the record in this case, as we discuss further

below. Notwithstanding the different record, however, we

find the district court’s reasoning persuasive. Accordingly,

we describe it in some detail.

The court began by explaining the relevant federal

legal standard for the admission of scientific expert evidence.

Cite as 340 Or App 661 (2025) 683

It explained that Daubert requires scientific testimony to be

“reliable.” “In order to qualify as science, a proposition ‘must

be derived by the scientific method.’ The expert’s assertion

need not be verified as an objective certainty, but it must

have been derived by a verified scientific process in order

to meet the necessary standard of evidentiary reliability.”

Adams, 444 F Supp 3d at 1257 (quoting Daubert, 509 US at

593-94; internal citation omitted). Daubert sets out five fac-

tors that overlap with the factors that we use under Brown

and O’Key, which assist the court in determining whether

the scientific evidence is sufficiently reliable to be relevant

and helpful to the jury. O’Key, 321 Or at 306-07.

Judge Mosman recognized that, “at a superficial

level,” the AFTE testimony could be viewed as satisfying

the Daubert requirements:

“Replicability? Check. Other forensic examiners trained in

ballistics comparisons can perform an examination using

the same basic methodology. In fact, double check: a foren-

sic examiner in this very case did a second comparison and

came to the same conclusion. Error rate? Check. Testing

seems to indicate an error rate hovering around two per-

cent. Testing and standards? Check, at least as to some of

what is done. Publication? Check. Journals put out by asso-

ciations of forensic examiners have published hundreds of

relevant articles. Acceptance? Check. All reputable forensic

examiners accept the ballistics comparison method used in

this case as valid.”

Adams, 444 F Supp 3d at 1258.

However, the court observed that, “when you look

beneath the surface,” “the problems with [the AFTE] meth-

odology begin to emerge.” Id. It explained that “[i]t is worth

remembering, at this point, that what we are analyzing here

is the admissibility of scientific evidence. It is certainly pos-

sible to know things—and to know them so well that you are

almost never wrong—without resorting to scientific meth-

ods.” Id. (emphasis in original).

“Imagine, for a moment, a school to train * * * art

experts. It is given the name, ‘The Academy for the Science

of Art Examination.’ Everyone there is trained to follow

a particular protocol, with several objective elements of

684 State v. Adams

scientific examination: carbon dating, microscopic exam-

ination, x-rays, and the like. The Academy establishes a

journal, in which its graduates regularly publish papers

describing their work in peer reviewed articles. There is

regular testing in which examiners are sent two works of

art and told to determine which one is a forgery, a test on

which the examiners have only a two percent rate of false

positives—i.e. of misidentifying original works as forgeries.

And they all agree that what they are doing is science. The

only rub is this: when all the objective work is done, and it

comes time to decide whether what they are looking at is

fake, they are told to call upon all the experience they have,

and the feeling that true art invokes in them, and then

make a call by listening to their inner voice telling them

what is right. The ultimate support for their conclusions,

then, is almost entirely subjective and inscrutable.

“Like any analogy, it does not map perfectly over

our facts. But the central point is this: If you have elements

of objective examination, and a lot of training, and you

form a like-minded group to review each other’s work and

publish papers, and if you maintain standards to guide the

objective elements of the process, and ensure that a cer-

tain protocol is followed, and if you are frequently correct

in your assessments, then even if you cannot explain how

you reach your ultimate conclusions, and even if therefore

your process fundamentally is not scientific, you can pass a

superficial reading of the Daubert factors.”

Id. at 1258-59.

The court noted that, even if expert evidence based

on a witness’s training and experience may be admissible as

non-scientific expert evidence, it remains important that evi-

dence not derived from science not be presented to the jury

cloaked with the persuasive appeal of science. Id. at 1259.

Then the court evaluated each of the Daubert factors, conclud-

ing, “On each factor, when one looks more closely at [Gover’s]

testimony and considers the underlying purpose of the inquiry,

the methodology used in this case has problems.” Id.

The court began with testability, which includes

falsifiability and replicability. Id. Although the court deter-

mined that the AFTE method is falsifiable—the proposition

that a particular gun fired a particular bullet or shell case

is capable of being proved false by features that exclude

Cite as 340 Or App 661 (2025) 685

a match between the two—the court concluded that the

method is not replicable, because, as a result of its ultimate

subjectivity, it cannot be applied the same way twice:

“In order to be replicable, a method must be objective

enough that someone else not associated with the case could

duplicate it and get the same results. That standard is very

problematic here. Much of what Mr. Gover did was, in fact,

objective. * * * But when it comes to his ultimate conclusion

of a match, Mr. Gover’s inquiry was not objective.

“The first problem is with the baseline comparator that

Mr. Gover used—the ‘best known non-match’ as a compar-

ator. This requires an examiner to compare the degree of

correspondence on his split screen against the best possi-

ble non-match comparison that anyone is aware of between

consecutively manufactured firearms. * * * But Mr. Gover

could not define this baseline in any objective way, nor

could he explain the role it played in the actual comparison

he made in this case. He apparently just kept a memory

of the baseline in his head and then made a comparison

unguided by any objective standards or benchmarks.”14

Id. at 1260 (internal citation omitted). After providing an

excerpt of Gover’s testimony illustrating the problem, the

court continued,

“With no objective benchmark below which we would know

that a shell casing was not fired from a particular firearm

* * *, replicability begins to fall apart.

“Throughout Mr. Gover’s testimony, this lack of any

objective standard became increasingly evident. It is worth

explaining at the outset that this flaw in Mr. Gover’s testi-

mony appears to derive from the AFTE methodology itself,

not from any deficiency in Mr. Gover’s understanding of

his work. AFTE requires an examiner to find ‘sufficient

agreement’ between crime scene shells and test fired shells

from the firearm in question in order to determine a match.

14

The district court seems to have considered the best known non-match to be

the closest degree of agreement that any examiner anywhere has ever seen between

items fired from different firearms. The record in our case is clear, however, that

the best known non-match is the closest degree of agreement that the particular

examiner has ever seen between items fired from different firearms. That under-

standing is consistent with Judge Mosman’s observation that Gover “kept a memory

of the baseline in his head.” That is what Todd testified that she does as well—her

subjective best known non-match is a product of her training and experience, and it

is not something that can be articulated or produced in written form.

686 State v. Adams

Here is Judge Garaufis of the Eastern District of New York

explaining why ‘sufficient agreement’ is not an objective

standard:

“ ‘First, the sufficient agreement standard is circular

and subjective. Reduced to its simplest terms, the AFTE

Theory “declares that an examiner may state that two tool-

marks have a ‘common origin’ when their features are in

‘sufficient agreement.’ ” PCAST Report at 60. “It then defines

‘sufficient agreement’ as occurring when the examiner con-

siders it a ‘practical impossibility’ that the toolmarks have

different origins.” Id. The NRC Report notes that the AFTE

Theory “is the best guidance available for the field of tool-

mark identification, [but] does not even consider, let alone

address, questions regarding variability, reliability, repeat-

ability, or the number of correlations needed to achieve a

given degree of confidence.” NRC Report at 155. Without

guidance as to the extent of commonality necessary to find

“sufficient agreement,” the AFTE Theory instructs exam-

iners to draw identification conclusions from what is essen-

tially a hunch—a hunch “based on the examiner’s training

and experience,” AFTE Revised Theory of Identification, 43

AFTE Journal at 287—but still a hunch.

“ ‘Moreover, the application of this circular standard is

“subjective in nature * * * based on the examiner’s training

and experience.” AFTE Revised Theory of Identification, 43

AFTE Journal at 287. Ostensibly, one hundred firearms tool-

mark examiners could hold one hundred different personal

standards of when two sets of toolmarks sufficiently agree,

and all one hundred of these personal standards may accord

with the AFTE Theory. Further, because the standard itself

offers so little guidance on when an examiner should make

an identification determination, some examiners may decide

that the two sets of toolmarks were made by the same tool

while others determine the toolmarks to be inconclusive

and still others decide the toolmarks were made by differ-

ent tools. To emphasize, these one hundred examiners could

come to these contradictory conclusions without a single

examiner running afoul of the AFTE Theory.’

“United States v. Shipp, No. 19-cr-029-NGG, 2019 WL

6329658 *13 (E.D.N.Y. Nov. 26, 2019).

“In other words, the AFTE ‘sufficient agreement’ stan-

dard is a tautology that doesn’t mean anything.”

Cite as 340 Or App 661 (2025) 687

Id. at 1261-62 (brackets and second omission in Shipp;

emphasis in Adams; internal citation omitted).

After again excerpting Gover’s testimony, the court

continued:

“Mr. Gover could not say that the person who checked

his work, and who relied on the same methodology, applied

the same standard in reaching the same conclusion. He

could not be sure what threshold [the second examiner]

used to decide that the shell casings were fired from the

Taurus—i.e. that Mr. Gover’s conclusion was correct. Not

only is the AFTE method not replicable for an outsider to

the method, but it is not replicable between trained mem-

bers of AFTE who are using the same means of testing.

“If this were truly a scientific inquiry, [Gover’s tes-

timony indicating that he could not conclusively know what

standard another examiner applied] would not be possible.

If a cancer researcher sought a second opinion from another

cancer researcher in order to reach a diagnosis, both people

would be able to say with certainty what the other person

was looking for and why. If their conclusions deviated, they

would be able to pinpoint the points of disagreement and

why those data points were meaningful.

“Over and over, Mr. Gover failed to do this. He

could not explain which data points he looked at or why

they were meaningful to him. And this is not purely a

fault of Mr. Gover. There is no evidence in this record or

elsewhere that the AFTE method relies on any scientific

standard that would explain to an examiner like Mr. Gover

how to interpret the data he sees in any kind of objective

way. What he is actually doing is applying his training and

experience to make a subjective conclusion about what he

sees before him, just like the art expert in [the earlier-de-

scribed] example. The AFTE method is therefore not rep-

licable—and not testable—because it cannot be explained

in a way that would allow an uninitiated person to perform

the same test in the same way that Mr. Gover did. This fac-

tor weighs heavily against admissibility under Daubert.”

Id. at 1263-64 (footnote omitted).15

15

We understand the court’s reference to an “uninitiated person” to mean

someone seeking to apply the AFTE method without resorting to unarticulated

and unmeasurable standards drawn from their own training and experience—

not someone entirely untrained in the method.

688 State v. Adams

After concluding that the AFTE method is not repli-

cable, the court considered error rates. Id. at 1264. Treating

the rate of false positives as the relevant error rate, it noted

that the error rates shown for the AFTE method vary

depending on the type of study, with black box studies—

which reduce confounding variables better than the other

types of studies but still do not mimic real-world condi-

tions16 —showing a rate of false positives of 2.2 percent. Id. at

1264-65. Given the significance of AFTE identification tes-

timony for convictions, the court pointed out that that error

rate could be expected to result in a wrongful conviction in

one of every 46 cases. Id. at 1264. It also noted that, because

of the design of the relevant studies, real-world error rates

may significantly exceed those reported in the studies:

“The incentive structure for the testing process is also con-

cerning. It appears to be the case that the only way to do

poorly on a test of the AFTE method is to record a false

positive. There seems to be no real negative consequence

for reaching an answer of inconclusive. Since the test tak-

ers know this, and know they are being tested, it at least

incentivizes a rate of false positives that is lower than real

world results. This may mean the error rate is lower from

testing than in real world examinations.”

Id. at 1265 (footnote omitted). That is, in the studies, when

an examiner finds a close case, they are incentivized to

record it as inconclusive—a result that is effectively counted

as a correct result in many studies—rather than a match,

which, if incorrect, will affect the error rate in the study. In

real-world conditions, that incentive is absent: There is no

way to check rates of false positives, and, as the record in

this case indicates, examiners do not keep records of their

16

In particular, the then-existing black box studies were not “test-blind,”

that is, the examiners were aware that they were being tested rather than doing

routine case work. See Dorfman & Valliant, Inconclusives, Errors, and Error

Rates in Forensic Firearms Analysis: Three Statistical Perspectives, 5 Forensic

Science International: Synergy, 100273, at § 2.3; see also PCAST Report at 59

(“PCAST believes that test-blind proficiency testing of forensic examiners should

be vigorously pursued, with the expectation that it should be in wide use, at least

in large laboratories, within the next five years.”).

More recently, at least one small test-blind study has been conducted. See

Maddisen Neuman et al., Blind Testing in Firearms: Preliminary Results from

a Blind Quality Control Program, 67 J Forensic Sci 964 (2022). In that study, 11

examiners made no false identifications or false exclusions in the course of 570

comparisons that were presented as if they were ordinary cases.

Cite as 340 Or App 661 (2025) 689

results (which, if they were kept, could show whether rates

of inconclusive results are lower in case work than in stud-

ies). Ultimately, the court found the rate of error to weigh

against admission. Id.

Moving on to the third Daubert factor, peer review,

the court noted the purpose of that factor in the overall

inquiry:

“The question of whether a methodology has been subjected

to peer review is a question of whether a methodology or

hypothesis has been published to the scientific community

for the purpose of detecting substantive flaws. Daubert,

509 US at 593. If a methodology has been published but

not for this purpose, this factor is not satisfied in favor of

admissibility.”

Id.

The court rejected the idea that the AFTE Journal,

which the government argued shows that the method is

subject to peer review, satisfies that factor for two reasons:

because the AFTE Journal “is a trade publication, meant

only for industry insiders, not the scientific community,” and,

more importantly, because “the purpose of publication in the

AFTE Journal is not to review the methodology for flaws but

to review studies for their adherence to the methodology.”

Id. at 1265-66. That “is not the purpose that Daubert sets

out for peer review.” Id. at 1266.

The court held that the “standards and quality con-

trol” factor was satisfied; the record before it showed that

there were standards and quality control measures in place,

although that did not mitigate the fundamental problem

with subjectivity and consequent lack of replicability. Id. As

to the last factor, “general acceptance in the broader scien-

tific community,” the court observed,

“The AFTE method that Mr. Gover uses has been widely

accepted within his own community of technical experts.

But it has been heavily criticized by other members of the

broader scientific community for failing to yield reproduc-

ible results or a precisely defined process. In fact, [the NRC

Report and PCAST Report] suggest to me that the wide-

spread acceptance within the law enforcement community

may have created a feedback loop that has inhibited the

690 State v. Adams

AFTE method from being further developed. * * * Here,

where the scientific community at large disavows the the-

ory because it does not meet the parameters of science, I

cannot find that the AFTE method enjoys ‘general accep-

tance’ in the scientific community.”

Id. (internal citations omitted).17

In conclusion, the court noted that comparison

analysis like the AFTE method, undertaken by a trained

examiner, may be effective at identifying matches, but the

problem is that, from what was in the record before the

court, the analysis is based on training and experience—

ultimately, hunches—not science:

“Even at its worst, comparison analysis has a very low rate

of error and yields results that cannot be random. But it is

not clear that those results are the product of a scientific

inquiry. Nothing in Mr. Gover’s testimony explains how or

why he reached his conclusion in any quantifiable, repli-

cable way. It is possible that the AFTE method could be

expressed in scientific terms, but I have not seen it done in

this case, nor elsewhere.”

Id. at 1266-67 (emphasis in original). Accordingly, the court

granted defendant’s motion to exclude the AFTE identifica-

tion evidence. Id. at 1267.

4. Brown and O’Key factors

Overall, we find Judge Mosman’s analysis highly

persuasive, and we adopt the portions discussed in this

opinion for purposes of our own analysis under Brown and

O’Key. We see no need to expound further on the factors

of peer review, standards and safeguards, and the tech-

nique’s acceptance in the scientific community. We provide

some additional discussion, however, regarding testability

(and the overlapping Brown factor of the extent to which

the technique relies on the subjective interpretation of the

expert) and error rates, as well as Brown’s additional factors

of the existence of specialized literature and the novelty of

the invention.

17

Accord PCAST Report at 55 (“[S]cientific validity of a method must be

assessed within the framework of the broader scientific field of which it is a part

(e.g., measurement science in the case of feature-comparison methods). The fact that

bitemark examiners defend the validity of bitemark examination means little.”).

Cite as 340 Or App 661 (2025) 691

Initially, we note that, although Judge Mosman—

and, in this opinion, we—occasionally refer to the funda-

mental problem with the AFTE method as relating to sub-

jectivity, we understand the problem to go beyond the fact

that the expert must apply some judgment in the course

of reaching a conclusion. As the PCAST Report suggests,

a method may be scientifically valid—assuming that the

method’s error rate and sensitivity can be empirically

shown—even if it requires some degree of subjective judg-

ment by the expert. PCAST Report at 49. Here, however,

the problem is not that the examiner must apply some judg-

ment in reaching their conclusion; rather, it is that, because

there are no objective standards or criteria at all—the only

question is whether, based on the particular examiner’s past

experience and memory of the pattern replications between

knowns, there is a match—the procedure is not “reproduc-

ible” or “consistent”; each examiner who applies it is actually

doing something different. Id. at 48 (a scientifically valid

method requires “a reproducible and consistent procedure”

for identifying features, comparing features, and determin-

ing whether the samples match).

As noted above, the record in this case differs in

some ways from the record before the federal district court

in Adams. We discuss those differences as they relate to the

O’Key/Daubert factors and the factors identified in Brown.

As explained below, we conclude that the state has failed to

show that the AFTE identification evidence is scientifically

valid.

i. Testability/Replicability and the Extent to

Which the Technique Relies on the Subjective

Interpretation of the Expert

As Judge Mosman noted, in the federal case, “Mr.

Gover could not say that the person who checked his work,

and who relied on the same methodology, applied the same

standard in reaching the same conclusion.” Id. at 1263. That

was a problem because scientific evidence should be based

on a method or analysis applied in a quantifiable way, such

that two people who have done it correctly will have done the

same thing. Accord O’Key, 321 Or at 301 n 19 (in order to be

692 State v. Adams

“reliable,” a method should “produce consistent results when

replicated”).

In this case, Alessio testified that, in his view, fully

trained examiners do use the same standards, and that is

evidenced through the low error rates shown in validation

studies conducted over many years:

“[PROSECUTOR:] And do all examiners correctly

applying [the] AFTE methodology apply the same standard

for declaring something as a match?

“[ALESSIO:] Yes. I think it’s—like the validation

studies and the low error rate show that with the proper

training—

“* * * * *

“[ALESSIO:] * * * I think the validation studies show

that firearm examiners with the proper training, that

rigorous training, when they’ve developed that personal

identification criteria, can very accurately determine

common-source origins of tool marks, and the validation

studies show that with the very low error rate in the false

positives.

“* * * * *

“[PROSECUTOR:] Would other examiners correctly

applying the AFTE methodology agree when a level of

agreement exceeds a best known non-match?

“[ALESSIO:] Yes.

“* * * * *

“[PROSECUTOR:] Even though there’s no numeric

attached, why is the standard of exceeds the best known

non-match able to be consistently applied amongst

examiners?

“[ALESSIO] Again, I think it goes back to those vali-

dation studies. We—we tested it and—

“* * * * *

“So just the low error rates, the misidentifications, the

false positives are really what we’re concerned with. Did

somebody identify something to a gun that didn’t fire it?

And those low error rates over all the years of the studies

show that that training method and that the developing of

Cite as 340 Or App 661 (2025) 693

their personal criteria and how close it is from examiner to

examiner with the proper training can be.”

We understand the state to suggest that the district court’s

reasoning—that the AFTE method is not replicable because

each examiner applies their own subjective and unarticu-

lated standards rather than uniform objective standards—

carries less force in this case because of that difference

between the records. We disagree.

Although Alessio testified that all trained examin-

ers correctly applying the methodology use the same stan-

dards for identifying a match, he also readily acknowledged

that those standards are subjective and unquantifiable. Nor

could he reasonably testify otherwise, as the AFTE theory

expressly notes the subjectivity of the examiner’s determi-

nation, AFTE Revised Theory of Identification, 43 AFTE

Journal at 287, and the AFTE has not articulated any quan-

tifiable standard.

The subjectivity and lack of quantifiability of the

standards undermines Alessio’s inference that, because

studies have indicated that examiners often reach correct

results, all examiners must be applying the same standard.

As Judge Mosman noted, when practitioners of a replicable

method disagree as to the result in a given case, the rea-

son for the disagreement can be identified—with the conse-

quence, in a forensic context, that the trier of fact can assess

the persuasiveness of the various views. That is not the case

with the AFTE method. If two examiners were to disagree

on whether something was a match, each would only be

able to explain that, according to their personal identifica-

tion criteria, the degree of agreement was or was not suffi-

cient to result in a match. See Shipp, 2019 WL 6329658 at

*13 (“Ostensibly, one hundred firearms toolmark examin-

ers could hold one hundred different personal standards of

when two sets of toolmarks sufficiently agree, and all one

hundred of these personal standards may accord with the

AFTE Theory.”).

The same problem confronts the factfinder when

there is no disagreement between examiners: Although

an examiner may show photographs or point to individ-

ual features that do or do not match, the significance of

694 State v. Adams

the similarities or dissimilarities depends entirely on the

examiner’s subjective criteria, based on their own particu-

lar training and experience—to what extent they have seen

that degree of similarity or dissimilarity in matches and

non-matches in the past.

Indeed, in this case, Todd presented a detailed

summary of her comparison process, highlighting numer-

ous similarities that she identified between the knowns and

the unknowns. However, the significance of those similari-

ties could only be evaluated through the lens of her personal

identification criteria:

“[PROSECUTOR:] And what area were you looking at

here for—or what were you looking at here for a match?

“[TODD:] So again I’m looking at those lines, the stri-

ated tool marks that came from the aperture shearing; the

spatial relationships of them; the widths of them; the fact

that they have correspondence basically from where they

start at the bottom on the left side, right above that black

area which is just a shadow from that firing pin impression.

“So from the bottom of it, up to the top where it kind

of whites out, there’s correspondence all the way through

that. So that would be consistent with what I saw in my

test fires. So again, if we’re always equating this back to

evaluating and trying to answer two questions, the first one

is, is it consistent with your best known match, which would

be your test fires; and the second one is does it exceed our best

known non-match.”

The trier of fact has to simply take the examiner’s word for

it that the standard is satisfied: The answer to the ques-

tion “how do you know the agreement is sufficient?” is, ulti-

mately, “I decided that it was sufficient.”18

18

That situation is particularly problematic because, as the district court

noted in Adams, “[i]f someone cloaked in the powerful robe of science tells a jury

he knows something, and the reason why amounts to ‘trust me, I’m a scientist,’

then the government gains most of the power of scientific evidence at none of

the cost. It becomes very difficult to engage such a witness in meaningful cross

examination.” 444 F Supp 3d at 1259 (slip op at 15). Accord PCAST Report at

53 (“Why is it essential to know a method’s false positive rate and sensitivity?

Because without appropriate empirical measurement of a method’s accuracy, the

fact that two samples in a particular case show similar features has no probative

value—and, as noted above, it may have considerable prejudicial impact because

juries will likely incorrectly attach meaning to the observation.”).

Cite as 340 Or App 661 (2025) 695

The state contends that the fact that an expert’s

conclusion based on the AFTE method is subjective does

not mean that it is not scientifically valid. It relies on the

Supreme Court’s discussion in O’Key of subjectivity in the

horizonal gaze nystagmus (HGN) test. 321 Or at 318. There,

the court noted that an officer’s testimony regarding the

officer’s observation of the defendant’s nystagmus was sub-

jective in the sense that the defendant could not meaning-

fully dispute the officer’s observations:

“The HGN test is based on subjective visual observa-

tion. The officer who administers the test has no physical

sample to take to a laboratory. A defendant is not able to

have an expert examine the evidence. Test conditions can-

not be duplicated, and the test results cannot be verified.

Thus, a defendant cannot contradict much of the officer’s

testimony.”

Id. (internal citations omitted).

That type of subjectivity is dispositively different from

the problem with subjectivity in the application of the AFTE

method. Neither the AFTE theory nor the AFTE method pre-

scribes or quantifies what the examiner is looking for; the

examiner is looking for sufficient agreement, which is defined

only by their own personal identification criteria. The subjec-

tivity is in every decision implementing the procedure: which

features are relevant, how they should be compared, and what

they indicate. By contrast, as the court explained in O’Key, the

HGN test quantitatively measures nystagmus—that is, there

is no subjectivity in the determination of the relevance or sig-

nificance of the movement of the person’s eyes. See 321 Or at 296

(HGN evidence rests on the proposition “that there is a causal

relationship between consumption of alcohol and the type of

nystagmus measured by the HGN test” (emphasis added)); id.

at 294-95 (describing the regulations and literature that pro-

vide objective standards for measurement). The HGN test is

subjective only in the sense that the officer’s visual perception

of the person’s eye movement cannot be recorded or confirmed

at a later time—which, the court noted, presents a situation

no more problematic than testimony about other observable

facts that cannot be independently confirmed later. Id. at 318

(“[O]bservation by an officer of the presence of nystagmus is

no more subjective than the observation by an officer of other

696 State v. Adams

indicia of alcohol impairment, such as swaying, staggering,

having bloodshot eyes, or using slurred speech.”). The court

did not, as the state contends, approve as scientifically valid

a method, like the AFTE method, whose application is subjec-

tive and cannot be articulated. Accord State v. Trujillo, 271 Or

App 785, 798, 353 P3d 609, rev den, 358 Or 146 (2015) (finding

that retrograde extrapolation mainly involves “objective appli-

cation of a [mathematical] formula” and thus does not require

subjective interpretation, which favored admissibility of the

scientific evidence); State v. Reed, 268 Or App 734, 745-46, 343

P3d 680, rev den, 357 Or 551 (2015) (application of an assess-

ment tool to determine ability to consent to sexual conduct

required “some subjective evaluation” of the assessed person’s

responses to questions, but the expert’s subjective interpreta-

tion was verified by interviews with others).

Returning to Alessio’s testimony that, because stud-

ies show low error rates, all examiners must apply the same

standards, we also disagree with the factual premise under-

lying his inference. In his view, because studies conducted

over many years show that examiners rarely make false

positive identifications, all examiners must be using the

same standards. But when members of the scientific commu-

nity have reviewed that body of studies, they have rejected

the studies conducted “over many years” that he relied on.

In PCAST’s words, the studies that existed in 2016, other

than Ames I, “involved designs that are not appropriate for

assessing the scientific validity or estimating the reliability

of the method as practiced.” PCAST Report at 111.

Finally, the data that are directly on point do not

support Alessio’s view that examiners apply the same stan-

dards. As explained above, in the Ames II study, investiga-

tors found that, upon repeating a comparison, examiners

disagreed with other examiners between 22 percent and 69

percent of the time and disagreed with themselves between

14 percent and 38 percent of the time. Ames II at 45, 52.

Those data indicate that, contrary to Alessio’s testimony,

different examiners do not reliably use the same standards

to determine how something should be categorized.19

19

The data also call into question whether even the same examiner is actu-

ally using the same standard across cases.

Cite as 340 Or App 661 (2025) 697

The state contends that, although those rates show

that examiners are not “perfect[ ],” results are “more repro-

ducible and repeatable than ‘would be attributed to chance.’ ”

(Quoting Ames II at 40). In the state’s view, that shows that

the examiners’ opinions “must be based on something objec-

tive,” which is a good enough basis on which to provide sci-

entific testimony to jurors.

We disagree. Our ultimate question is whether the

method actually measures what it purports to measure and

whether it reliably produces the same results when repli-

cated, that is, applied to the same items in the same way. The

AFTE method does not measure against anything beyond

the particular examiner’s subjective standards, and for that

reason, it also cannot be applied the same way by more than

one examiner. Ultimately, it is different to say that a method

is valid and replicable—it operates according to a consistent

standard that governs the process and decision-making of

examiners and it reaches accurate, repeatable, and repro-

ducible results—than that a group of practitioners can often

make correct identifications based on subjective standards

arrived at through training and experience. Adams, 444 F

Supp 3d at 1263-64 (“What [an AFTE examiner] is actually

doing is applying his training and experience to make a sub-

jective conclusion about what he sees before him, just like

the art expert in [the earlier-described] example.”).

In sum, the testability factor and Brown’s additional

factor, “the extent to which the technique relies on subjec-

tive interpretation of the expert,” 297 Or at 436, weigh very

heavily against the possibility that AFTE identification evi-

dence is scientifically valid such that it may be presented to

the jury as evidence derived from science.

ii. Potential Rates of Error

The state contends that the potential rate of error

is low and, consequently, weighs in favor of admitting AFTE

identification evidence as scientific evidence. It disagrees with

the district court’s reasoning in Adams, contending that, even

considering only the Ames studies, the error rate is low enough

to justify admission. It also points out that Adams was decided

before the Ames II study was published and asserts that Ames

698 State v. Adams

II is a second black-box study that, now that it exists, fills in

as the single additional black-box study that PCAST identified

as necessary to validate the AFTE method.

For his part, defendant contends that an additional

problem exists with rates of error that goes beyond the issues

identified in the PCAST Report. In his view, the two Ames

studies do not reflect accurate rates of error because of the

way they account for results of “inconclusive.” He argues

that the state has not met its burden to show that the error

rate of the AFTE method is low because no study has cor-

rectly identified the rate of error for the AFTE method.

Defendant asserts that we cannot rely on the

reported error rates in any of the existing studies because

none of the studies recognizes or accounts for the fact that

some examiners identify as inconclusive pairings that other

examiners identify as matches and that, when that occurs,

one of the two examiners must be making a mistake. In

other words, one examiner cannot be correct that there is

insufficient information to arrive at an identification or

exclusion if another examiner is correct that there is enough

information (and, based on that information, reaches a cor-

rect conclusion). The Ames studies both focused only on

“hard errors,” i.e. false identifications and false exclusions.

As we have calculated their error rates, explained above,

they treat answers of “inconclusive”—including answers

of “inconclusive” given for pairs that other examiners have

counted as identifications or exclusions—as immaterial.

We agree with defendant that the fact that two

answers are both treated as correct is problematic, but we

view the problem as resulting, at the most basic level, from

the AFTE method’s inherent subjectivity, that is, the fact

that each examiner’s standard for sufficient agreement is

different. Because each examiner is applying their own

version of the AFTE method, with individual error rates

and sensitivity, they will sometimes reach different results

based on the same comparison. We agree with defendant

that that cuts against the conclusion that AFTE evidence is

scientific.

Cite as 340 Or App 661 (2025) 699

However, we do not view that as a problem specific

to rates of error.20 In our view, the central question for the

rate-of-error factor is the frequency of false positives—how

often an AFTE examiner will testify that a given item was

fired from a particular firearm when, in fact, it was not fired

from that firearm. On that question, the two Ames studies

provide some, albeit imperfect, information about how often

examiners make false identifications.

The state points to the two Ames studies as indi-

cating that the rate of false positives of the AFTE method

is low. That is somewhat true, as a matter of averages. As

the district court observed, the 2.2 percent false positive

rate from Ames I translates to a potentially wrong convic-

tion in one out of every 46 cases. As we have noted, we are

unable to calculate the false positive rate at a 95 percent

confidence interval from Ames II; however, the observed rate

was around 2 percent.

Ultimately, on this record, we conclude that the

rate of error factor is neutral. Even if we were to agree with

the state that the Ames studies show low rates of error and

accept that those rates are representative of average rates

of error in case work, because every examiner applies a dif-

ferent standard for the AFTE method—resulting in the fact

that, as the Ames studies indicate, each examiner has a dif-

ferent rate of error and even a different error probability—

identifying an average error rate across all examiners is not

particularly helpful to the trier of fact, who, on this record,

lacks information that would allow them to determine where

on the error spectrum the testifying examiner falls.21

20

We also recognize that, as defendant points out, the problem with incen-

tives that Judge Mosman identified—in a study, examiners are incentivized to

classify close cases as inconclusive, thereby avoiding a match or exclusion that

may be hard error—makes ignoring inconclusives a problematic study-design

choice in any study that is not test blind. Further, we do not disagree that, ulti-

mately, the inconclusives may include some mistakes, where, under an examin-

er’s subjective criteria, there is enough information to make a conclusive determi-

nation, but the examiner nevertheless classifies the comparison as inconclusive.

As explained in the text, notwithstanding those issues, we find it more practical,

given the currently available data, to focus on the rate of false positives.

21

As the PCAST Report observes,

“Demonstrating that an examiner is capable of reliably applying the method

is crucial—especially for subjective methods, in which human judgment plays

a central role. From a scientific standpoint, the ability to apply a method

700 State v. Adams

iii. The existence of specialized literature

As mentioned previously, both parties cite scien-

tific literature regarding the AFTE method, although the

content of that literature—whether it criticizes or lauds the

AFTE method—varies. As the Supreme Court explained in

Brown, this factor does not hinge on the substantive content

of the cited specialized literature, but rather on there being

“no dearth of literature available on [a] controversial sub-

ject.” 297 Or at 433. Here, the contrast between the parties’

views on the AFTE method is reflected in the cited litera-

ture. We find that this factor favors admissibility.

iv. The novelty of the invention

The evidence in this record shows that the basic

premise of toolmark comparison has been used since the

late 1920s and that the AFTE theory of identification was

“adopted in 1992.” Given that the method has been formally

included in the field of forensic science for over thirty years,

we find the AFTE to not be novel, and thus this factor favors

admissibility.

5. AFTE identification evidence is based on practical

expertise, not science, and therefore will not assist the

jury when it is presented as being based on science.

Having considered each factor separately, we return

to the overall inquiry under OEC 401 and 702, considering

“the probative value of the proffered scientific evidence” and

“how that evidence might impair rather than help the trier

of fact.” O’Key, 321 Or at 299. The factors outlined in Brown

reliably can be demonstrated only through empirical testing that measures

how often the expert reaches the correct answer.”

PCAST Report at 56. That reference to empirical testing appears to contemplate

more than the routine proficiency testing that examiners take. See id. at 68 (not-

ing that “easy [proficiency] tests are favored by the community, with the result

that tests that are too challenging could jeopardize repeat business for a commer-

cial vendor” (internal quotation marks omitted)); see also Hofmann, Carriquiry,

& Vanderplas, Treatment of Inconclusives in the AFTE Range of Conclusions, 19

Law, Probability, and Risk 317, 343 (2020) (proposing use of “decision-specific

error rates” in court, which would take into account “examiner-specific probabili-

ties of incorrect conclusions as well as examiner-specific historical decisions”). In

this case, Todd testified that she had not made any errors on her competency test-

ing, which occurs before an examiner is certified, and that she had participated

in a study (which was a closed-set study) and had not made any errors. She also

acknowledged that she did not keep data on her previous comparisons.

Cite as 340 Or App 661 (2025) 701

and O’Key are ultimately nonexhaustive and were meant to

be dynamic enough to address the nuances of various prof-

fered scientific techniques. Id. at 305 (explaining that the

proper inquiry is a “flexible” inquiry into the evidentiary

relevance and reliability of the principles underlying the

proposed method, “not on the conclusions that they gener-

ate”); see also Brown, 297 Or at 417-18 (“What is important

is not lockstep affirmative findings as to each factor, but

analysis of each factor by the court in reaching its decision

on the probative value of the evidence under OEC 401 and

OEC 702.” (Internal footnote omitted.)).

Here, both parties agree that the AFTE theory of

identification is relevant in this case, and thus we need not

make any further inquiry under OEC 401. However, after

applying the principles articulated in Brown and O’Key, we

conclude that AFTE identification evidence in this case was

not helpful to the jury. The subjective nature of the AFTE

method, and the related fact that each practitioner has a

different rate of error, makes it impossible for the court to

“ensure that the persuasive appeal” of the evidence to the

jury “is legitimate.” O’Key, 321 Or at 291. We therefore con-

clude on this record that the state did not carry its burden

of showing that the AFTE method is scientifically valid.

C. Harmlessness

The state argues that, because the AFTE identifi-

cation evidence was cumulative of other “more probative”

evidence, any error was harmless. We disagree. The state

contends that surveillance camera footage from the crime

scene, which was presented to the jury, “leave[s] no room

for reasonable doubt that defendant is the same person as

the shooter” in the footage. Thus, the state argues, “the jury

[was] unlikely to have rested its verdict on the toolmark iden-

tification.” The state’s theory of harmlessness would require

us to reweigh the evidence, which we may not do. State v.

Davis, 336 Or 19, 32, 77 P3d 111 (2003) (In reviewing the

record to evaluate whether error was harmless, “we do not

determine, as a factfinder, whether the defendant is guilty.

That inquiry would invite this court to engage improperly

in weighing the evidence and, essentially, retrying the case,

while disregarding the error committed at trial, to determine

702 State v. Adams

whether the defendant is guilty. Rather, when we review the

record, we do so in light of the error at issue. We ask whether

there was little likelihood that the error affected the jury’s

verdict.”); see also, e.g., State v. Ramirez, 310 Or App 62, 67,

483 P3d 1232 (2021) (“[W]e do not usurp the role of the fact-

finder and determine if defendant is guilty or reweigh the

evidence.” (Internal quotation marks omitted.)). We cannot

say that there is little likelihood that admission of the AFTE

identification evidence affected the verdict.

II. MOTION TO CONTROVERT

In defendant’s second assignment of error, he con-

tends that the trial court erred by denying his motion to

controvert search-warrant affidavits and suppress evidence

obtained pursuant to those warrants. In support of his con-

tention, defendant relies on Article I, section 9, of the Oregon

Constitution, ORS 133.693, and the Fourth Amendment to

the United States Constitution.

Defendant filed a pretrial motion to controvert

certain facts in the affidavits in support of the state’s war-

rants, and to suppress evidence discovered as a result of the

warrants. He argued that the affidavits left out material

evidence concerning an alternative suspect—specifically,

an eyewitness description of the shooter’s clothing that dif-

fered from what defendant was wearing in the surveillance

video—and that, when reevaluated considering that mate-

rial omission, the affidavits did not establish probable cause.

The trial court denied that motion for two reasons: First, the

court found that the omission of the eyewitness description

was not a material omission. Second, the court held that,

even if the omission were material, the magistrate granting

the warrant would have reasonably concluded that “the facts

as alleged with all reasonable inferences establish probable

cause.”

On appeal, defendant asserts that the court erred

in both its determination that the omission was not mate-

rial and its conclusion that, even if the omission was mate-

rial, probable cause nevertheless existed. Assuming with-

out deciding that defendant is correct on the first point, we

disagree with him on the second point. The omission of the

Cite as 340 Or App 661 (2025) 703

eyewitness account of defendant’s clothing did not under-

mine the inference that defendant was the shooter depicted

on the surveillance video because, as the trial court found,

the description was not significantly inconsistent with

defendant’s clothing as depicted on the video and, conse-

quently, did not undermine the affiant’s belief that the video

indicated that defendant was the shooter.22 Accordingly, we

affirm the denial of the motion to controvert.

III. CONCLUSION

Because the court erred in admitting the AFTE

identification evidence and that error was not harmless, we

reverse and remand.

Reversed and remanded.

22

To the extent that defendant takes issue with the wording of the trial

court’s conclusion that “the facts as alleged with all reasonable inferences estab-

lish probable cause,” we understand the court to have described the correct stan-

dard in imprecise terms rather than applying an incorrect legal standard.