Standards for Conduct and Evaluation of Activities Carried Out by the Office of Educational Research and Improvement (OERI); Evaluation of the Performance of Recipients of Grants, Cooperative Agreements, and Contracts

SUMMARY: The Assistant Secretary proposes to establish regulations

pursuant to OERI's authorizing legislation, the Educational Research,

Development, Dissemination, and Improvement Act of 1994. The major

purpose of these standards is to ensure that the research, development,

and dissemination activities carried out by the recipients of grants

from and contracts and cooperative agreements with OERI meet the

highest standards of professional excellence.

DATES: Comments must be received by the Department on or before April

27, 1998.

ADDRESSES: All comments concerning these proposed regulations should be

addressed to Sharon Bobbitt, U.S. Department of Education, 555 New

Jersey Avenue, NW., room 508c, Washington, DC 20202-5651. Comments may

also be sent through the Internet to: [email protected]

You must include the term Phase III in the subject line of your

electronic message.

Comments that concern information collection requirements must be

sent to the Office of Management and Budget at the address listed in

the Paperwork Reduction Act section of this preamble. A copy of those

comments may also be sent to the Department representative named in

this section.

FOR FURTHER INFORMATION CONTACT: Sharon Bobbitt. Telephone: (202) 219-

2126. Internet: (Sharon__B[email protected]). Individuals who use a

telecommunications device for the deaf (TDD) may call the Federal

Information Relay Service (FIRS) at 1-800-877-8339 between 8 a.m. and 8

p.m., Eastern time, Monday through Friday.

Individuals with disabilities may obtain this document in an

alternate format (e.g., Braille, large print, audiotape, or computer

diskette) on request to the contact person listed in the preceding

paragraph.

SUPPLEMENTARY INFORMATION:

Invitation to Comment

Interested persons are invited to submit comments and

recommendations regarding these proposed regulations. To ensure that

public comments have maximum effect in developing the final

regulations, the Department urges commenters to identify clearly the

specific section or sections of the proposed regulations that each

comment addresses and to arrange comments in the same order as the

proposed regulations.

The Secretary particularly requests comments on the role of

Department of Education staff in the implementation of the Standards.

For example, should Department staff serve as reviewers on peer review

panels under these regulations? See proposed Sec. 702.10(d) of these

regulations in this regard. Should there be a maximum number or maximum

percentage of Department staff on peer review panels? Should the

participation of Department staff vary by size of the grant, contract,

or cooperative agreement? What other issues about the role of

Department staff in the peer review process should the Secretary

consider?

All comments submitted in response to these proposed regulations

will be available for public inspection, during and after the comment

period, in Room 600, 555 New Jersey Avenue, N.W., Washington, D.C.,

between the hours of 8:30 a.m. and 4:00 p.m., Eastern time, Monday

through Friday of each week except Federal holidays.

On request the Department supplies an appropriate aid, such as a

reader or print magnifier, to an individual with a disability who needs

assistance to review the comments or other documents in the public

rulemaking docket for these proposed regulations. An individual with a

disability who wants to schedule an appointment for this type of aid

may call (202) 205-8113 or (202) 260-9895. An individual who uses a TDD

may call the Federal Information Relay Service at 1-800-877-8339,

between 8 a.m. and 8 p.m., Eastern time, Monday through Friday.

To assist the Department in complying with the specific

requirements of Executive Order 12866 and its overall requirement of

reducing regulatory burden, the Secretary invites comments on whether

there may be further opportunities to reduce any regulatory burdens

found in these proposed regulations.

Background

On March 31, 1994, President Clinton signed Pub. L. 103-227, which

includes Title IX, the Educational Research, Development,

Dissemination, and Improvement Act of 1994 (the Act). The Act

restructured OERI and provided it with a broad mandate to conduct an

array of research, development, dissemination, and improvement

activities aimed at strengthening the education of all students.

Statutory Requirements

The Act directed the Assistant Secretary to develop, in

consultation with the National Educational Research Policy and

Priorities Board (the Board), such standards as may be necessary to

govern the conduct and evaluation of all research, development, and

dissemination activities carried out by OERI to ensure that these

activities meet the highest standards of professional excellence. The

Board is responsible for reviewing and approving the standards. The

legislation requires that the standards be developed in three phases.

In the first phase, standards were created and promulgated to

establish the peer review process and evaluation criteria to be used

for the review of applications for grants and cooperative agreements

and proposals for contracts. The final regulations setting out these

standards were published on September 14, 1995 (60 FR 47808). In the

second phase, standards were created and promulgated to establish the

criteria to be used in reviewing potentially exemplary and promising

educational programs. The final regulations setting out these standards

were published on November 17, 1997 (62 FR 61427).

In the third phase, which is the subject of this notice of proposed

rulemaking (NPRM), the Act requires that OERI develop standards for

evaluating and assessing the performance of all recipients of grants

from and cooperative agreements and contracts with OERI. This

evaluation must take place both during and at the conclusion of the

performance of the grant, cooperative agreement, or contract, and must

include the use of a system of peer review for the final assessment.

In developing the standards, the Assistant Secretary was required

to review the procedures utilized by the National Institutes of Health

(NIH), the National Science Foundation (NSF), and other Federal

departments or agencies engaged in research and development and to

solicit recommendations from research organizations and members of the

general public. OERI has reviewed the procedures used to evaluate the

performance of recipients of grants, contracts, or cooperative

agreements by several offices within NIH and NSF, the Office of Energy

Research in the

[[Page 9393]]

Department of Energy, the Food and Drug Administration, the National

Institute of Standards and Technology, the National Aeronautics and

Space Administration, and the University Research Initiative of the

Department of Defense. Recommendations concerning these standards have

been obtained from the American Educational Research Association, the

Council for Educational Development and Research, and the Organization

of Research Centers. Public comment is invited in response to this

NPRM.

Standards

The standards have been developed by the Assistant Secretary in

consultation with the Board. The standards in this NPRM would:

Require interim and final assessments of the performance

of recipients of grants, cooperative agreements, and contracts.

Establish procedures for selecting peer review panels to

conduct these assessments.

Establish procedures and criteria that the peer review

panels use in conducting these assessments.

Establish specific additional criteria that peer review

panels use in conducting these assessments for National Research and

Development Centers, Regional Educational Laboratories, Field-Initiated

Studies, and ERIC Clearinghouses.

In an effort to fulfill the law's intention of ensuring high-

quality research, development, and evaluation, OERI has developed

standards in which interim and final assessments may be supplemented by

a self-assessment by the recipient of a grant, cooperative, agreement,

or contract. The Board and the Assistant Secretary believe that the

collection and review of evidence on one's own performance is itself a

useful tool for improvement.

These standards cover all grants, cooperative agreements, and

contracts administered by OERI, ranging from the smallest purchase

orders and commissioned papers to the largest research projects and

research centers. The Department will require a single interim

assessment by a peer review panel for total awards of $5,000,000 or

less. At least one interim review by peer review panel will be required

for larger awards. A final assessment by a peer review panel will be

required for all awards.

The Government Performance and Results Act requires the

establishment of performance indicators for Department activities.

Information collected pursuant to those indicators will be considered,

as appropriate, in the evaluation of individual recipients.

Executive Order 12866

1. Potential Costs and Benefits

These proposed regulations have been reviewed in accordance with

Executive Order 12866. Under the terms of the order the Secretary has

assessed the potential costs and benefits of this regulatory action.

The potential costs associated with the proposed regulations are

those resulting from statutory requirements and those determined by the

Secretary as necessary for administering this program effectively and

efficiently. Burdens specifically associated with information

collection requirements are identified and explained elsewhere in this

preamble under the heading Paperwork Reduction Act of 1995.

In assessing the potential costs and benefits--both quantitative

and qualitative--of these proposed regulations, the Secretary has

determined that the benefits of the proposed regulations justify the

costs.

The Secretary has also determined that this regulatory action does

not unduly interfere with State, local, and tribal governments in the

exercise of their governmental functions.

To assist the Department in complying with the specific

requirements of Executive Order 12866, the Secretary invites comment on

whether there may be further opportunities to reduce any potential

costs or increase potential benefits resulting from these proposed

regulations without impeding the effective and efficient administration

of the program.

Summary of Potential Costs and Benefits

The potential costs of the proposed regulations are discussed in

this preamble under the Paperwork Reduction Act of 1995. The benefit of

these standards is to ensure that the research, development, and

dissemination activities carried out by the recipients of grants from

and contracts and cooperative agreements with OERI meet the highest

standards of professional excellence.

2. Clarity of the Regulations

Executive Order 12866 requires each agency to write regulations

that are easy to understand.

The Secretary invites comments on how to make these proposed

regulations easier to understand, including answers to questions such

as the following: (1) Are the requirements in the proposed regulations

clearly stated? (2) Do the proposed regulations contain technical terms

or other wording that interferes with their clarity? (3) Does the

format of the proposed regulations (grouping and order of sections, use

of headings, paragraphing, etc.) aid or reduce their clarity? Would the

proposed regulations be easier to understand if they were divided into

more (but shorter) sections? (A ``section'' is preceded by the symbol

``Sec. '' and a numbered heading; for example, Sec. 702.2 What

activities must be evaluated by these standards?) (4) Is the

description of the proposed regulations in the ``Supplementary

Information'' section of this preamble helpful in understanding the

proposed regulations? How could this description be more helpful in

making the proposed regulations easier to understand? (5) What else

could the Department do to make the proposed regulations easier to

understand?

A copy of any comments that concern how the Department could make

these proposed regulations easier to understand should be sent to

Stanley M. Cohen, Regulations Quality Officer, U.S. Department of

Education, 600 Independence Avenue, SW. (Room 5121, FB-10), Washington,

D.C. 20202-2241.

Regulatory Flexibility Act Certification

The Secretary certifies that these proposed regulations would not

have a significant economic impact on a substantial number of small

entities.

The small entities that would be affected by these proposed

regulations are small local educational agencies (LEAs) and private

schools receiving Federal funds under this program. However, the

regulations would not have a significant economic impact on the small

LEAs and private schools affected because the proposed regulations

would not impose excessive regulatory burdens or require unnecessary

Federal supervision. The proposed regulations would impose minimal

requirements to ensure the proper expenditure of program funds.

Paperwork Reduction Act of 1995

Sections 702.22 and 702.23 contain information collection

requirements. As required by the Paperwork Reduction Act of 1995 (44

U.S.C. 3504(h)), the Department of Education has submitted a copy of

these sections to the Office of Management and Budget (OMB) for its

review.

Collection of Information: Standards for Evaluation of the

Performance of Recipients of OERI Grants, Cooperative Agreements, and

Contracts.

These regulations affect the following types of entities eligible

to enter into

[[Page 9394]]

grants, cooperative agreements, or contracts: any public or private

agency, organization or institution, or individual.

The public reporting burden is estimated to range from 8 to 120

hours for each interim or final assessment. The actual burden will be

determined by how much descriptive information each recipient wishes to

provide.

Organizations and individuals desiring to submit comments on the

information collection requirements should direct them to the Office of

Information and Regulatory Affairs, OMB, room 10235, New Executive

Office Building, Washington, DC 20503; Attention: Desk Officer for U.S.

Department of Education.

The Department considers comments by the public on this proposed

collection of information in--

Evaluating whether the proposed collection of information

is necessary for the proper performance of the functions of the

Department, including whether the information will have practical use;

Evaluating the accuracy of the Department's estimate of

the burden of the proposed collection of information, including the

validity of the methodology and assumptions used;

Enhancing the quality, usefulness, and clarity of the

information to be collected; and

Minimizing the burden of the collection of information on

those who are to respond, including through the use of appropriate

automated, electronic, mechanical, or other technological collection

techniques or other forms of information technology; e.g., permitting

electronic submission of responses.

OMB is required to make a decision concerning the collection of

information contained in these proposed regulations between 30 and 60

days after publication of this document in the Federal Register.

Therefore, a comment to OMB is best assured of having its full effect

if OMB receives it within 30 days of publication. This does not affect

the deadline for the public to comment to the Department on the

proposed regulations.

Assessment of Educational Impact

The Secretary particularly requests comments on whether the

proposed regulations in this document would require transmission of

information that is being gathered by or is available from any other

agency or authority of the United States.

Electronic Access to This Document

Anyone may view this document, as well as all other Department of

Education documents published in the Federal Register, in text or

portable document format (pdf) on the World Wide Web at either of the

following sites:

http://gcs.ed.gov/fedreg.htm

http://www.ed.gov/news.html

To use the pdf you must have the Adobe Acrobat Reader Program with

Search, which is available free at either of the previous sites. If you

have questions about using the pdf, call the U.S. Government Printing

Office toll free at 1-888-293-6498.

Anyone may also view these documents in text copy only on an

electronic bulletin board of the Department. Telephone: (202) 219-1511

or, toll free, 1-800-222-4922. The documents are located under Option

G-- Files/Announcements, Bulletins and Press Releases.

Note: The official version of this document is the document

published in the Federal Register.

List of Subjects in 34 CFR Part 702

Education, Educational research, Reporting and recordkeeping

requirements.

Dated: December 23, 1997.

Ricky Takai,

Acting Assistant Secretary for Educational Research and Improvement.

(Catalog of Federal Domestic Assistance Number does not apply)

The Secretary proposes to amend Chapter VII of Title 34 of the

Code of Federal Regulations by adding a new part 702 to read as

follows:

PART 702--STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES CARRIED

OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT (OERI)--

EVALUATION OF THE PERFORMANCE OF RECIPIENTS OF GRANTS, COOPERATIVE

AGREEMENTS, AND CONTRACTS

Subpart A--General

Sec.

702.1 What is the purpose of these standards?

702.2 What activities must be evaluated by these standards?

702.3 What additional activities may be evaluated by these

standards?

702.4 When is performance assessed under these standards?

702.5 What definitions apply?

Subpart B--Selection of Peer Review Panels

702.10 What are the characteristics of peer reviewers?

702.11 What constitutes a conflict of interest for grants and

cooperative agreements?

702.12 What constitutes a conflict of interest for contracts?

702.13 How are peer reviewers selected for panels?

Subpart C--The Evaluation Process

702.21 How does a peer review panel evaluate the performance of a

recipient?

702.22 What information does a peer review panel consider for an

interim assessment?

702.23 What information does a peer review panel consider for a

final assessment?

702.24 What evaluation criteria are used for performance

assessments?

Authority: 20 U.S.C. 6011(i), unless otherwise noted.

Subpart A--General

Sec. 702.1 What is the purpose of these standards?

(a) The standards in this part implement section 912(i) of the

Educational Research, Development, Dissemination, and Improvement Act

of 1994 (the Act).

(b) These standards are intended to ensure that the research,

development, and dissemination activities carried out by the recipients

of grants from and contracts and cooperative agreements with the Office

of Educational Research and Improvement (OERI) meet the highest

standards of professional excellence.

(Authority: 20 U.S.C. 6011(i)(2)(F))

Sec. 702.2 What activities must be evaluated by these standards?

These standards apply to activities carried out by OERI using funds

appropriated under section 912(m) of the Act including activities

carried out by the following entities or programs:

(a) The National Education Research Institutes.

(b) The Office of Reform Assistance and Dissemination.

(c) The Educational Resources Information Center.

(d) The Regional Educational Laboratories.

(e) The Teacher Research Dissemination Demonstration Program.

(f) The Goals 2000 Community Partnerships Program.

(g) The National Educational Research Policy and Priorities Board.

(Authority: 20 U.S.C. 6011(i)(1))

[[Page 9395]]

Sec. 702.3 What additional activities may be evaluated by these

standards?

(a) The Secretary may apply these standards to other activities

funded by the Department.

(Authority: 20 U.S.C. 6011(I)(1))

Sec. 702.4 When is performance assessed under these standards?

(a) The Secretary will assess the performance of recipients of OERI

grants, contracts, and cooperative agreements subject to these

standards during and at the conclusion of their period of performance.

(b) The Department requires a single interim assessment by a peer

review panel for total awards of $5,000,000 or less. At least one

interim review by peer review panel is required for larger awards.

(c) A final assessment by a peer review panel is required for all

awards.

(d) As used in this part--

(1) Interim assessment is one conducted during a recipient's period

of performance.

(2) Final assessment is one conducted at the conclusion of a

recipient's period of performance.

(Authority: 20 U.S.C. 6011(I)(2)(F))

Sec. 702.5 What definitions apply?

(a) Definitions in the Educational Research, Development,

Dissemination, and Improvement Act of 1994.

The following terms used in this part are defined in 20 U.S.C.

6011(1):

Development

Dissemination

Educational research

(b) Definitions in the Education Department General Administrative

Regulations. The following terms used in this part are defined in 34

CFR 77.1:

Application

Award

Department

Grant

Project

Secretary

(c) Definitions in the Federal Acquisition Regulation. The

following term used in this part is defined in 48 CFR Chapter 1:

Contract Proposal

(Authority: 20 U.S.C. 6011(I)(2)(F))

Subpart B--Selection of Peer Review Panels

Sec. 702.10 What are the characteristics of peer reviewers?

(a) The Assistant Secretary selects each peer reviewer. Each peer

reviewer must have the necessary knowledge and expertise in the area of

the project being reviewed to evaluate the performance of a recipient.

This experience may include--

(1) Expert knowledge of subject matter in the area of the

activities to be reviewed;

(2) Expert knowledge of theory or methods or both in the area of

the activities to be reviewed;

(3) Practical experience in the area of the activities or type of

institution or both to be reviewed;

(4) Knowledge of a broad range of education policies and practices;

(5) Experience in managing complex organizations; or

(6) Expertise and experience in evaluation theory and practice.

(b) Each peer reviewer must be free of conflict of interest, as

determined in accordance with Sec. 702.11 or 702.12.

(c) The Assistant Secretary may solicit nominations for peer

reviewers from professional associations, nationally recognized

experts, and other sources.

(d) OERI and other Department staff who possess the qualifications

in paragraphs (a) and (b) of this section may serve as peer reviewers.

(Authority: 20 U.S.C. 6011(I)(2)(B))

Sec. 702.11 What constitutes a conflict of interest for grants and

cooperative agreements?

A peer reviewer assessing the performance of the recipient of a

grant from or cooperative agreement with OERI is considered an employee

of the Department for the purposes of conflict of interest analysis. As

an employee of the Department, the peer reviewer is subject to the

provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's

policies used to implement those provisions.

(Authority: 20 U.S.C. 6011(I)(2)(B))

Sec. 702.12 What constitutes a conflict of interest for contracts?

A peer reviewer assessing the performance of the recipient of a

contract with OERI is considered an employee of the Department in

accordance with the Federal Acquisition Regulation (FAR), 48 CFR 3.104-

4(h)(2). As an employee of the Department, the peer reviewer is subject

to the provisions of the FAR, 48 CFR Part 3, Improper Business

Practices and Personal Conflict of Interest.

(Authority: 41 U.S.C. 423)

Sec. 702.13 How are peer reviewers selected for panels?

(a) The Assistant Secretary assigns peer reviewers to panels that

conduct the performance assessments.

(b) The Assistant Secretary may establish panels by category of

recipient, such as a panel to review the performance of all Regional

Educational Laboratories. Each recipient is evaluated individually by

reviewers who have been assigned to this type of panel.

(Authority: 20 U.S.C. 6011(I)(2)(B))

Subpart C--The Evaluation Process

Sec. 702.21 How does a peer review panel evaluate the performance of a

recipient?

(a) In each evaluation, a peer review panel--

(1) Considers relevant information about the recipient's

performance, as described in Secs. 702.22 and 702.23; and

(2) Makes judgments about the recipient's performance, using the

criteria in Sec. 702.24.

(b) Each peer reviewer prepares a report based on the reviewer's

assessment of the quality of the project according to the evaluation

criteria.

(c) After each peer reviewer has evaluated each project

independently, the panel may be convened to discuss the strengths and

weaknesses of the project. Each reviewer may then independently re-

evaluate each project with appropriate changes made to the written

report.

(d) The report of the interim assessment must include any

recommendations the peer reviewer may have for improving the

recipient's performance.

(e) The report of the final assessment must contain each peer

reviewer's evaluative summary of the recipient's performance, from the

beginning of the contract, grant, or cooperative agreement to its

conclusion.

(Authority: 20 U.S.C. 6011(I)(2)(F))

Sec. 702.22 What information does a peer review panel consider for an

interim assessment?

(a) Sources of information for the interim assessment must

include--

(1) The original request for proposals or grant announcement and

the contract proposal or grant application;

(2) Documentation of any changes in the work described in the

contract, grant, or cooperative agreement, including reasons for the

changes;

(3) Any progress reports delivered to the Department or made

available to the public by the recipient;

(4) Examples of products delivered to the Department or made

available to the public by the recipient;

(5) Any relevant reports written by OERI staff, including reports

of site visits by OERI staff;

(6) Any performance evaluations conducted under the FAR or the

Education Department General Administrative Regulations (34 CFR part

75).

[[Page 9396]]

(7) Any relevant information provided by the recipient in response

to Government Performance and Results Act (GPRA) (Pub. L. 103-62)

requirements; and

(8) Any reports from program evaluations commissioned by the

Department.

(b) Sources of information for the interim assessment may also

include--

(1) A self-assessment, prepared by the recipient, addressing the

criteria in Sec. 702.24;

(2) One or more site visits by the peer review panel;

(3) One or more oral or written presentations to the panel by the

recipient describing its performance; or

(4) Other information about the recipient's performance.

(Authority: 20 U.S.C. 6011(I)(2)(F))

Sec. 702.23 What information does a peer review panel consider for a

final assessment?

(a) Sources of information for the final assessment must include--

(1) The original request for proposals or application notice and

the contract proposal or grant application, together with documentation

of any changes in the work described in the proposal or application,

including reasons for the changes;

(2) If consistent with the recipient's contract, grant, or

cooperative agreement with OERI, a written report or oral presentation

or both by the recipient summarizing its activities and

accomplishments;

(3) Any relevant information provided by the recipient in response

to Government Performance and Results Act (GPRA) (Pub. L. 103-62)

requirements; and

(4) Any reports from program evaluations commissioned by the

Department.

(b) The final assessment may also include other sources of

information, such as one or more of those listed in Sec. 702.22.

(Authority: 20 U.S.C. 6011(I)(2)(F))

Sec. 702.24 What evaluation criteria must be used for performance

assessments?

(a) Peer reviewers (and those recipients who conduct self-

evaluations) shall use the criteria in paragraph (b) of this section to

assess performance and, in case of interim assessments, to identify

areas in which the performance of recipients may need improvement.

(b) The following evaluation criteria are to guide the assessment

process undertaken by peer reviewers. The peer reviewers determine the

extent to which recipients meet these criteria:

(1) Implementation and management. (i) Peer reviewers shall

consider the degree to which the recipient has fully executed its

program of work. In doing so, peer reviewers shall consider evidence on

the extent to which the recipient completes the work described in the

approved application or contract, including any approved modifications,

in the time period proposed and in an efficient manner.

(ii) In examining the degree of implementation, peer reviewers may

also consider evidence on the extent to which--

(A) The recipient implements and utilizes a quality assurance

system for its products or services or both; and

(B) The recipient conducts self-assessment or self-evaluation

activities, including periodically seeking out independent critiques

and evaluations of its work, and uses the results to improve

performance.

(2) Quality. (i) Peer reviewers shall consider the degree to which

the recipient's work approaches or attains professional excellence. In

determining quality, peer reviewers shall consider evidence on the

extent to which--

(A) The recipient utilizes processes, methods, and techniques

appropriate to achieve the goals and objectives for the program of work

in the approved application; and

(B) The recipient applies appropriate processes, methods, and

techniques in a manner consistent with the highest standards of the

profession.

(ii) In determining quality, peer reviewers may also consider the

extent to which the recipient conducts a coherent, sustained program of

work informed by relevant research.

(3) Utility. (i) In determining the utility of the recipient's

products or services or both, peer reviewers shall consider evidence on

the extent to which the recipient's work (including information,

materials, processes, techniques, or activities) is effectively used by

and is useful to its customers in appropriate settings.

(ii) In determining utility, peer reviewers may also consider the

extent to which the recipient has received national recognition; e.g.,

articles in refereed journals and presentations at professional

conferences.

(4) Outcomes and impact. (i) Peer reviewers shall consider the

results of the recipient's work. In examining outcomes and impact, peer

reviewers shall consider evidence on the extent to which--

(A) The recipient meets the needs of its customers; and

(B) The recipient's work contributes to the increased knowledge or

understanding of educational problems, issues, or effective strategies.

(ii) In examining outcomes and impact, peer reviewers may also

consider the extent to which recipients address issues of national

significance through its products or services or both.

(c) For National Research and Development Centers, peer reviewers

also shall consider evidence on the extent to which recipients meet the

following criteria:

(1) Quality. (i) The recipient uses a well-conceptualized framework

and sound theoretical and methodological tools in conducting

professionally rigorous studies; and

(ii) The recipient conducts work of sufficient size, scope, and

duration to produce sound guidance for improvement efforts and future

research.

(2) Utility. The recipient documents, reports, and disseminates its

work in ways to facilitate the effective use of its work in

appropriately targeted settings.

(3) Outcomes and impact. (i) The recipient's work contributes to

the development and advancement of theory in the field of study,

including its priority area; and

(ii) The recipient addresses issues of national significance

through its products or services or both.

(d) For the Regional Educational Laboratories, peer reviewers also

shall consider evidence on the extent to which recipients meet the

following criteria:

(1) Quality. (i) The recipient utilizes a well-conceptualized

framework and sound theoretical and methodological tools in conducting

professionally rigorous studies;

(ii) The recipient conducts work of sufficient size, scope, and

duration to produce sound guidance for improvement efforts; and

(iii) The recipient's products are well-tested and based on sound

research.

(2) Utility. The recipient documents, reports, and disseminates its

work in ways to facilitate its effective use in appropriately targeted

settings, particularly in school improvement efforts of States and

localities.

(3) Outcomes and impact. (i) The recipient assists States and

localities to implement comprehensive school improvement strategies

through the provision of research-based information (including well-

tested models and strategies), materials and assistance; and

(ii) The recipient's work results in widespread access to

information regarding research and best practices, particularly within

its region.

(e) For Field-Initiated Studies, peer reviewers also shall consider

evidence on the extent to which recipients meet the following criteria:

[[Page 9397]]

(1) Implementation and management. The recipient's work responds to

the goals, objectives and mission of the National Institute from which

it is funded.

(2) Quality. The recipient utilizes a well-conceptualized framework

and sound theoretical and methodological tools in conducting

professionally rigorous studies.

(3) Utility. The recipient documents, reports, and disseminates its

work in ways to facilitate its effective use in appropriately targeted

settings.

(4) Outcomes and impact. (i) The recipient's work contributes to

the development and advancement of theory and knowledge in the field of

study; and

(ii) The recipient addresses issues of national significance

through its products or services or both.

(f) For the ERIC Clearinghouses, peer reviewers also shall consider

evidence on the extent to which recipients meet the following criteria:

(1) Quality. The recipient applies an integrated approach to

acquiring and disseminating significant and high-quality educational

literature and materials to maintain and enhance the ERIC database.

(2) Utility. The recipient contributes to the development of the

ERIC database as a source of literature and materials that reflects

trends and issues within its scope.

(3) Outcomes and impact. (i) The recipient meets the informational

and educational needs of its customers through dissemination and

outreach approaches and the development of an array of print and non-

print materials; and

(ii) The recipient provides national leadership on the use of

current computer, networking, and information technology.

(Authority: 20 U.S.C. 6011(I)(2)(F))

[FR Doc. 98-4690 Filed 2-23-98; 8:45 am]

BILLING CODE 4000-01-P