Stewardship Contracting Pilot Projects: Multiparty Monitoring and Evaluation Process

DEPARTMENT OF AGRICULTURE

Forest Service

Stewardship Contracting Pilot Projects: Multiparty Monitoring and

Evaluation Process

AGENCY: Forest Service, USDA.

ACTION: Notice.

-----------------------------------------------------------------------

[[Page 44686]]

SUMMARY: Section 347 of the FY 1999 Omnibus Appropriations Act

authorizes the Forest Service to enter into 28 stewardship end results

contracts. These contracts are intended to provide a means for pilot-

testing an array of potential new authorities for giving national

forest managers greater administrative flexibility to improve forest

conditions and potentially help meet the needs of local communities.

One of the key provisions of the statute, embodied in subsection (g) of

the Act, is a provision requiring the establishment of a process for

multiparty monitoring and evaluation of the stewardship contracts. The

Forest Service hereby gives notice that a draft framework for

multiparty monitoring and evaluation has been developed and is now

available for public review and comment.

DATES: Comments must be received, in writing, on or before September

16, 1999.

ADDRESSES: Written comments on this proposal may be sent to Cliff

Hickman, USDA Forest Service, Forest Management Staff, Mail Stop 1105,

P.O. Box 96090, Washington, D.C. 20090-6090 or electronically to

chickman/[email protected].

FOR FURTHER INFORMATION CONTACT: Cliff Hickman, Forest Management

Staff, (202) 205-1162, or chickman/[email protected]. Electronic copies of

the Act, and of this proposed framework for multiparty monitoring and

evaluation, may be obtained via the INTERNET at www.fs.fed.us/land/fm/

stewardship.

SUPPLEMENTARY INFORMATION: Section 347 of the FY 1999 Omnibus

Appropriations Act (Act) authorizes the Forest Service to implement up

to 28 stewardship end results contracts. The Forest Service provided

background information about the provisions of Section 347 and its

progress in implementing the legislation, in a notice that appeared in

the July 9, 1999, issue of the Federal Register (64 FR 37096). That

notice identified the stewardship pilot projects that the agency had

already selected. This notice sets out the proposed framework and

requests public comment. A notice summarizing the public comment and

the agency's response to that comment will be published along with the

finalized framework.

The framework that the agency proposes to use to comply with the

requirements of subsection (g) of Section 347 of the Act consists of

two parts: (1) A process for securing multiparty monitoring and

evaluation, and (2) a set of criteria to be considered during

monitoring and evaluation. Besides ensuring proper documentation of any

treatments and their resulting environmental effects, the proposed

framework is intended to provide an objective basis for assessing the

implications of the stewardship pilots regarding the following:

(1) The potential advantages of greater collaboration during period

planning and implementation;

(2) The potential for new authorities to facilitate efficient

implementation of desired ecosystem restoration, maintenance, or

protection activities; and

(3) The potential of stewardship contracting to help meet the needs

of local communities.

This proposed framework represents only one option for satisfying

the multiparty monitoring and evaluation requirement of the Act. It is

designed to comply with the provisions of the Federal Advisory

Committee Act (FACA). The proposed framework is being developed to

facilitate the data inventory required by Section 347 of the Act and is

not a public information survey. Comments and suggestions on this

proposal and on other ways to accomplish multiparty monitoring and

evaluation, that would be in compliance with FACA requirements, are

requested.

Proposed Process

At the national level, a single Advisory Committee will be

established under the provisions of the Federal Advisory Committee Act

(FACA). The framework proposes establishing two types of teams at the

local (project) level: (1) Data Inventory teams, and (2) Assessment and

Evaluation Teams. Due to the composition, duties, and purpose of these

teams, FACA chartering is not required. Additional details concerning

the structure and functions of these proposed groups are provided

below:

Local Data Inventory Teams. Each forest with a stewardship pilot

project will be responsible for organizing a team of interested publics

to gather the data needed to assess project and program results. These

teams will operate under the direction of a Forest Service employee who

will encourage participation of all team members, develop monitoring

methods, schedule team assignments, compile and validate the team's

data, and interact, as needed, with the appropriate Assessment and

Evaluation Team. Where a Forest has more than one pilot project,

separate Data Inventory teams will be established to compile the

relevant data for each project.

As the name suggests, the principal function of the Data Inventory

teams will be to gather project (local) level data for analysis by the

appropriate Assessment and Evaluation Team. The criteria that the Data

Inventory teams will be required to consider are described under the

subheading ``Criteria for Local Data Inventory Teams.'' Once validated,

the facts gathered by the Data Inventory teams will be made available

for public review and use.

The Data Inventory teams will be free to compile facts and other

evidence deemed relevant by the responsible Assessment and Evaluation

Team, but any additional data must be factual rather than interpretive

data. Limiting the role of these teams to the compilation of factual

evidence eliminates the need to establish a FACA committee.

Participation on the teams will be open to all interested parties.

Local Monitoring Assessment and Evaluation Team. Each forest with a

stewardship pilot project will be responsible for organizing a team of

government employees, federal, state, local, or tribal. This Monitoring

Assessment and Evaluation Team's tasks are to assess and evaluate the

data compiled by the corresponding Data Inventory Team and, also, to

formulate appropriate recommendations. The Assessment and Evaluation

Teams will operate under the direction of a Forest Service employee,

serving as chairperson. A non-Forest Service person will be selected as

the vice-chairperson of the committee. If other governmental entities

are unavailable or unwilling to participate in the evaluation process,

a Forest Service team will be organized. Where a forest has more than

one stewardship pilot, only one Assessment and Evaluation Team will be

established to deal with all the projects on that forest.

As already suggested, a key function of the Assessment and

Evaluation Teams will be to analyze and weigh the significance of the

factual evidence compiled by the Data Inventory Teams. The criteria

that the Assessment and Evaluation Teams will be required to consider

are described under the subheading ``Criteria for Local Assessment and

Evaluation Teams.'' These teams, at their discretion, will be free to

develop other site-specific criteria that they deem relevant--e.g.,

additional criteria pertaining to forest conditions before and after

treatment, effects on the local economy, and relations between and

among community members, including the Forest Service. Finally, the

Assessment and Evaluation Teams will be free to interact with the Data

Inventory Teams and the National Advisory Committee, as needed.

[[Page 44687]]

The Assessment and Evaluation Teams will be responsible for making

judgments, reaching conclusions, and formulating recommendations on the

basis of the data assimilated. Because the teams, as envisioned in this

proposal, will be comprised of government employees only, they qualify

for the FACA requirement exemption, authorized by Section 204 of the

Unfunded Mandates Act of 1995. Section 204 provides for the formation

of intergovernmental committees to exchange official views concerning

the implementation or administration of intergovernmental

responsibilities.

A second key function of the local Assessment and Evaluation teams

will be to prepare annual reports to the National Advisory Committee.

Annual reports will be submitted to the National Advisory Committee

within 60 days of the close of each fiscal year. In situations where a

forest has more than one stewardship pilot, each project will be

analyzed, evaluated, and discussed in a separate report.

The local Assessment and Evaluation Team reports will include the

descriptive data that has been compiled by the local Data Inventory

Teams, the responses to the criteria described under the subheading

``Critria for Local Assessmet and Evaluation Teams,'' and any other

issues that the teams determine to be relevant.

The Assessment and Evaluation Teams will actively seek input from

the public. Therefore, they will open their meetings to the public and

give adequate notice of the times and locations of their meetings.

Draft annual evaluation results will be shared with the public for

comment by posting a notice of availability of the results in the local

newspapers, posting the results on each forest's INTERNET website (if

available), and by holding one or more public meetings. Public comments

will be considered in preparing the annual reports to the National

Advisory Committee. All public comments will be retained in the

monitoring and evaluation file for each project.

National Advisory Committee. A FACA advisory committee will be

chartered to monitor and evaluate the stewardship pilot program at the

national level. The Forest Service will recommend potential National

Advisory Committee members to the Office of the Secretary of USDA.

Committee representation will reflect the need to represent all

communities of interest and to ensure that there is balance in the

views represented. To ensure a connection to the projects at the local

level, at least two Committee members will be people serving on local

monitoring and evaluation teams. This arrangement will help ensure that

the broad array of criteria considered and the resulting

recommendations present an accurate, coherent picture of what the

section 347 stewardship pilots have actually accomplished and

demonstrated. Notices concerning the National Advisory Committee's

establishment, its membership, and meeting information will be

published in the Federal Register. The Forest Service will provide

organizational support to the National Committee.

The National Advisory Committee will be responsible for obtaining

the information it needs from the local monitoring and evaluation teams

and will interact with these teams as needed to discharge its duties.

The Committee, at a minimum, will consider the criteria described under

the subheading ``Criteria for the National Advisory Committee.'' These

criteria focus on national issues, such as, whether national forest

policies and priorities were advanced by the new authorities (processes

and procedures) being tested, whether the interests of non-local

publics were adequately considered, and whether agency accountability

for actions and outcomes was appropriately maintained. The National

Committee will also identify and evaluate the important ``lessons

learned'' from the stewardship pilots; they will assess what worked

well and what did not work well. As part of this process, they will

describe any barriers that had to be overcome or that prevented smooth

implementation of the pilot projects.

The National Advisory Committee will be responsible for preparing

annual reports to the Forest Service's Washington Office Forest

Management Staff. The Committee's reports will form the basis for the

Forest Service's required annual reports to Congress.

The Committee's annual reports are to be completed within 60 days

of its receiving the reports from the local Assessment and Evaluation

Teams. These reports will contain a compilation of descriptive data

pertaining to such things as: the acreages treated for different

purposes; the costs incurred; the sources of project funding; the types

of products produced; the revenues generated; the types of

collaborators involved in project planning, implementation, and

monitoring; the roles played by different collaborators; and the

processes and procedures that were tested. The reports also will

address the information requested by Congress, the criteria listed

under the subheading ``Criteria for National Advisory Committee,'' and

any other issues that the National Committee determines to be

important.

Other Process Principles. Other principles guiding the monitoring

and evaluation process include the following:

* All monitoring and evaluation teams will be structured so as to

encompass a diverse mix of resource management skills.

* As needed and as is reasonable, the Forest Service will

compensate monitoring and evaluation team members for any travel costs

that they incur as a result of their service to the agency.

* All monitoring and evaluation team members will be encouraged to

network with their constituents and bring new information and issues

forward.

* Whenever possible, pilots will be designed to include two types

of controls: (1) areas where no vegetative treatment is occurring, and

(2) areas where standard timber sale and/or service contracting

procedures are being observed.

Proposed Criteria

To measure whether the new authorities have achieved the desired

results, four categories of criteria will be considered by the

different monitoring and evaluation (ME) groups. These categories are:

(1) Biophysical critiera, (2) economic criteria, (3) social criteria,

and (4) administrative criteria.

Within each category, some of the criteria call for compiling

numerical or descriptive data, while other criteria require that some

aspect of a pilot project's performance be assessed. As noted earlier,

to assure compliance with FACA requirements, the local Data Inventory

Teams will consider only the criteria calling for numerical or

descriptive responses. Within this constraint, and recognizing that

there may be certain questions that may be relevant in some, but not

all, situations, it is proposed that all teams be free to entertain

questions beyond those listed. In all instances, however, the criteria

that are listed, as a minimum, would be addressed by the designated

groups. In addition to addressing the criteria set-forth, the local

Data Inventory Teams will be encouraged to establish photo points that

will record the condition of the landscape, before, during, and after

project implementation.

Criteria for the Local Data Inventory Teams. The local Data

Inventory Teams will answer, at a minimum, the 16 criteria listed in

this section. Additional criteria may be added, but they must be of an

objective, factual nature.

Biophysical Criteria will include:

[[Page 44688]]

(1) The stated purpose and need for the project.

(2) The project objectives.

(3) The land management treatments being applied. All treatments

applied in connection with a particular are to be considered,

including: the mileage of road maintained or obliterated; the mileage

of trails maintained or obliterated; the acreage of soil and water

improvement; the acreage of terrestrial habitat restored or enhanced;

the mileage of stream habitat restored or enhanced; the acreage of

fuels management by mechanical means and/or prescribed burning; the

acreage treated to decrease insect, disease, or fire risks, and/or to

enhance forest health; and the acreage of noxious and/or exotic weed

control. For each treatment that is applied, the local Data Inventory

teams with gather data on the accomplishments realized to date using

regular appropriations, the exchange of goods for services, or retained

receipts.

Economic Criteria will include:

(4) The total project costs, and the breakdown of these costs into

the following categories: formal project planning and NEPA (including

citizen/public involvement in the process); sale/contract preparation;

sale/contract administration; citizen involvement (during project

implementation); monitoring, evaluation & reporting (including citizen

involvement in this process); and other (to be specified).

(5) The funding of project implementation, and the breakdown of

this funding into the following categories: exchange of goods for

services; retained receipts; regular appropriations; cooperator

contributions (cash); cooperator contributions (materials or in-kind

services); and other funding sources (to be specified). For

appropriated funds, the local Data Inventory teams will show the

amounts provided from each of the Forest Service's recognized fund

codes.

(6) The types and amounts of forest products produced, including:

sawtimber; pulpwood; posts and poles; and different types of special

forest products (ferns, pine boughs, pine straw, mushrooms, etc.). In

all instances, product amounts will be expressed in terms of commonly

recognized units.

(7) The total project receipts, and the proportion of these

receipts attributable to: timber products; special forest products; and

other products (to be specified).

(8) The disposition of the project receipts, showing the amounts:

returned to Treasury; exchanged for services; retained and reinvested;

or distributed in some other manner (to be specified).

(9) The manner in which the pilot changed employment or

entrepreneurial opportunities in the local community.

(10) The special skills required of a contractor for the project.

(11) The difficulties encountered in hiring contractors with the

needed skills.

(12) The average wage paid in connection with the project and

whether this wage rate represented woods a worker, service contract, or

Davis-Bacon wage rate.

(13) The duration of the contract for this project and whether the

contract period was longer or shorter than what is common with

conventional timber sale or service contract projects.

Social Criteria will include:

(14) The individuals and/or groups (other than the Forest Service)

who collaborated in planning, implementing, or monitoring the project,

and the manner in which they were selected.

(15) The roles that each collaborator performed.

Administrative Criteria will include:

(16) The new processes and/or procedures that were used in

connection with the project. The possibilities to be considered

include: awarding of contracts on a ``best value'' basis (specify how

``best value'' was determined); designation by prescription; end

results contracting; exchange of goods for services; retention of

receipts; use of an appraisal method other than standard procedures

(method to be specified); offering sales (appraised value of over

$10,000) without advertisement; using state foresters as federal

agents; using service contracts of over 5 years duration; using

contract logging with subsequent sale of the cut products; or using

some other new process or procedure (to be specified).

Criteria for the Local Assessment and Evaluation Teams. The local

Assessment and Evaluation teams will use the evidence compiled by the

local Data Inventory teams to, as a minimum, address the 14 criteria

that follow. The Assessment and Evaluation teams may consider any

additional criteria that they deem relevant.

Biophysical Criteria will include:

(1) An assessment as to whether the stated purposes and needs for

the project were fulfilled and the basis for the conclusion.

(2) An assessment as to whether the resource management objectives

of the project were realized and an explanation for the conclusion.

(3) An assessment as to whether the Forest Service was able to do a

better job of ecosystem management by giving a single contractor the

responsibility for a ``bundled group'' of resource work activities

(e.g., timber extraction, watershed restoration, habitat improvement,

and road obliteration) on the project area and an explanation of the

conclusion.

Economic Criteria will include:

(4) An assessment as to whether employment opportunities for local

communities were enhanced as a result of the project and the basis for

the conclusion.

(5) An assessment as to whether the prevailing wage rate in the

local community was enhanced as a result of the project and the basis

for the conclusion.

Social Criteria will include:

(6) An assessment as to whether the dynamics of the collaborative

process permitted all interested parties to participate and the basis

for the conclusion.

(7) An assessment as to whether and how collaboration facilitated

planning, implementing, and monitoring for the project.

Administrative Criteria will include:

(8) An assessment as to whether difficulties were experienced in

interpreting or implementing the Section 347 authorities.

(9) An assessment as to whether the project planning and

implementation timelines were being met and what contributed to that

outcome.

(10) An assessment as to how the new processes and/or procedures

that were tested in this project compare to the Forest Service's

conventional timber sale or service contract authorities. As

appropriate, in making that determination, the teams will consider the

following performance variable: attractiveness to potential bidders;

fairness to potential bidders; implications for the Forest Service's

ability to maintain accountability for the treatments being applied and

the forest products being removed; implications for the Forest

Service's ability to implement ecosystem management projects

efficiently and effectively; implications for the Forest Service's

ability to successfully manage small diameter, under-utilized material;

the ease of administration; the agency's ability to help meet the needs

of rural, resource dependent communities; and their performance with

regard to any other indicators deemed to be relevant.

(11) An assessment as to how useful the new, tested authorities

were overall and the team's recommendations for applying the

authorities more broadly.

(12) An assessment as to what other legislative, regulatory, or

administrative changes would have helped make the project more

effective.

[[Page 44689]]

(13) An assessment as to what type of contractual non-compliance

problems occurred, if any, and how expeditiously they were resolved.

(14) A recommendation as to what should be done differently on

another pilot project.

Criteria for the National Advisory Committee. The National Advisory

Committee will be required to address the following nine criteria. Some

of the criteria are the same as those asked at the local level. Where

this is the case, the duplication is deliberate and reflects the belief

that these criteria have relevancy at both levels. The national team

members, like their local counterparts, will be free to address other

criteria that they deem to be relevant; however, it is worth noting

that their ability to do so may be constrained by the nature of the

data compiled locally.

Biophysical Criteria will include: Based on the collective

experience of the pilot projects;

(1) A determination as to whether the stated purposes and needs for

the projects were fulfilled and an explanation for the conclusion.

(2) A determination as to whether the resource management

objectives of the projects were realized and the basis for the

conclusion.

(3) An assessment as to whether the Forest Service was able to do a

better job of ecosystem management by giving a single contractor the

responsibility for a ``bundled group'' of resource work activities

(e.g., timber extraction, watershed restoration, habitat improvement,

and road obliteration) on the project area and an explanation for the

conclusion.

Economic Criteria will include: Based on the collective experience

of the pilots:

(4) A determination as to whether any of the new processes and

procedures that were tested appear to represent effective ways to

create new or enhance existing employment or entrepreneurial

opportunities in local communities.

(5) A determination of what administrative costs were incurred at

the regional and national levels in order to carry out the stewardship

pilots.

Social Criteria will include: Based on the collective experience of

the pilots:

(6) An assessment as to what steps were taken to ensure that

regional and/or national publics were not excluded or placed at a

disadvantage in the collaborative process, and a determination of

whether the steps taken were effective.

(7) A determination as to the potential for stewardship contracting

to improve the quality of life within local resource-dependent

communities (jobs, environmental conditions, economic infrastructure,

etc.).

Administrative Criteria will include: Based on the collective

experience of the pilots:

(8) An assessment as to what difficulties were experienced in

interpreting or implementing the Section 347 authorities.

(9) An assessment as to how the new processes and/or procedures

that were tested compare to the Forest Service's conventional timber

sale or service contract authorities. As appropriate, in making these

determinations, the committee will consider the following performance

variables: attractiveness to potential bidders; fairness to potential

bidders; implications for the Forest Service's ability to maintain

accountability for the treatments being applied and the forest products

being removed; implications for the Forest Service's ability to

implement ecosystem management projects efficiently and effectively;

implications for the Forest Service's ability to successfully manage

small diameter, under-utilized material; ease of administration;

ability to help meet the needs of rural, resource dependent

communities; and any other indicators deemed to be relevant.

Lastly, the National Advisory Committee will make a recommendation

for which of the new authorities that were tested appear to warrant

broader application on a permanent basis.

Dated: August 10, 1999.

Phil Janik,

Acting Chief.

[FR Doc. 99-21247 Filed 8-16-99; 8:45 am]

BILLING CODE 3410-11-M