United States Employment Service; Labor Exchange Performance Measures

DEPARTMENT OF LABOR

Employment and Training Administration

United States Employment Service; Labor Exchange Performance

Measures

AGENCY: Employment and Training Administration.

ACTION: Notice and request for comments.

-----------------------------------------------------------------------

SUMMARY: In response to the requirements of the Government Performance

and Results Act (GPRA) of 1993, the national call for government

programs to be more accountable and results-oriented, the Department of

Labor (DOL) Employment and Training Administration (ETA) launched a

project called the Workforce Development Performance Measures

Initiative (WDPMI) to develop a menu of key performance measures for

use in the workforce development system. Due to the absence of

established key performance indicators for the public labor exchange

program that measures both the self-service and staff-assisted service

options now available, the United States Employment Service (USES)

initiated a project at about the same time to work cooperatively with

States in developing program-specific performance measures. This

Federal-State workgroup also was charged with ensuring that the labor

exchange performance measures it proposed would be compatible with and

complementary to the overall WDPMI performance measures being

developed.

The Labor Exchange Performance Measures workgroup prepared a

discussion draft issues paper entitled ``America's Labor Exchange

Performance Measures'' that identified a list of potential measures.

This paper was shared informally among the State Employment Security

Agencies during a period beginning on September 10, 1997, until October

31, 1997. More than 20 States provided reactions and feedback on the

proposed measures. The Labor Exchange Performance Measures workgroup

considered these comments and in subsequent discussions refined the

potential program measures.

This Federal Register Notice (FRN) provides a description of the

conceptual framework within which public labor exchange services are

delivered, and requests comment from interested parties and USES'

stakeholders on proposed performance measures for labor exchange

services. The Department of Labor also is interested in comments on the

appropriate number of measures, and whether the proposed measures take

into account the full range of services and service options in the

modern labor exchange. In addition, the Department is interested in

learning about other measures that State labor exchange agencies have

found useful for management and continuous improvement purposes.

There currently are two bills being considered by the Congress

(H.R. 1385 and S. 1186) which will provide the framework for the

Nation's workforce development system. Among the provisions of the

bills is a requirement that the Wagner-Peyser funded labor exchange

program functions are provided through the One-Stop system (called

``full-service'' in H.R. 1385).

These proposed measures are a starting point for development of

comprehensive measures for the labor exchange function of the emerging

workforce development system. This FRN does not address the data

elements needed to produce the performance measures nor proposes

specific changes to the ETA reporting requirements. That will be the

subject of a subsequent notice.

DATES: Comments on the proposed Labor Exchange function performance

measures must be received by the U.S. Department of Labor on or before

July 27, 1998. Late-filed comments will be considered to the extent

possible.

ADDRESSES: Comments must be filed in Room N4470, U.S. Department of

Labor, Employment and Training Administration, United States Employment

Service, 200 Constitution Avenue NW., Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT: John R. Beverly, III, United States

Employment Service, U.S. Department of Labor, 200 Constitution Avenue

NW., Room N4470, Washington, DC 20210, Tel. 202-219-5257, Fax 202-219-

6643, E-mail [email protected]

SUPPLEMENTARY INFORMATION:

Contents

I. Authority

II. Introduction

A. Mission of the US Department of Labor

B. Role of the Public Labor Exchange

III. Performance Measures Development Process

A. Labor Exchange Performance Measures Workgroup

B. Models for Service in the Modern Labor Exchange

1. Self-Service strategy

2. Facilitated Self-Help strategy

3. Staff-Assisted strategy

C. Labor Exchange Performance Measurement Strategy

D. Initial Comments on the Draft for Discussion Issues Paper

IV. Definitions

A. Job Search Assistance

B. Facilitated Self-help

C. Self-help Tools

D. Staff Assisted Labor Exchange Services

E. Job Listing Services

F. Referral Related Action

G. Business Assistance Services

H. Entered Employment

I. Entered Employment Rate

J. Cost per Entered Employment

V. Proposed Labor Exchange Performance Measures

A. For Self-Service Strategies

1. Holdings

2. Users

3. Transactions

B. For Facilitated Self-help Strategies

1. Number of Job Seekers Using Self-Help Service

2. Customer Satisfaction

C. For Staff-assisted Service Strategies

Measures of Job Seeker Customer Services

1. Entered Employment Rate

[Option] Job Development Entered Employment Rate

Measures of Employer Customer Services

2. Job Listing Return Business Rate

3. Business Assistance Service Return Business

4. Referral Response Time

5. Average Time Lapse to Successful Referral

6. Job Order Fill Rate

D. System Measures

1. Cost Per Entered Employment

2. Duration of Benefits Compensated

I. Authority

Labor exchange performance measures will be implemented under the

following authority:

A. Wagner-Peyser Act section 10 (c), 29 U.S.C. 49i(c)

(c) Reports

Each State receiving funds under this Act shall (1) make such

reports concerning its operations and expenditures in such form and

containing such information as shall be prescribed by the Secretary,

and (2) establish and maintain a management system in accordance with

guidelines established by the Secretary designed to facilitate the

compilation and analysis of programmatic and financial data necessary

for reporting, monitoring and evaluating purposes.

B. Government Performance and Results Act of 1993

Purposes--The purposes of this Act are to improve the confidence of

the American people in the capability of the Federal Government, by

systematically holding Federal agencies accountable for achieving

program results; --improve Federal program effectiveness and public

accountability by promoting a new focus on results,

[[Page 32565]]

service quality, and customer satisfaction.

II. Introduction

A. Mission of the US Department of Labor

The 1998 USDOL Strategic Performance Plan states that it is the

mission of the Department of Labor to foster and promote the welfare of

the job seekers, wage earners and retirees of the United States by

improving their working conditions, advancing their opportunities for

profitable employment, and protecting their retirement, health, and

other benefits.

In carrying out a part of this mission, the Department, through the

Employment and Training Administration, administers a variety of

Federal programs to help employers find workers, to help job seekers

find employment, to provide unemployment insurance benefits and

reemployment services to jobless workers, to track changes in

employment and to provide job training to meet the changing workplace

skill needs.

B. Role of the Public Labor Exchange in the USDOL Mission

In a May 1, 1933, report to the Senate from the Education and Labor

Committee, and a May 22, 1933, report to the House of Representatives

from the Labor Committee, the stated purpose of the public labor

exchange is:

(1) To foster, promote and develop the welfare of the wage earners

of the United States including juniors (youth) regularly employed;

(2) To improve their working conditions;

(3) To advance their opportunities for profitable employment by

regularly collecting, furnishing, and publishing employment information

as to opportunities for employment;

(4) For maintaining a system for clearing labor between the several

States; and

(5) For cooperating and coordinating the public employment offices

throughout the country.

On June 6, 1933, the Wagner-Peyser Act (29 U.S.C. 49 et seq.)

became law and has since provided the basic funding support for the

public labor exchange system.

Over the 64 years of its history, the role of the public labor

exchange has changed and evolved to meet the needs of the nation,

employers and job seekers. Dedicated and hard working employees have

labored in service to the United States from its earliest role in

assisting the nation to recover from the depths of the Great Depression

as part of the national recovery effort, through the efforts in support

of the nation's economy by assisting private sector employers as their

need for workers was revived. In the years that followed, the public

labor exchange was an important part of the mobilization in support of

war industries during World War II and the return to a post war

economy. During that time the federal/State partnership was

restructured with a redirection of effort to support private sector

employers' workforce needs. Through the 1960's and 70's the national

system of Wagner-Peyser funded labor exchange offices was instrumental

in advancing a variety of social programs designed to expand

participation and conditions of employment for several target groups

including jobless workers, the economically disadvantaged and those who

had particular difficulty in gaining access to employment

opportunities.

In 1986, the Secretary of Labor conducted a series of four public

hearings at which 138 individuals testified and 562 submitted written

testimony representing a cross section of the stakeholders of the

nation's public labor exchange, including business and industry, the

unemployed, target group advocates (e.g. veterans etc.), labor unions,

State governments, employees and the training and education

communities. The intent was to assess the current role of the labor

exchange and to secure input on the appropriate role for the Wagner-

Peyser Act funded labor exchange in the 21st century.

The vast majority of respondents (97.6%) indicated that the

nationwide public labor exchange system was viable, and should be

maintained and improved. Many of the respondents highlighted the need

to streamline the system and to return to providing basic labor

exchange activities. The list which follows represents the most

frequently identified services in their order of priority:

Intake, general assessment, and referral to jobs, training

or re-training opportunities, and/ or support services;

Labor market and occupational information;

Preliminary screening of applicants for employers' job

orders;

Basic skill and/or aptitude testing;

Direct placement (includes job referral);

Job Search Assistance, including training and tips on

interviewing, etc.; and

Specialized recruitment for employers with large staffing

requirements and workforce needs.

The majority of commentators indicated that the public labor

exchange services should be available free to all employers and job

seekers, and that the role of the federal, State and local governments

and the private sector should be maintained as they were at the time of

the review and are today.

In addition to responding to specific questions posed in the FRN

that announced the hearings, the commentators also offered other

recommendations including:

The public image of the labor exchange should be improved

to promote use of the available services,

There be an increased use of automation and technology to

provide linkage among states and localities and related programs to

assist job matching, and to provide more timely and accurate labor

force projections,

DOL develop better standards to evaluate performance and

incentive systems,

Staffing levels be increased,

Data collection and reporting systems be enhanced,

Staff training be increased,

Better job skills assessment tools be developed,

Labor market information be improved,

Federal support for State-level program integration

efforts be increased, and

Alternative funding sources to finance operations be

found.

In the 12 years since the re-examination, the public labor exchange

system has been streamlined. Revitalized operations now provide service

using flexible tiered service delivery strategies. The public labor

exchange has significantly increased the use of automation and

technology to provide service and information, and implemented many of

the changes and improvements which were suggested. In cooperation with

the Unemployment Insurance Service, the public labor exchange also

developed a system of worker profiling and re-employment services for

Unemployment Insurance (UI) claimants to speed their return to

productive employment.

The Wagner-Peyser funded public labor exchange has become a core

component of the new Workforce Development System and is the universal

access component of the nation's One-Stop Career Centers. For many

business and job seeking customers, their only experience and contact

with the Workforce Development System will be Wagner-Peyser funded

services.

The lack of additional financial investment in the public labor

exchange may be traced, to some degree, to its failure to develop

national labor

[[Page 32566]]

exchange performance measures. Performance data that have been

collected and used provides an inaccurate picture of the overall

success and value of the services provided. Moreover, these data fail

to address the employer customer who is taxed to pay for the public

labor exchange services.

In this fiscal environment where labor exchange funding has either

remained the same or declined, a good performance measurement system

will help States and agencies manage programs and decide on the most

appropriate mix of labor exchange services to offer.

For the most part, previous measurement schemes were simple counts

of service outputs, e.g., individuals referred to jobs, with the

effectiveness of those outputs gauged through ratios made by comparing

those counts to the total pool of persons registering with the labor

exchange, e.g., 24% of all labor exchange applicants received a job

referral. Moreover, measurement of the effectiveness of employer

services were confined to a count of the number of job openings

received by the public labor exchange as compared to the number of

openings on which a placement action occurred, i.e., the job openings

filled rate.

The ETA 9002 reporting and data collection system has been the

source of these program data. The authority to collect ETA 9002 data

will expire on August 31, 1999. This sunset date offers an opportunity

to re-examine how the labor exchange function is measured. It is DOL's

hope to secure comment from all of the stakeholders including

employers, wage earners, unions, and partner agencies in the emerging

workforce development system, including the education, training, and

public assistance communities on both the performance measures proposed

herein and the conceptual framework within which those measures would

operate. The comments received will be reviewed and adjustments will be

made to the proposed measures as appropriate. Following this

participatory process, another FRN will be issued to promulgate the key

performance measures, to identify the data elements needed to produce

those measures, and to establish State data reporting requirements.

ETA recognizes its responsibility to continue to collect

information on the race, age, sex, religion, ethnicity, disability

status, and veteran status of the job seekers who register to use its

services.

III. Performance Measurement Development Process

A. Labor Exchange Performance Measures Workgroup

The USES Labor Exchange Performance Measures Initiative (LEPMI)

Workgroup was composed of fifteen representatives drawn from State

employment service agencies, USDOL regional and national offices, the

Veterans' Employment and Training Service, and the Interstate

Conference of Employment Security Agencies (ICESA). The members of the

Labor Exchange Performance Measures Initiative (LEPMI) possessed on

average over 20 years of experience in a variety of program areas and

disciplines including tracking and monitoring program performance.

The workgroup was convened in June 1997 to develop a menu of

conceptual performance measures which could be adopted for the Labor

Exchange function of the Workforce Development System. The Workgroup

used guidelines similar to those used by the ETA Workforce Development

System Performance Measures Initiative Policy Committee to assess

alternative measures, and maintained a close coordination between that

effort and its work. Several persons served on both workgroups. This

close working relationship between the two complementary performance

measures initiatives has continued.

The Workgroup began by agreeing that the new performance

measurement system should be directed toward measuring the outcomes of

service delivery, whenever possible, and not the inputs or outputs of

the processes used to provide service. The first task the LEPMI work

group tackled was to identify the core services being provided in

today's labor exchange. The workgroup identified the following as the

core service categories of the public labor exchange: for Employers the

core services identified were Job Listing, Business Assistance, and Job

Matching and Initial Screening of Candidates for Employment; for Job

Seekers the core services identified were Job Search Assistance, Job

Development, and Facilitating Access to Self-Help Tools.

There are a wide variety of activities which are included within

the broad categories of service, examples of which can be seen in

Figure #1 which follows this FRN.

To develop performance measures, the workgroup benchmarked a

variety of measurement strategies including the way in which the

results of the public labor exchange has been measured in the past,

workload-based measures similar to measures used in UI programs,

measures of the effectiveness of training programs, approaches used to

measure private sector business success and indicators of success in

the public library system.

The workgroup also agreed early in its deliberations to a set of

four guiding principles:

1. The Performance Measures proposed are intended to apply to the

labor exchange function without regard to the administrative or program

structure in which labor exchange services are delivered.

Traditionally, the Labor Exchange function has been within the purview

of State Employment Security Agencies (SESA's) nationwide. With the

emergence of the Electronic Labor Exchange, worker profiling and

intensive job search assistance activities under the Dislocated Worker

programs, Welfare Reform, and One-Stop Career Center service delivery

approaches, the Labor Exchange function is now being performed in a

variety of agency and program models. These measures focus on the

function not the program.

2. The customer universe against which staff-assisted service

outcomes are to be measured would be limited to those persons who have

received staff-assisted services and not all persons registered with

the public labor exchange, whether or not they are seeking work.

There are several customer groups who use the public labor exchange

for purposes other than immediate employment. These include but are not

limited to applicants and recipients of Food Stamps, Home Relief or

State General Assistance, Medicaid, temporary housing, and day care

programs. Many of these programs require registration with the public

employment service as part of the program application process.

Other customers have an intermittent interest in pursuing

employment. These can include applicants who are not in the labor

market and those who are currently employed but interested in new job

opportunities. Many of these individuals will periodically register for

employment with the labor exchange. A segment of this group includes

candidates for employment with prominent local firms that have

developed exclusive recruitment agreements with the public labor

exchange office. Some employers, such as auto industry manufacturing

companies and major retail outlets, post signs outside of their plants

and personnel offices which inform candidates that they hire candidates

for employment who are referred by the

[[Page 32567]]

public labor exchange only, directing the job seekers to complete

applications at a local labor exchange office.

Mass recruitment control services are valued by many employers and

result in registration of thousands of job seekers who are interested

in working for only one of these major area employers. If any of the

members of these customer groups fail to respond to other employer

recruitments or job referral call-ins, and never return to the offices

for job search assistance, they will ultimately be inactivated with no

service. These customers are excluded from consideration when

calculating the outcomes for labor exchange service because they did

not receive and were not interested in staff-assisted service.

3. Although the performance measures presented in this FRN are not

segmented by target group, e.g., Welfare-to-Work, Migrant and Seasonal

Farm Workers, etc., ETA encourages segmenting performance measures to

allow a comparison among groups of customers. Measuring program

performance based simply on overall entered employment outcomes,

without taking into account the labor market characteristics of the

various target groups of job seekers being serviced, may under-

represent the true performance of the public labor exchange system.

Because public labor exchange services are available to all job seekers

and all employers (universal access), its performance outcomes

routinely include hard-to-place customer groups, e.g., potential UI

Exhaustees, Public Assistance Recipients, Migrant/Seasonal Farm

Workers, or Youth. However, current methodology does not account for

the difficulty factors these groups may present. Segmentation of

performance measures may offer insight into these overall outcomes. For

example, an entered employment rate might be developed for Welfare-to-

Work job seekers which could be compared to the entered employment rate

for all job seekers.

ETA believes segmentation of outcomes for specific groups of

customers can provide valuable insights into overall performance

outcomes. This FRN does not address the data elements needed to produce

the segmented outcome measures nor does it propose specific changes to

ETA reporting. These will be the subject of a subsequent notice.

4. Data gathering, to the extent possible, should be directly

related to service delivery and the cost of data collection and

reporting should be proportionate to the cost of the service provided.

A performance measurement system developed in the current environment

that has been shaped by the national effort to reduce the

administrative paperwork burden and to focus on the results of

government programs should not propose measures that cost substantially

more to collect and report than the unit price for delivering the

service.

B. Service Delivery Strategies in the Modern Labor Exchange

To continue to meet customer needs for labor exchange services in a

period of diminishing resources, States operating Employment Security,

One-Stop, and Workforce Development programs have devised a variety of

approaches and service delivery models which are significantly

different from the traditional model of an ``Unemployment'' office that

provided one-on-one staff-intensive interviews and individualized

assistance to job seekers in finding jobs. This model, as the sole

means delivery of labor exchange, is now largely extinct.

Today, labor exchange services are typically provided using a

tiered delivery system composed of three flexible and adaptive service

strategies: ``Self-Service''; ``Facilitated Self-Help''; and ``Staff-

Assisted Service.'' Figure #2, which follows this FRN, shows this

service model. Each strategy is designed to respond to differing

service needs and differing service populations. It is the Department's

expectation that State agencies providing labor exchange service will

use each of the three service delivery strategies.

This FRN represents the first effort to describe the tiered service

delivery continuum, particularly the electronic Self-Service and

Facilitated Self-Help strategies. Accordingly, more descriptive text

has been devoted to the descriptions of the Self-Service and

Facilitated Self-Help service strategies compared to the description of

the traditional Staff-Assisted service strategy. This should not be

interpreted as a preference for one strategy over another. Each has a

necessary and appropriate role in labor exchange service delivery.

1. In a Self-Service strategy, States make labor exchange resources

available which customers can utilize independently, i.e., without

staff intervention. Three current trends have helped drive the

development of self-service strategies in today's public labor

exchange.

First, government is increasingly required to do more with less,

and so must find ways to deliver its products and services more

efficiently.

Second, the labor market itself has changed, with unemployment

being less associated with cyclical pressures and more with structural

causes where increasing numbers of job seekers are considered

dislocated workers. Studies have suggested that in the future job

market most workers will change careers (not just jobs) several times

during their work lives.

Third, customer expectations have changed, and the public labor

exchange system faces new demands from its customers regarding the

quality of the services that are provided. Customers also expect to

participate in the decisions involving what services are provided and

how they are provided.

This delivery mode places the customers in charge of the services

they receive. The acceptance of self-service modes of service delivery

is demonstrated by the increased number and availability of public

access computers, automated teller machines, debit cards, and library

swipe cards with bar-coded data, etc. There has also been a general

movement within the United States toward placing more responsibility

on, and expecting greater participation from, an individual who wishes

to receive governmental services or benefits. In line with these

trends, customers of the public labor exchange increasingly have made

clear their preference for exercising informed choices in determining

which products and services they receive and how they receive them.

An additional and significant benefit of the self-service mode of

service delivery is that self-service expands the capacity of the

system beyond the limits of the available staff resources, thereby

efficiently handling a wide variety of customer labor market and

employment information needs.

2. In the Facilitated Self-Help strategy, customers are provided

access to self-help resources at the One-Stop Career Center or local

labor exchange office, generally through a dedicated Resource Room. For

many customers this is their first exposure to self-service tools and

computer-based systems. In addition, one or more staff are assigned to

assist customers who need help in using those resources. These staff

interact with the customer, as needed, to facilitate the customer's job

search using government-provided resources, e.g., personal computers,

word processing and/or resume writing software, fax and copy machines,

etc., and online access to the DOL-funded Internet-based tools, e.g.,

America's Job Bank (AJB), America's Talent Bank (ATB), America's Career

InfoNet, etc. After being introduced to and assisted to use these

Internet-based

[[Page 32568]]

self-service systems many customers will be able to use them in the

future without assistance both in the One-Stop Center and in other

locations with access to the Internet, e.g., schools, libraries, and at

home. As funding for the traditional labor exchange or employment

service has remained flat or declined, use of this mode of service

delivery has increased. It continues to place the customers in charge

of the services they wish to access, but provides the staff

facilitation needed to ensure that the customers can obtain the service

or information they require. Again, the cost for providing services

using this strategy is less than under a Staff-Assisted service

strategy, since the ratio of staff to customers is much higher.

3. The Staff-Assisted strategy ranges from intensive one-on-one

services where a staff person is assigned to a job seeker as a case

manager, to those where a customer interacts with service staff in a

small group setting. This is the more traditional mode of operation for

delivery of public labor exchange services. The services most

frequently delivered in this manner are assessment, intensive job

search assistance, employment plan development, case management,

counseling and vocational guidance, and job development. The expected

output is a job referral and the expected outcome is entry to

employment. Group services, such as job clubs and workshops on

interviewing techniques, the world of work, and use of labor market

information are also classified as staff-assisted services, although

they are provided on a one-to-many service basis.

C. Discussion of Labor Exchange Performance Measurement Approaches

DOL is proposing a different approach for measuring performance

under each of these service strategies. The Department does this for

two primary reasons. The first reason is that each strategy results in

significantly different relative costs. Costs can be measured per unit

of service provided or as an overall cost for providing access to

resources. The costs to provide a service should be the primary

determinant in deciding how much to invest to measure that service and

how much customer information to collect. The more significant the

service intervention, e.g., the higher the unit cost of that service,

the greater the rationale for collecting data to justify the

expenditure of public funds. For example it would not appear to be

prudent to spend $10.00 pursuing a follow-up action to determine the

outcome of a service that costs less than 25 cents to deliver. However,

one should be willing to expend significant resources in determining

whether a $5000 investment in training results in a long term payoff to

the taxpayer.

The following are examples of labor exchange services and resources

that might routinely be provided in a self-service or facilitated self-

help mode and for which it might be difficult to document an employment

outcome without a substantial data collection and follow up effort.

Labor Market Information.

Self-help pamphlets, magazines, newspapers, and reference

books.

Internet access for use of self-help job finding tools,

e.g., AJB/ATB.

Access to computer hardware and software, fax and copy

machines, telephones, and other equipment in resource centers.

Gaining Internet access to resources and tools such as these from a

job seeker's home or from a community resource, such as a library or

community center, may cost the public labor exchange system only a few

pennies per transaction. First, because economies of scale are achieved

in the development and maintenance of Internet-based systems and

resources. Second, because the cost of the computer equipment, Internet

service provider access, telecommunications software and hardware,

supplies, electric power, and facilities are borne by the user or by

another organization.

This cost per transaction rises slightly when these latter costs

are borne directly by the public employment service, such as when

offered through a resource room. When these self-service resources are

further augmented by making staff resources available to assist and/or

instruct customers in using these tools and resources, the unit cost

again rises.

A similar case might be made for employer services. An employer may

post an unsuppressed (broadcast) job opening directly to AJB over the

Internet instructing interested parties to contact that employer

directly. This is a very low cost self-service option.

Employers may also request assistance from public labor exchange

staff in placing their job openings on America's Job Bank. The cost of

this service can increase further when the employer also asks the Labor

Exchange to screen and refer applicants to that job order. The cost to

provide this level of staff-assisted service may rise to the level

where tracking outcome data is important.

The second reason for pursuing different kinds of performance

measures when using different service delivery strategies is that, for

program outcome measures, one must be able to make a rational argument

that there is a nexus between the service(s) provided and the outcome

measured. For example, it might be difficult to argue that a single

user session of browsing through labor market information in a self-

service mode over the Internet can be directly linked to a person

obtaining a job 30 days later. However, should that same person come

into the local public labor exchange office several times over the

course of a few weeks and attend workshops, obtain help in his or her

job search from the local office staff, and receive one or more job

referrals, such a nexus can be reasonably concluded.

Clearly, finding a way to gauge the results of self-service and

facilitated self-help strategies presents a challenge to the system.

This is particularly true since the Government Performance and Results

Act (GPRA) became law. Under GPRA, government agencies must account to

their investors (Congress and the American people) as to both how their

programs perform and what results they achieved. The challenge is to

find a means to satisfy this requirement where the service costs only a

few pennies per transaction to deliver and the relationship to

traditional program results is difficult to draw. As the level of

service intervention increases, staff involvement grows, and dollar

investment rises, the use of outcome and impact performance measures

may become necessary to justify and explain the expenditure of public

resources.

DOL hopes that it has addressed these concerns in the performance

measures which follow.

1. Measurement Approach for Self-Service Strategies

A small ad-hoc task team was convened to come up with an approach

to measuring the self-service components of the workforce development

system. This team included several persons who were members of both the

Workforce Development Performance Measure Initiative (WDPMI) and the

Labor Exchange Performance Measures Initiative (LEPMI) workgroups. The

team concluded that the following factors should be used to determine

when outcomes measures would be appropriate for gauging the performance

of a particular self-service or facilitated self-help service:

There is a significant value-added provided by the

service, i.e., the service is more than just an information exchange.

[[Page 32569]]

The cost to collect and track outcomes is less than the

cost per unit to provide the service.

The service itself is intended to lead to an employment or

other measurable outcome.

The cost of providing the service rises to a point which

indicates a need to justify the expenditure of public resources.

The time lapse between receiving the service and the

measurement of the outcome is short enough that the service can be

considered to be causative of and/or contributory to that outcome.

The Ad-hoc Task Team used these factors to help determine whether

self-service and facilitated self-help strategies should be included in

outcome measurement calculations. The first group of services

considered were those which could be provided in a completely self-

service mode, requiring no staff assistance. These services included:

accessing and using electronic labor exchange information

services,

completing online self-assessments,

finding information, such as labor market information

(LMI) or community services,

online training, self improvement, and skill enhancement,

and

applying for services and benefits (e.g., using telephones

for original and continued UI claims).

The Ad-hoc Task Team agreed that customers receiving services in

this fashion should not be included in outcome measurements, but rather

services should be measured using output measures.

In general, the Ad-hoc Task Team agreed with the performance

measurement strategy which had been developed by the Labor Exchange

Performance Measures workgroup. Under this strategy, measuring the

value of self-service strategies, such as using DOL and State-developed

Internet-based tools (America's Job Bank, America's Talent Bank, and

America's Career InfoNet) should be analogous to measuring the value of

services measured in public library systems. Using this concept,

performance would be measured by counting the growth over time of:

Holdings: the numbers of jobs and resumes that are

maintained in its inventory;

Usage: the number of times customers use its services; and

Transactions: how many times specific kinds of services

are used, e.g., referral information on job listings are accessed or

downloads of resumes are requested.

Figure #3, which follows this FRN, shows a model of this approach

to measure program performance.

2. Measurement Approach for Facilitated Self-Help Strategies

Under the facilitated self-help service strategy, staff assist

customers to use self-help tools and resources. The services provided

include:

the self-service activities noted above;

access to computer hardware, software, telephones, office

equipment and other physical resources in a Labor Exchange office; and

staff assistance in using any of the above services or

resources.

The approach to measurement of these services would be a

combination of usage and customers' satisfaction. Usage would be

measured as a simple count of the number of persons using resources

rooms. This could be collected using swipe cards, tallies, sign-in

sheets, and/or automated session counts. Customers using self-help

services should not be included in outcome measurement until or unless

they are provided more intensive staff-assisted service. Customer

satisfaction would be used to look at the dimensions of service:

accessibility, quality, timeliness, and security. This could be done by

surveys that would not be reported nationally. These surveys would be

locally developed, controlled, and used for continuous improvement

projects at the local level.

3. Measurement Approach for Staff-Assisted Strategies

Services delivered using a staff-assisted service strategy

represent a value-added service. Measuring the performance of these

value-added services requires addressing their employment-related

outcomes for job seekers and employers. Additionally, measures should

be adopted which can be used to assess the overall effectiveness and

impact of the public labor exchange system.

D. Initial Comments on the Draft for Discussion Issues Paper

The United States Employment Service in cooperation with the

Interstate Conference of Employment Security Agencies (ICESA) released

the issues paper, entitled, ``Draft for Discussion--America's Labor

Exchange Performance Measures,'' in September, 1997. It provided a

review of the rationale and a suggested conceptual framework for

establishing a set of key performance measures of labor exchange

services. Included was a discussion of a variety of alternative

measures and the identification of 10 potential performance measures of

staff-assisted job seeker and employer services. In addition,

performance measures were suggested for the electronic labor exchange

which counts holdings, usage, and transactions. Also included were two

system measures. One was a cost-to-benefit measure that looks at unit

cost per outcome, and the other was a measure of the impact of service

on Unemployment Insurance benefits compensated.

State Employment Security Agency Administrators were asked to

provide written reactions to the suggested measures by October 31,

1997. The ``Draft for Discussion'' issue paper was also made available

on the Internet on the USDOL-supported ICESA Workforce ATM website

(http://www.icesa.org/national/docs/LABOREX.HTML). This approach

provided an opportunity for the public labor exchange system to have

direct input to the initial development of the performance measures

that are the subject of this FRN.

Based on the comments received from States, and subsequent

additional discussion and consideration by the workgroup, the number of

job seeker measures has been reduced and several improvements have been

made to other measures. In some cases, where different approaches were

suggested and there was no clear-cut decision as to which approach was

the better choice, more than one option has been included in this FRN

for comment.

Current system redesign efforts to enhance and integrate the

features of America's Job Bank and America's Talent Bank systems will

make it possible to collect and aggregate some data electronically

without additional State data collection or transmission effort. It is

the Department's intention to actively pursue electronic data

collection approaches to the extent feasible.

IV. Definitions

A. Job Search Assistance (JSA): Includes services currently defined

in the ETA 9002 Data Preparation Handbook NO. 406 under the ``Received

Some Reportable Service'' categories and in items 12 through 42. The

service activities included are: job search workshops, job finding

clubs, resume assistance, providing specific labor market information,

job search plan development, job matching, job development, referral to

jobs, vocational guidance, assessment interviews, testing, vocational

counseling, federal bonding, and referral to other services

[[Page 32570]]

(including skills training, educational services and supportive

services).

B. Facilitated Self-help: Is a staff-assisted service in which

staff facilitates customer access to a variety of in-office self-help

job finding tools and resources. Support staff provides limited

assistance on an as needed basis. Most job seekers require limited

assistance. Normally, job-seeking customers are not required to

register for service before they are allowed access to self-help tools

or resource areas, rooms or centers.

C. Self-help Tools: Include, but are not limited to, resource

rooms, automated labor exchange system devices, Internet access, fax

machines, telephones, photocopiers, personal computers, word

processors, career and labor market information, and reference

materials.

D. Staff Assisted Labor Exchange Services: Include job search

assistance, job development assistance and job referrals, and can be

characterized as service which is conducted one-on-one or in small

groups.

E. Job Listing Services (JLS): Include activities performed on

behalf of employers to assist them in filling their workforce needs.

JLS includes but is not limited to services and activities such as job

order taking, job order maintenance, referral follow-up and interview

support, together with connecting activities including searching job

seeker databases, transmitting resumes, and marketing job orders to the

job seeker (applicant) pool, etc.

F. Referral Related Action: Includes the following services and

activities:

(1) Referral of qualified candidates, or

(2) Contact with the employer to review potential referrals, to

develop a candidate recruitment strategy or adjust features of the job

order which prevent identification of candidates for referral, i.e.,

salary, experience requirements, etc.

G. Business Assistance Service (BAS): Includes, but is not limited

to, providing employer education seminars, job and task analysis,

providing local labor market information, and referral to other

workforce development or economic development services or agencies.

There are a number of employer service activities which are

excluded from the definition of BAS. They are: periodic mass mailings

and routine promotional mailings to increase employer job listings,

providing labor market information via the Internet and employer

contacts for the provision of other core services, e.g., job listing,

job matching, initial screening or referral services.

H. Entered Employment: The unduplicated count of job seekers

(applicants) who enter employment by job placement or obtained

employment. See page II-12 of ETA 9002 Data Preparation Handbook, ET

Handbook NO. 406.

(1) Job Placement: The hiring of a job seeker by a private or

public employer after referral to a job by the Agency staff, or

collocated or out-stationed staff in cooperation with the Agency,

provided that the following conditions were fulfilled:

(a) Prepared a job order prior to referral, except in the case of a

job development contact on behalf of a specific job seeker,

(b) Made prior referral arrangement with the employer,

(c) Referred an individual who was not designated by the employer,

and

(d) Verified from a reliable source that the job seeker had entered

work, and the placement was recorded in the agency data base.

See page II-12 of ETA 9002 Data Preparation Handbook ET Handbook

NO. 406.

(2) Obtained Employment: Individuals who secure employment within

the current quarter or the next completed quarter following the last

staff-assisted job search assistance service that was partially funded

by the Agency, such as:

(a) Participating in job search activities,

(b) Accepting a position resulting from the use of an agency-

sponsored automated labor exchange,

(c) Receiving employment counseling or testing or development of an

employability plan,

(d) Receiving bonding assistance,

(e) Terminating from a skill training program to which a job seeker

was referred by the Agency, or

(f) Receiving tax credit voucher, and receipt of verification from

a reliable source, preferably the employer.

See page II-13 of ETA 9002 Data Preparation Handbook ETA NO. 406.

Entered Employment can be counted in the current quarter or the next

completed quarter following the receipt of the last staff-assisted

service (a maximum of 180 days).

I. Entered Employment Rate (EER): Is the percentage of job-seekers

securing employment after receiving Staff-Assisted Services.

J. Cost per Entered Employment (CPEE): Is the cost of achieving the

positive outcome of entry to employment following the provision of

service. The CPEE is determined by dividing the total of those who

entered employment by the total funding received for the federal

Wagner-Peyser Act, and Veteran Services grants and State appropriations

used for the Labor Exchange function during a program year (July 1-June

30).

V. Proposed Labor Exchange Performance Measures

Specific questions have been developed regarding these performance

measures for which public comment is sought. Questions can be found at

the end of sections A., C., and D. below, and they are numbered

sequentially, 1 through 16. Commentators need not repeat questions as

part of their response, but addressing the specific questions by number

would be most helpful to this effort.

A. For Self-Service Strategies

Performance measures will be output measures based on holdings,

usage and transactions (much like a library) until customer-friendly,

non-invasive sign-on and low cost follow-up approaches are developed to

identify job-seekers in the electronic labor exchange.

In developing these performance measures, the needs and interests

of legislators, policy makers, program managers, budget planners,

analysts, employers and other investors were considered. These output

measures may be used for strategic planning, program management,

continuous improvement and research. This information combined with

customer satisfaction surveys and feedback could assist in the design

of system improvements to meet customer needs.

It is DOL's expectation that national data reporting will be

produced by the AJB/ATB system, and will be aggregated and reported

electronically. It is also DOL's expectation that these performance

measures will not require additional state data collection.

1. Holdings: This measure is a count of the number of employer job

orders and the number of job seeker resumes in the AJB/ATB system.

Continued growth in the number of job orders and resumes in the AJB/ATB

system provides an indication of customer satisfaction and perceived

value by the customers.

2. Usage: This measure is a count of user sessions on the AJB/ATB

system. A user is an individual who accesses the AJB/ATB system for any

purpose. Some users of the system supply personal identification

numbers and passwords; others are anonymous. A user session represents

each single continuous access. This is different than a ``hit'', which

measures the number of Internet server accesses (ie. computer to

computer communications). Under this measure, we will count registered

employers, registered job seekers and anonymous sessions. Enhancements

to

[[Page 32571]]

the AJB/ATB system are underway to provide better information on

anonymous sessions. This output measure will provide a gauge of how

many customers are using these electronic labor exchange services over

some defined period of time and whether the usage is growing, remaining

static or declining.

3. Transactions: This is an output measure of a higher level of

interaction with the AJB/ATB system beyond a review of the information

resources. For example, among the variety of transactions which could

be collected electronically, two stand out. They are Job Seeker

Referral Requests and Employer Resume Downloads.

``Job Seeker Referral Requests'' measures the number of job order

referral instruction screens that are viewed by job seekers on AJB.

This output measure is a count of the number of times job order

referral instruction screens are viewed by job seekers on AJB after

viewing a job order description. This measure identifies the number of

users who have requested specific job referral information and are

likely to respond to the employer's job opportunity. There are two ways

in which this occurs: unsuppressed job orders and suppressed job

orders. In unsuppressed job orders the employer contact information is

broadcast and not hidden from the job seeker. In this case, the job

seeker self refers. In suppressed job orders the employer contact

information is not viewable or accessible through computers and the job

seeker must be referred to the employer by the local labor exchange

office holding the job order.

``Resumes Downloaded'' is a count of job seeker resumes that

employers have selected from a list of job seekers, who met an employer

defined search criteria, to obtain job seeker contact information.

Questions

The Department specifically invites interested persons to provide

comments, data, information and views concerning the following:

Q1. Do these proposed measures adequately represent an employer's

or job seeker's satisfaction with the Electronic Labor Exchange?

Q2. How would you rate the importance of the three types of self-

service outputs (holdings, usage, and transactions)? Which would be

most or least important for strategic planning, program management,

etc.?

Q3. What other ways would you suggest for measurement of self-help

services?

Q4. How do you currently or how would you suggest the national

system collect customer satisfaction data and information?

Q5. Will the proposed Self-Service Electronic Labor Exchange

measures enhance State quality improvement initiatives?

Q6. What specifically defined period of time would be most

appropriate for self-service performance measurement: monthly,

quarterly, or annually?

B. For Facilitated Self-Help Strategies

Two Facilitated Self-help measures are being proposed: the number

of users and customer satisfaction. These measures are intended to

determine the extent to which customers value facilitated self-help

services. The measures address this by answering two questions: do

customers use the service and are customers satisfied with the

services?

1. Number of Users of Self-Help Service

The Number of Users of Self-Help Service is an output measure of

the number of customers who access the self-help resources which

includes the assistance of a knowledgeable staff member assigned to

facilitate customer access to and use of the self-help tools. It is the

intent that facilitated self-help provides service to a large number of

job seekers with a minimum investment of staff time.

The accumulation of a count of users can be as rudimentary as an

office stroke tally or a sign-in log. It would not be necessary to

track which self-help tools are used, although valuable information

needed for the continuous improvement of facilitated self-help programs

might be gained from a periodic assessment of which tools are most

popular or useful to job seekers.

2. Customer Satisfaction

Customer Satisfaction can be measured by conducting periodic formal

telephone or mail surveys or by collecting customer in-person feedback

using a structured approach at specified intervals. The measure could

capture the frequency of use of self-help resources. Customers normally

are not required to register to use the self-help tools in a labor

exchange office or one-stop career center. These measures recognize the

customer's use of and satisfaction with self-help resources.

Facilitated Self-Help is a widely used and valid form of service.

Self-Help service is consistent with the reality of shrinking

resources, and relates directly to customers' demand. This measure can

contribute to continuous improvement with input from the customer.

C. For Staff-Assisted Service Strategies

Measures of Job Seeker Customer Services

Staff-Assisted Labor Exchange Services are the core activities of

the modern Labor Exchange Office. The purpose of these performance

measures is to determine the outcomes, effectiveness, and system impact

of Staff-Assisted job seeker service. Performance will be evaluated

through a combination of interrelated performance measures. These

measures will be useful to legislators, employers, policy makers,

agency administrators and program managers since they provide a tool

for managing programs, baseline data for continuous quality

improvement, and feedback that will allow States to respond to customer

needs. These quantitative measures should be supplemented by

qualitative measures of customer satisfaction and are not intended to

be the sole measures of satisfaction.

1. Entered Employment Rate (EER): The EER is the percentage of job

seekers securing employment after receiving staff-assisted labor

exchange services divided by the total number of job seekers who

received staff-assisted labor exchange services. This measure uses as

its denominator only those customers who receive staff-assisted

service(s) and not the total number of applicants registered in the

labor exchange office database. This performance measure looks at the

effectiveness of the Job Search Assistance services that have

employment as the expected outcome. This measure encourages an

increased level of job order and job seeker follow-up, tracking and

employer feedback. Service and entered employment data would be

collected and reported by staff to the labor exchange reporting system.

Unemployment Insurance wage records can be an additional data source

used to conduct job seeker follow-up on entry to employment.

[Optional Measure] Job Development Entered Employment Rate: Is an

additional and optional measure of Job Search Assistance service. Job

Development (JD) has been an effective tool to help individuals who

have barriers to employment or difficulty in finding employment to find

jobs. With the expected expansion in the role of the Labor Exchange in

Welfare-to-Work programs, it has been suggested that a JD Entered

Employment Rate would be valuable for program management and continuous

improvement.

Job Development Entered Employment Rate is the percentage of job

seekers who entered employment after receiving Job Development referral

[[Page 32572]]

compared to the total number of job seekers who received JD referral.

An outcome measurement of the pro-active solicitation of a job

opportunity for an individual or group, this measure assesses the

effectiveness of job development efforts in contributing to job seeker

entry to employment. Job development is defined as development of a job

opening for a job seeker or group of job seekers through direct contact

with potential employers when no suitable job openings are currently

listed. Job development contacts are currently reported within the

category of ``Received Reportable Service'' in the ETA 9002 report.

This performance measure is intended to create an incentive for

staff to use their considerable knowledge of the labor market and

employers' needs to increase the number and effectiveness of job

development contacts and referrals on behalf of job seekers.

Measures of Employer Customer Services

The variety of staff-assisted services that are made available to

employers can be grouped into three categories: Job Listing Services,

Job Matching, and Business Assistance service.

The purpose of the proposed performance measures is to determine

the extent to which employers' needs are being fulfilled in terms of

the ``Timeliness,'' ``Quality,'' and ``Impact'' of our work. Staff-

assisted employer service performance should be evaluated by this

combination of interrelated performance measures. The individual

measures should not be viewed in isolation and these measures should be

supplemented by qualitative measures of employer satisfaction.

The proposed measures provide an indirect indication of employer

customer satisfaction. They are not intended to be the sole measures of

satisfaction. For example, the Return (Repeat) Business measure may be

a good measure of employer satisfaction at the macro (state or

national) level. However, federal contractors and others who are

required to list their jobs with the Agency may or may not be satisfied

employer customers.

It is recognized that services provided to employers are not

currently reported to the USDOL and may not now be collected by the

labor exchange agency. These employer service measures may require

States to develop new administrative reporting or program record

keeping systems.

2. Job Listing Return Business Rate: The number of employers who

list more than one job order divided by the total number of employers

who use the Staff-Assisted Job Listing Services during a reporting

year. This outcome measure of repeat business serves as a means to

determine employer satisfaction with the job listing service.

The intent of this measure is to provide an incentive for service

delivery staff to improve customer service and relationships including

quick response time, quality referrals, and employer follow-up

contacts. Most of the data needed for the Return Business measure is

readily available with the possible exception that some States do not

capture employer identification numbers on job orders. This measure

would require that employer identification information be collected and

reported for all job orders in the agency reporting and Job Bank

system.

Some States have expressed a preference for measuring job openings

rather than job orders. The focus of this measure is employer

satisfaction, not the volume of job openings or the number of job

orders the labor exchange receives or an employer market penetration

rate. Repeat business is a better indicator of employer satisfaction,

and this measure is used in the private sector as an important

indicator of success in service sector business enterprises.

Some States and labor exchange staff were also concerned with the

effect of employer direct job order entry into AJB and employer access

to resumes on ATB and its impact on the number of job orders they would

receive. In the past the number of job orders secured by a local office

and listed in the State job bank has been a key local office

performance measure in some States. Again, this measure represents the

percent of Return Business which results from the employer's

satisfaction with the Staff-Assisted service. Job orders entered

directly by employers are captured in the electronic labor exchange

measures of AJB and ATB holdings and are not lost to the state. An

unintended consequence of this measure may be a counter-productive

competition between a State's staff-assisted service component and the

State's electronic labor exchange component. From a local office

viewpoint, this measure may motivate staff members to discourage

employers use of the job order self entry service options in AJB to

assure that there is a high rate of repeat job listing business

credited to the staff assisted service component. Enhancements to the

AJB/ATB systems and report generators are currently underway as

mentioned above. This labor exchange system performance issue will be

addressed in future AJB/ATB versions. The Department would appreciate

comments and suggestions for dealing with this potential consequence of

measuring Repeat Employer Business in the staff-assisted service

component of the labor exchange.

Other States commented on the impact of having a large number of

small employers or adverse economic conditions which would reduce the

opportunity for return business during a reporting period as short as a

year. DOL acknowledges the concern and would encourage comments which

are based on a review of State Employment Service, Unemployment

Insurance or Tax administrative data which demonstrate this phenomenon.

DOL would also be interested to learn if this is a result of an agency

policy to target small employers who have limited workforce replacement

or expansion needs.

Finally, even if the suggested defect in this performance measure

is accurate, the Return Business measure would be useful for strategic

planning, performance management and continuous improvement efforts.

This measure is intended to be used by States for national performance

reporting and state program management as opposed to being used for

State-to-State comparisons.

3. Business Assistance Service Return Business: In addition to Job

Listing and Job Matching services, there are a number of other types of

Staff-Assisted labor market information and services which employers

require. These were defined in section IV. G. above. The provision of

Business Assistance Services (BAS) is intended as a means of

stimulating increased employer job listings and employer support for

the Labor Exchange and Workforce Development System. The BAS Return

Business Rate is the number of employers who utilize more than one BAS

service divided by the total number of employers who use at least one

Job Listing, Job Matching or Business Assistance staff-assisted

service. The intent is to measure the level of employer customer

satisfaction with BAS provided and to be an incentive to increase

service to an important customer.

4. Referral Response Time: Is an outcome measure of Job Matching

Service (JMS) effectiveness, which combines the interests of job-

seekers and employers. The Job Order Response Time measure is the

percentage of job orders for which referral-related action took place

within three business days compared to the total number of job orders

listed. This performance measure provides an incentive for quick

response

[[Page 32573]]

to job orders by job-seeker referral or by direct referral-related

contact with employers. Three business days have been identified as the

generally accepted standard for referral and/or follow-up contact with

employers who use the Job Listing Service (JLS).

Some States indicated concerns regarding their ability to track

employer follow-up contacts. It is recognized that current systems may

have to be adjusted to capture this information. Although it is

understood that not all job orders can have a quality referral made

within three days, this measure encourages rapid response to the

employer's need by the provision of other employer services such as,

contact with the employer to review potential referrals, to develop a

candidate recruitment strategy and/or to adjust features of the job

order which prevent identification of candidates for referral, i.e.,

salary, experience requirements, etc.

5. Average Time Lapse to Successful Referral: This outcome measure

will provide a picture of how quickly labor exchange agencies respond

to job orders with referral of a qualified candidate for employment.

The performance measure is the average time lapse (in days) from the

date an employer's job order is listed to the date of referral of the

first agency-referred individual hired by that employer. The underlying

presumption is that when the employer decides to hire one of the labor

exchange referrals, the agency has met the employer's candidate

qualification need.

Not included in this calculation are job orders where the employer

hires from another source. This removes from the equation situations

where, although qualified candidates are referred, the employer decides

to hire an equally qualified candidate from another source.

This measure focuses on the time lapse to the date of job-seeker

referral. An employer's hiring practices which may delay the offer of

employment for many days or weeks does not affect the outcome of this

measure. This measure provides a snapshot of the agency's referral

practices in terms of responsiveness with a quality feature. It is

intended to balance the rapid referral measure and mitigates the

possibility that fast, but poor quality, job-seeker referrals will be

the reaction to a rapid response performance measure.

Job Order Fill Rate: Based on State comment, an additional measure

of the Employer Satisfaction is proposed. The Job Order Fill Rate is

the number of job orders for which a placement is made divided by the

total number of suppressed job orders received within the reporting

year. A job order for which the employer name, address, and other

identification or contact information is hidden from the job seekers'

view is considered to be a suppressed job order.

Employers have the option of listing a job order in a suppressed or

unsuppressed mode depending on the level of initial screening, and/or

referral control service the employer wish the labor exchange to exert.

Frequently employers are guided in the decision to suppress contact

information based on the local availablity of candidates for employment

who meet the skill, background and experience required to be successful

candidates for employment by labor exchange staff.

This outcome measure will provide a picture of how effectively

agencies or States respond to these job listings with qualified

candidates for employment. In this measure, the fact that the employer

hires one of the job seekers referred by the agency is verification

that the candidate was qualified for the job and that the employer was

satisfied with the service provided.

The intent of this measure is to provide an incentive to meet and/

or exceed employer customer expectations by referral of qualified

candidates for employment. Not including in the denominator of this

performance measure job orders which are listed in an unsuppressed or

broadcast fashion avoids the possible unintended consequence that the

labor exchange fill rate performance is measured against job orders

where staff-assisted service is neither needed nor wanted by the

businesses that want to broadcast available employment opportunities.

The required data are currently being collected in many States and are

available.

In some States, a Job Opening Fill Rate performance measure has

been used for many years as the approach to gauge the efficiency of

local Labor Exchange offices and operations. In other States, the Job

Opening Fill Rate has been discontinued because it has proven to be a

disincentive to securing employer job orders in occupational areas

where local management and staff would find it difficult to identify

local and immediately available candidates. By including in the

performance measure denominator only those job orders and openings

which have the employer contact information suppressed, there will be

less of an incentive to withhold job orders from the system or to

artificially reduce the number of job openings. Based on labor market

information, and in consultation with the employer, job orders that are

not likely to be filled by a local candidate can be listed in an

unsuppressed fashion and would therefore not count in the Fill Rate

measure denominator.

Questions

The Department specifically invites interested persons to provide

comments, data, information and views concerning the following:

Q7. Are there other services which your State or agency provides

which have entry to employment as a goal which should be included in a

list of Job Search Assistance services?

Q8. Veteran service and other target group service programs

continue to provide a higher level of one-on-one job-seeker service and

job development assistance. Is the level of the job development

assistance activity in your State or agency significant enough to

require measurement of the Job Development Entered Employment Rate

separately and apart from the Staff-Assisted Entered Employment Rate

(EER)?

Q9. What would be the impact of a requirement to collect or have

the facilities necessary to associate an employer ID number with a job

order?

Q10. Is the proposed three business day standard for measuring

referral-related follow-up contact with employers consistent with your

agency or State approach?

Q11. The Job Order Fill Rate measure is intended to provide an

indication of employer satisfaction with staff-assisted labor exchange

services. Are the unintended consequences of a fill rate performance

measure serious enough to eliminate it from consideration as a program

performance measure?

Q12. Are there other services and/or service delivery approaches

which should be included in the mix of services suggested under the

Business Assistance Service definition?

Q13. Will these proposed Staff-Assisted Labor Exchange service

measures enhance State continuous quality improvement initiatives?

D. System Measures

1. Cost Per Entered Employment (CPEE)

This is an outcome measure of the efficiency of the labor exchange

function. The CPEE is calculated by dividing the total number of

entered employment counts by the total funds allocated to the labor

exchange function. Funding sources would include Wagner-Peyser Act,

Veteran Employment and Training Service (VETS) allocations to States,

and supplemental State funding provided for the labor exchange

function. This measure will provide the context for the public labor

exchange systems to

[[Page 32574]]

compare its key outcome--entered employment--against other workforce

development components. For example, a 40% EER that costs only $300 per

entered employment may seem more favorable when compared to an 80% EER

when each EE costs $5000.

Another intent of the measure is to promote efficiency by providing

management with cost-of-operation information. This information could

be used for strategic planning, allocation and distribution of scarce

resources, as well as continuous quality improvement efforts.

Administrative data sources are available to produce this system

measure, and includes SESA administrative records and labor exchange

job-seeker service records. In many States, Unemployment Insurance wage

record data can be used to improve entered employment information.

Funding data can be captured from the annual grant allocation

documents for Wagner-Peyser and Veteran's Services. The information on

State funding for the labor exchange function would have to be

collected by canvassing the States to determine the level of additional

State support. The cost should be calculated based on the Wagner-Peyser

Program Year basis, which is consistent with other employment and

training program funding cycles. The Veterans' program grants would

have to be prorated since they are allocated based on a Federal Fiscal

Year. A similar proration would be required for State funding which is

not allocated on a program year basis.

The CPEE measure is easy to understand and is generally accepted.

These data are also useful for program management and the resource

allocation process.

2. Duration of Benefits Compensated

Employers are the primary customers of Workforce Development

programs nationwide. To meet employers' workforce needs, public labor

exchange agencies focus on job-seekers who possess a labor market

attachment and marketable skills. In addition, Wagner-Peyser Act

funding is provided to the public labor exchange, in part, to

administer the work test for the State Unemployment Insurance

compensation system and to provide job search and placement services

for claimants. The effect and/or impact of services directed toward UI

benefit customers has many dimensions which can be significantly

affected by movement in the economy and local business conditions.

However, there is a general belief that labor exchange service can have

a direct and measurable impact on the duration of benefits.

Two options are proposed to measure the impact of staff-assisted

service on unemployment insurance benefit customers:

The first option is a measure of duration of benefits compensated

for claimants who receive services compared to prior year(s)''

duration. This measure can be adjusted based on economic conditions.

This measure seeks to determine the impact of staff-assisted labor

exchange services on the duration of benefits for claimants who

received services. Data can be collected from Unemployment Insurance

average duration data records (UI claims first payments/weeks

compensated), labor exchange agency records, and local office data

reporting. USDOL Unemployment Actuarial Unit data on business cycles

and economic conditions could also be useful in explaining the impact

of job-seeker services on claimants. Benchmark data would be available

from previous State reporting.

An easy-to-understand measure of the success of labor exchange

services, this measure can be adjusted to consider State and local

economic conditions. Most of the required data is easily accessible.

This duration-of-benefits model could also be adapted to other groups

such as Welfare-to-Work customers and Veterans, etc., and could be

expressed as a reduction in the number of income transfer payments, as

a trust fund savings, or as a part of a return on investment statement.

The second option is a measure of the impact of Labor Exchange

Services on the Duration of Benefits Compensated for UI Claimants

Required to Search for Work (Work-Test Claimants) who receive Staff-

Assisted Labor Exchange services and enter employment compared to the

average duration-of-benefits compensated to all claimants who are

required to search for work.

This outcome measure also seeks to determine the impact of staff-

assisted labor exchange service on the duration of benefits claimants.

In this option, the group to be studied will consist of UI claimants

who: (1) were not exempt from an active work search; (2) received

staff-assisted labor exchange job-seeker services; and (3) entered

employment.

Staff-Assisted Labor Exchange services for claimant job-seekers

includes job search assistance (JSA) and a variety of facilitated self-

help services enumerated in the definitions above in section IV. A, B,

and C. Since the benefit customer job-seeker's identity is known, they

are usually registered in the labor exchange system.

A claimant who is required to search for work is a job-seeker who,

as a condition of receiving Unemployment Insurance benefits, is

required to perform an active work search. This criterion excludes UI

benefit customers who are on a temporary layoff, required to secure

employment through a union agent, partially unemployed and receive

benefits under an approved partial benefit program, or exempt from work

search requirements due to enrollment in an approved training program.

This performance measure provides an incentive to deliver high

quality JSA and other service to UI job seekers, to begin the provision

of service early in the claim, to aggressively follow-up on job

referrals, to track UI job-seekers' follow-through on work search

plans, and to determine when the benefit claimant became re-employed.

Administrative data are available for development of this measure

of impact from Unemployment Insurance Wage Records (claims filed and

average duration data), ES records (referral and service data elements

in the current ETA 9002 compliant reporting systems and the Employment

Security Systems Institute systems), and local office data reporting.

In addition, federal and State benchmark databases which can be used to

measure the State against its previous accomplishments are available.

The measure can be used to show the effectiveness of service when a

reduction in duration can be shown, and would be useful to legislators,

employers, and policy makers. The impact or reduction in the duration

of benefits performance measure could be:

(a) translated into a reduction in the number of weeks compensated

and a calculation of Unemployment Insurance Trust Fund savings using

the State's average benefit rate, and

(b) adapted for other target groups including Welfare-to-Work

customers, Veterans, etc. In this case the result could be expressed as

a reduction in the number of monthly income transfer payments, as a

dollar savings in benefits using average benefit rate data, or as part

of a return on investment statement.

Questions

The Department specifically invites interested persons to provide

comments, data, information and views concerning the following:

Q14. Can a measure of UI Benefit duration provide a meaningful

measure of the impact of labor exchange services on those claimants who

receive staff-assisted services?

Q15. Would the measurement, and comparison of the rate of UI claim

[[Page 32575]]

exhaustion be a better system measure of the labor exchange staff-

assisted service to UI benefit customers?

Q16. Considering the measures proposed in this FRN, is there an

optimal mix of performance measures which your State or agency would

suggest?

Finally, it should be noted that the Workforce Development

legislation currently pending before the Congress (H.R. 1385 and S.

1186) provides core performance measures that include, in addition to

entered employment, retention in employment six months after entry and

increases in earnings. Under the proposed legislation, these measures

would apply to participants in the Workforce Development activities

provided through the One-stop System, including labor exchange

activities.

We would appreciate receiving views on the appropriateness of these

additional measures for measuring public labor exchange programs.

Paperwork Reduction Act

The notice issued here is not subject to the requirements of the

Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.) Because it

contains no ``collection of information'' as defined in 44 U.S.C.

3502(3).

Signed at Washington, D.C., this 8th day of June 1998.

Raymond J. Uhalde,

Acting Assistant Secretary for Employment and Training.

BILLING CODE 4510-30-P

[[Page 32576]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.016

[[Page 32577]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.017

[[Page 32578]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.018

[FR Doc. 98-15754 Filed 6-11-98; 8:45 am]

BILLING CODE 4510-30-C