# United States Employment Service; Labor Exchange Performance Measures

> Briefs, arguments, decisions, and more.

URL: https://www.frixlaw.com/law-library/documents/fr%3A98-15754

## Record

- **Collection:** Federal Register
- **Document type:** Notice
- **Published:** June 12, 1998
- **Citation:** 63 FR 32564

## Text

DEPARTMENT OF LABOR

Employment and Training Administration

United States Employment Service; Labor Exchange Performance
Measures

AGENCY: Employment and Training Administration.

ACTION: Notice and request for comments.

-----------------------------------------------------------------------

SUMMARY: In response to the requirements of the Government Performance
and Results Act (GPRA) of 1993, the national call for government
programs to be more accountable and results-oriented, the Department of
Labor (DOL) Employment and Training Administration (ETA) launched a
project called the Workforce Development Performance Measures
Initiative (WDPMI) to develop a menu of key performance measures for
use in the workforce development system. Due to the absence of
established key performance indicators for the public labor exchange
program that measures both the self-service and staff-assisted service
options now available, the United States Employment Service (USES)
initiated a project at about the same time to work cooperatively with
States in developing program-specific performance measures. This
Federal-State workgroup also was charged with ensuring that the labor
exchange performance measures it proposed would be compatible with and
complementary to the overall WDPMI performance measures being
developed.
The Labor Exchange Performance Measures workgroup prepared a
discussion draft issues paper entitled ``America's Labor Exchange
Performance Measures'' that identified a list of potential measures.
This paper was shared informally among the State Employment Security
Agencies during a period beginning on September 10, 1997, until October
31, 1997. More than 20 States provided reactions and feedback on the
proposed measures. The Labor Exchange Performance Measures workgroup
considered these comments and in subsequent discussions refined the
potential program measures.
This Federal Register Notice (FRN) provides a description of the
conceptual framework within which public labor exchange services are
delivered, and requests comment from interested parties and USES'
stakeholders on proposed performance measures for labor exchange
services. The Department of Labor also is interested in comments on the
appropriate number of measures, and whether the proposed measures take
into account the full range of services and service options in the
modern labor exchange. In addition, the Department is interested in
learning about other measures that State labor exchange agencies have
found useful for management and continuous improvement purposes.
There currently are two bills being considered by the Congress
(H.R. 1385 and S. 1186) which will provide the framework for the
Nation's workforce development system. Among the provisions of the
bills is a requirement that the Wagner-Peyser funded labor exchange
program functions are provided through the One-Stop system (called
``full-service'' in H.R. 1385).
These proposed measures are a starting point for development of
comprehensive measures for the labor exchange function of the emerging
workforce development system. This FRN does not address the data
elements needed to produce the performance measures nor proposes
specific changes to the ETA reporting requirements. That will be the
subject of a subsequent notice.

DATES: Comments on the proposed Labor Exchange function performance
measures must be received by the U.S. Department of Labor on or before
July 27, 1998. Late-filed comments will be considered to the extent
possible.

ADDRESSES: Comments must be filed in Room N4470, U.S. Department of
Labor, Employment and Training Administration, United States Employment
Service, 200 Constitution Avenue NW., Washington, DC 20210.

FOR FURTHER INFORMATION CONTACT: John R. Beverly, III, United States
Employment Service, U.S. Department of Labor, 200 Constitution Avenue
NW., Room N4470, Washington, DC 20210, Tel. 202-219-5257, Fax 202-219-
6643, E-mail [email protected]

SUPPLEMENTARY INFORMATION:

Contents

I. Authority

II. Introduction

A. Mission of the US Department of Labor
B. Role of the Public Labor Exchange

III. Performance Measures Development Process

A. Labor Exchange Performance Measures Workgroup
B. Models for Service in the Modern Labor Exchange
1. Self-Service strategy
2. Facilitated Self-Help strategy
3. Staff-Assisted strategy
C. Labor Exchange Performance Measurement Strategy
D. Initial Comments on the Draft for Discussion Issues Paper

IV. Definitions

A. Job Search Assistance
B. Facilitated Self-help
C. Self-help Tools
D. Staff Assisted Labor Exchange Services
E. Job Listing Services
F. Referral Related Action
G. Business Assistance Services
H. Entered Employment
I. Entered Employment Rate
J. Cost per Entered Employment

V. Proposed Labor Exchange Performance Measures

A. For Self-Service Strategies
1. Holdings
2. Users
3. Transactions
B. For Facilitated Self-help Strategies
1. Number of Job Seekers Using Self-Help Service
2. Customer Satisfaction
C. For Staff-assisted Service Strategies
Measures of Job Seeker Customer Services
1. Entered Employment Rate
[Option] Job Development Entered Employment Rate
Measures of Employer Customer Services
2. Job Listing Return Business Rate
3. Business Assistance Service Return Business
4. Referral Response Time
5. Average Time Lapse to Successful Referral
6. Job Order Fill Rate
D. System Measures
1. Cost Per Entered Employment
2. Duration of Benefits Compensated

I. Authority

Labor exchange performance measures will be implemented under the
following authority:

A. Wagner-Peyser Act section 10 (c), 29 U.S.C. 49i(c)

(c) Reports
Each State receiving funds under this Act shall (1) make such
reports concerning its operations and expenditures in such form and
containing such information as shall be prescribed by the Secretary,
and (2) establish and maintain a management system in accordance with
guidelines established by the Secretary designed to facilitate the
compilation and analysis of programmatic and financial data necessary
for reporting, monitoring and evaluating purposes.

B. Government Performance and Results Act of 1993

Purposes--The purposes of this Act are to improve the confidence of
the American people in the capability of the Federal Government, by
systematically holding Federal agencies accountable for achieving
program results; --improve Federal program effectiveness and public
accountability by promoting a new focus on results,

[[Page 32565]]

service quality, and customer satisfaction.

II. Introduction

A. Mission of the US Department of Labor

The 1998 USDOL Strategic Performance Plan states that it is the
mission of the Department of Labor to foster and promote the welfare of
the job seekers, wage earners and retirees of the United States by
improving their working conditions, advancing their opportunities for
profitable employment, and protecting their retirement, health, and
other benefits.
In carrying out a part of this mission, the Department, through the
Employment and Training Administration, administers a variety of
Federal programs to help employers find workers, to help job seekers
find employment, to provide unemployment insurance benefits and
reemployment services to jobless workers, to track changes in
employment and to provide job training to meet the changing workplace
skill needs.

B. Role of the Public Labor Exchange in the USDOL Mission

In a May 1, 1933, report to the Senate from the Education and Labor
Committee, and a May 22, 1933, report to the House of Representatives
from the Labor Committee, the stated purpose of the public labor
exchange is:
(1) To foster, promote and develop the welfare of the wage earners
of the United States including juniors (youth) regularly employed;
(2) To improve their working conditions;
(3) To advance their opportunities for profitable employment by
regularly collecting, furnishing, and publishing employment information
as to opportunities for employment;
(4) For maintaining a system for clearing labor between the several
States; and
(5) For cooperating and coordinating the public employment offices
throughout the country.
On June 6, 1933, the Wagner-Peyser Act (29 U.S.C. 49 et seq.)
became law and has since provided the basic funding support for the
public labor exchange system.
Over the 64 years of its history, the role of the public labor
exchange has changed and evolved to meet the needs of the nation,
employers and job seekers. Dedicated and hard working employees have
labored in service to the United States from its earliest role in
assisting the nation to recover from the depths of the Great Depression
as part of the national recovery effort, through the efforts in support
of the nation's economy by assisting private sector employers as their
need for workers was revived. In the years that followed, the public
labor exchange was an important part of the mobilization in support of
war industries during World War II and the return to a post war
economy. During that time the federal/State partnership was
restructured with a redirection of effort to support private sector
employers' workforce needs. Through the 1960's and 70's the national
system of Wagner-Peyser funded labor exchange offices was instrumental
in advancing a variety of social programs designed to expand
participation and conditions of employment for several target groups
including jobless workers, the economically disadvantaged and those who
had particular difficulty in gaining access to employment
opportunities.
In 1986, the Secretary of Labor conducted a series of four public
hearings at which 138 individuals testified and 562 submitted written
testimony representing a cross section of the stakeholders of the
nation's public labor exchange, including business and industry, the
unemployed, target group advocates (e.g. veterans etc.), labor unions,
State governments, employees and the training and education
communities. The intent was to assess the current role of the labor
exchange and to secure input on the appropriate role for the Wagner-
Peyser Act funded labor exchange in the 21st century.
The vast majority of respondents (97.6%) indicated that the
nationwide public labor exchange system was viable, and should be
maintained and improved. Many of the respondents highlighted the need
to streamline the system and to return to providing basic labor
exchange activities. The list which follows represents the most
frequently identified services in their order of priority:
Intake, general assessment, and referral to jobs, training
or re-training opportunities, and/ or support services;
Labor market and occupational information;
Preliminary screening of applicants for employers' job
orders;
Basic skill and/or aptitude testing;
Direct placement (includes job referral);
Job Search Assistance, including training and tips on
interviewing, etc.; and
Specialized recruitment for employers with large staffing
requirements and workforce needs.
The majority of commentators indicated that the public labor
exchange services should be available free to all employers and job
seekers, and that the role of the federal, State and local governments
and the private sector should be maintained as they were at the time of
the review and are today.
In addition to responding to specific questions posed in the FRN
that announced the hearings, the commentators also offered other
recommendations including:
The public image of the labor exchange should be improved
to promote use of the available services,
There be an increased use of automation and technology to
provide linkage among states and localities and related programs to
assist job matching, and to provide more timely and accurate labor
force projections,
DOL develop better standards to evaluate performance and
incentive systems,
Staffing levels be increased,
Data collection and reporting systems be enhanced,
Staff training be increased,
Better job skills assessment tools be developed,
Labor market information be improved,
Federal support for State-level program integration
efforts be increased, and
Alternative funding sources to finance operations be
found.
In the 12 years since the re-examination, the public labor exchange
system has been streamlined. Revitalized operations now provide service
using flexible tiered service delivery strategies. The public labor
exchange has significantly increased the use of automation and
technology to provide service and information, and implemented many of
the changes and improvements which were suggested. In cooperation with
the Unemployment Insurance Service, the public labor exchange also
developed a system of worker profiling and re-employment services for
Unemployment Insurance (UI) claimants to speed their return to
productive employment.
The Wagner-Peyser funded public labor exchange has become a core
component of the new Workforce Development System and is the universal
access component of the nation's One-Stop Career Centers. For many
business and job seeking customers, their only experience and contact
with the Workforce Development System will be Wagner-Peyser funded
services.
The lack of additional financial investment in the public labor
exchange may be traced, to some degree, to its failure to develop
national labor

[[Page 32566]]

exchange performance measures. Performance data that have been
collected and used provides an inaccurate picture of the overall
success and value of the services provided. Moreover, these data fail
to address the employer customer who is taxed to pay for the public
labor exchange services.
In this fiscal environment where labor exchange funding has either
remained the same or declined, a good performance measurement system
will help States and agencies manage programs and decide on the most
appropriate mix of labor exchange services to offer.
For the most part, previous measurement schemes were simple counts
of service outputs, e.g., individuals referred to jobs, with the
effectiveness of those outputs gauged through ratios made by comparing
those counts to the total pool of persons registering with the labor
exchange, e.g., 24% of all labor exchange applicants received a job
referral. Moreover, measurement of the effectiveness of employer
services were confined to a count of the number of job openings
received by the public labor exchange as compared to the number of
openings on which a placement action occurred, i.e., the job openings
filled rate.
The ETA 9002 reporting and data collection system has been the
source of these program data. The authority to collect ETA 9002 data
will expire on August 31, 1999. This sunset date offers an opportunity
to re-examine how the labor exchange function is measured. It is DOL's
hope to secure comment from all of the stakeholders including
employers, wage earners, unions, and partner agencies in the emerging
workforce development system, including the education, training, and
public assistance communities on both the performance measures proposed
herein and the conceptual framework within which those measures would
operate. The comments received will be reviewed and adjustments will be
made to the proposed measures as appropriate. Following this
participatory process, another FRN will be issued to promulgate the key
performance measures, to identify the data elements needed to produce
those measures, and to establish State data reporting requirements.
ETA recognizes its responsibility to continue to collect
information on the race, age, sex, religion, ethnicity, disability
status, and veteran status of the job seekers who register to use its
services.

III. Performance Measurement Development Process

A. Labor Exchange Performance Measures Workgroup

The USES Labor Exchange Performance Measures Initiative (LEPMI)
Workgroup was composed of fifteen representatives drawn from State
employment service agencies, USDOL regional and national offices, the
Veterans' Employment and Training Service, and the Interstate
Conference of Employment Security Agencies (ICESA). The members of the
Labor Exchange Performance Measures Initiative (LEPMI) possessed on
average over 20 years of experience in a variety of program areas and
disciplines including tracking and monitoring program performance.
The workgroup was convened in June 1997 to develop a menu of
conceptual performance measures which could be adopted for the Labor
Exchange function of the Workforce Development System. The Workgroup
used guidelines similar to those used by the ETA Workforce Development
System Performance Measures Initiative Policy Committee to assess
alternative measures, and maintained a close coordination between that
effort and its work. Several persons served on both workgroups. This
close working relationship between the two complementary performance
measures initiatives has continued.
The Workgroup began by agreeing that the new performance
measurement system should be directed toward measuring the outcomes of
service delivery, whenever possible, and not the inputs or outputs of
the processes used to provide service. The first task the LEPMI work
group tackled was to identify the core services being provided in
today's labor exchange. The workgroup identified the following as the
core service categories of the public labor exchange: for Employers the
core services identified were Job Listing, Business Assistance, and Job
Matching and Initial Screening of Candidates for Employment; for Job
Seekers the core services identified were Job Search Assistance, Job
Development, and Facilitating Access to Self-Help Tools.
There are a wide variety of activities which are included within
the broad categories of service, examples of which can be seen in
Figure #1 which follows this FRN.
To develop performance measures, the workgroup benchmarked a
variety of measurement strategies including the way in which the
results of the public labor exchange has been measured in the past,
workload-based measures similar to measures used in UI programs,
measures of the effectiveness of training programs, approaches used to
measure private sector business success and indicators of success in
the public library system.
The workgroup also agreed early in its deliberations to a set of
four guiding principles:
1. The Performance Measures proposed are intended to apply to the
labor exchange function without regard to the administrative or program
structure in which labor exchange services are delivered.
Traditionally, the Labor Exchange function has been within the purview
of State Employment Security Agencies (SESA's) nationwide. With the
emergence of the Electronic Labor Exchange, worker profiling and
intensive job search assistance activities under the Dislocated Worker
programs, Welfare Reform, and One-Stop Career Center service delivery
approaches, the Labor Exchange function is now being performed in a
variety of agency and program models. These measures focus on the
function not the program.
2. The customer universe against which staff-assisted service
outcomes are to be measured would be limited to those persons who have
received staff-assisted services and not all persons registered with
the public labor exchange, whether or not they are seeking work.
There are several customer groups who use the public labor exchange
for purposes other than immediate employment. These include but are not
limited to applicants and recipients of Food Stamps, Home Relief or
State General Assistance, Medicaid, temporary housing, and day care
programs. Many of these programs require registration with the public
employment service as part of the program application process.
Other customers have an intermittent interest in pursuing
employment. These can include applicants who are not in the labor
market and those who are currently employed but interested in new job
opportunities. Many of these individuals will periodically register for
employment with the labor exchange. A segment of this group includes
candidates for employment with prominent local firms that have
developed exclusive recruitment agreements with the public labor
exchange office. Some employers, such as auto industry manufacturing
companies and major retail outlets, post signs outside of their plants
and personnel offices which inform candidates that they hire candidates
for employment who are referred by the

[[Page 32567]]

public labor exchange only, directing the job seekers to complete
applications at a local labor exchange office.
Mass recruitment control services are valued by many employers and
result in registration of thousands of job seekers who are interested
in working for only one of these major area employers. If any of the
members of these customer groups fail to respond to other employer
recruitments or job referral call-ins, and never return to the offices
for job search assistance, they will ultimately be inactivated with no
service. These customers are excluded from consideration when
calculating the outcomes for labor exchange service because they did
not receive and were not interested in staff-assisted service.
3. Although the performance measures presented in this FRN are not
segmented by target group, e.g., Welfare-to-Work, Migrant and Seasonal
Farm Workers, etc., ETA encourages segmenting performance measures to
allow a comparison among groups of customers. Measuring program
performance based simply on overall entered employment outcomes,
without taking into account the labor market characteristics of the
various target groups of job seekers being serviced, may under-
represent the true performance of the public labor exchange system.
Because public labor exchange services are available to all job seekers
and all employers (universal access), its performance outcomes
routinely include hard-to-place customer groups, e.g., potential UI
Exhaustees, Public Assistance Recipients, Migrant/Seasonal Farm
Workers, or Youth. However, current methodology does not account for
the difficulty factors these groups may present. Segmentation of
performance measures may offer insight into these overall outcomes. For
example, an entered employment rate might be developed for Welfare-to-
Work job seekers which could be compared to the entered employment rate
for all job seekers.
ETA believes segmentation of outcomes for specific groups of
customers can provide valuable insights into overall performance
outcomes. This FRN does not address the data elements needed to produce
the segmented outcome measures nor does it propose specific changes to
ETA reporting. These will be the subject of a subsequent notice.
4. Data gathering, to the extent possible, should be directly
related to service delivery and the cost of data collection and
reporting should be proportionate to the cost of the service provided.
A performance measurement system developed in the current environment
that has been shaped by the national effort to reduce the
administrative paperwork burden and to focus on the results of
government programs should not propose measures that cost substantially
more to collect and report than the unit price for delivering the
service.

B. Service Delivery Strategies in the Modern Labor Exchange

To continue to meet customer needs for labor exchange services in a
period of diminishing resources, States operating Employment Security,
One-Stop, and Workforce Development programs have devised a variety of
approaches and service delivery models which are significantly
different from the traditional model of an ``Unemployment'' office that
provided one-on-one staff-intensive interviews and individualized
assistance to job seekers in finding jobs. This model, as the sole
means delivery of labor exchange, is now largely extinct.
Today, labor exchange services are typically provided using a
tiered delivery system composed of three flexible and adaptive service
strategies: ``Self-Service''; ``Facilitated Self-Help''; and ``Staff-
Assisted Service.'' Figure #2, which follows this FRN, shows this
service model. Each strategy is designed to respond to differing
service needs and differing service populations. It is the Department's
expectation that State agencies providing labor exchange service will
use each of the three service delivery strategies.
This FRN represents the first effort to describe the tiered service
delivery continuum, particularly the electronic Self-Service and
Facilitated Self-Help strategies. Accordingly, more descriptive text
has been devoted to the descriptions of the Self-Service and
Facilitated Self-Help service strategies compared to the description of
the traditional Staff-Assisted service strategy. This should not be
interpreted as a preference for one strategy over another. Each has a
necessary and appropriate role in labor exchange service delivery.
1. In a Self-Service strategy, States make labor exchange resources
available which customers can utilize independently, i.e., without
staff intervention. Three current trends have helped drive the
development of self-service strategies in today's public labor
exchange.
First, government is increasingly required to do more with less,
and so must find ways to deliver its products and services more
efficiently.
Second, the labor market itself has changed, with unemployment
being less associated with cyclical pressures and more with structural
causes where increasing numbers of job seekers are considered
dislocated workers. Studies have suggested that in the future job
market most workers will change careers (not just jobs) several times
during their work lives.
Third, customer expectations have changed, and the public labor
exchange system faces new demands from its customers regarding the
quality of the services that are provided. Customers also expect to
participate in the decisions involving what services are provided and
how they are provided.
This delivery mode places the customers in charge of the services
they receive. The acceptance of self-service modes of service delivery
is demonstrated by the increased number and availability of public
access computers, automated teller machines, debit cards, and library
swipe cards with bar-coded data, etc. There has also been a general
movement within the United States toward placing more responsibility
on, and expecting greater participation from, an individual who wishes
to receive governmental services or benefits. In line with these
trends, customers of the public labor exchange increasingly have made
clear their preference for exercising informed choices in determining
which products and services they receive and how they receive them.
An additional and significant benefit of the self-service mode of
service delivery is that self-service expands the capacity of the
system beyond the limits of the available staff resources, thereby
efficiently handling a wide variety of customer labor market and
employment information needs.
2. In the Facilitated Self-Help strategy, customers are provided
access to self-help resources at the One-Stop Career Center or local
labor exchange office, generally through a dedicated Resource Room. For
many customers this is their first exposure to self-service tools and
computer-based systems. In addition, one or more staff are assigned to
assist customers who need help in using those resources. These staff
interact with the customer, as needed, to facilitate the customer's job
search using government-provided resources, e.g., personal computers,
word processing and/or resume writing software, fax and copy machines,
etc., and online access to the DOL-funded Internet-based tools, e.g.,
America's Job Bank (AJB), America's Talent Bank (ATB), America's Career
InfoNet, etc. After being introduced to and assisted to use these
Internet-based

[[Page 32568]]

self-service systems many customers will be able to use them in the
future without assistance both in the One-Stop Center and in other
locations with access to the Internet, e.g., schools, libraries, and at
home. As funding for the traditional labor exchange or employment
service has remained flat or declined, use of this mode of service
delivery has increased. It continues to place the customers in charge
of the services they wish to access, but provides the staff
facilitation needed to ensure that the customers can obtain the service
or information they require. Again, the cost for providing services
using this strategy is less than under a Staff-Assisted service
strategy, since the ratio of staff to customers is much higher.
3. The Staff-Assisted strategy ranges from intensive one-on-one
services where a staff person is assigned to a job seeker as a case
manager, to those where a customer interacts with service staff in a
small group setting. This is the more traditional mode of operation for
delivery of public labor exchange services. The services most
frequently delivered in this manner are assessment, intensive job
search assistance, employment plan development, case management,
counseling and vocational guidance, and job development. The expected
output is a job referral and the expected outcome is entry to
employment. Group services, such as job clubs and workshops on
interviewing techniques, the world of work, and use of labor market
information are also classified as staff-assisted services, although
they are provided on a one-to-many service basis.

C. Discussion of Labor Exchange Performance Measurement Approaches

DOL is proposing a different approach for measuring performance
under each of these service strategies. The Department does this for
two primary reasons. The first reason is that each strategy results in
significantly different relative costs. Costs can be measured per unit
of service provided or as an overall cost for providing access to
resources. The costs to provide a service should be the primary
determinant in deciding how much to invest to measure that service and
how much customer information to collect. The more significant the
service intervention, e.g., the higher the unit cost of that service,
the greater the rationale for collecting data to justify the
expenditure of public funds. For example it would not appear to be
prudent to spend $10.00 pursuing a follow-up action to determine the
outcome of a service that costs less than 25 cents to deliver. However,
one should be willing to expend significant resources in determining
whether a $5000 investment in training results in a long term payoff to
the taxpayer.
The following are examples of labor exchange services and resources
that might routinely be provided in a self-service or facilitated self-
help mode and for which it might be difficult to document an employment
outcome without a substantial data collection and follow up effort.
Labor Market Information.
Self-help pamphlets, magazines, newspapers, and reference
books.
Internet access for use of self-help job finding tools,
e.g., AJB/ATB.
Access to computer hardware and software, fax and copy
machines, telephones, and other equipment in resource centers.
Gaining Internet access to resources and tools such as these from a
job seeker's home or from a community resource, such as a library or
community center, may cost the public labor exchange system only a few
pennies per transaction. First, because economies of scale are achieved
in the development and maintenance of Internet-based systems and
resources. Second, because the cost of the computer equipment, Internet
service provider access, telecommunications software and hardware,
supplies, electric power, and facilities are borne by the user or by
another organization.
This cost per transaction rises slightly when these latter costs
are borne directly by the public employment service, such as when
offered through a resource room. When these self-service resources are
further augmented by making staff resources available to assist and/or
instruct customers in using these tools and resources, the unit cost
again rises.
A similar case might be made for employer services. An employer may
post an unsuppressed (broadcast) job opening directly to AJB over the
Internet instructing interested parties to contact that employer
directly. This is a very low cost self-service option.
Employers may also request assistance from public labor exchange
staff in placing their job openings on America's Job Bank. The cost of
this service can increase further when the employer also asks the Labor
Exchange to screen and refer applicants to that job order. The cost to
provide this level of staff-assisted service may rise to the level
where tracking outcome data is important.
The second reason for pursuing different kinds of performance
measures when using different service delivery strategies is that, for
program outcome measures, one must be able to make a rational argument
that there is a nexus between the service(s) provided and the outcome
measured. For example, it might be difficult to argue that a single
user session of browsing through labor market information in a self-
service mode over the Internet can be directly linked to a person
obtaining a job 30 days later. However, should that same person come
into the local public labor exchange office several times over the
course of a few weeks and attend workshops, obtain help in his or her
job search from the local office staff, and receive one or more job
referrals, such a nexus can be reasonably concluded.
Clearly, finding a way to gauge the results of self-service and
facilitated self-help strategies presents a challenge to the system.
This is particularly true since the Government Performance and Results
Act (GPRA) became law. Under GPRA, government agencies must account to
their investors (Congress and the American people) as to both how their
programs perform and what results they achieved. The challenge is to
find a means to satisfy this requirement where the service costs only a
few pennies per transaction to deliver and the relationship to
traditional program results is difficult to draw. As the level of
service intervention increases, staff involvement grows, and dollar
investment rises, the use of outcome and impact performance measures
may become necessary to justify and explain the expenditure of public
resources.
DOL hopes that it has addressed these concerns in the performance
measures which follow.
1. Measurement Approach for Self-Service Strategies
A small ad-hoc task team was convened to come up with an approach
to measuring the self-service components of the workforce development
system. This team included several persons who were members of both the
Workforce Development Performance Measure Initiative (WDPMI) and the
Labor Exchange Performance Measures Initiative (LEPMI) workgroups. The
team concluded that the following factors should be used to determine
when outcomes measures would be appropriate for gauging the performance
of a particular self-service or facilitated self-help service:
There is a significant value-added provided by the
service, i.e., the service is more than just an information exchange.

[[Page 32569]]

The cost to collect and track outcomes is less than the
cost per unit to provide the service.
The service itself is intended to lead to an employment or
other measurable outcome.
The cost of providing the service rises to a point which
indicates a need to justify the expenditure of public resources.
The time lapse between receiving the service and the
measurement of the outcome is short enough that the service can be
considered to be causative of and/or contributory to that outcome.
The Ad-hoc Task Team used these factors to help determine whether
self-service and facilitated self-help strategies should be included in
outcome measurement calculations. The first group of services
considered were those which could be provided in a completely self-
service mode, requiring no staff assistance. These services included:
accessing and using electronic labor exchange information
services,
completing online self-assessments,
finding information, such as labor market information
(LMI) or community services,
online training, self improvement, and skill enhancement,
and
applying for services and benefits (e.g., using telephones
for original and continued UI claims).
The Ad-hoc Task Team agreed that customers receiving services in
this fashion should not be included in outcome measurements, but rather
services should be measured using output measures.
In general, the Ad-hoc Task Team agreed with the performance
measurement strategy which had been developed by the Labor Exchange
Performance Measures workgroup. Under this strategy, measuring the
value of self-service strategies, such as using DOL and State-developed
Internet-based tools (America's Job Bank, America's Talent Bank, and
America's Career InfoNet) should be analogous to measuring the value of
services measured in public library systems. Using this concept,
performance would be measured by counting the growth over time of:
Holdings: the numbers of jobs and resumes that are
maintained in its inventory;
Usage: the number of times customers use its services; and
Transactions: how many times specific kinds of services
are used, e.g., referral information on job listings are accessed or
downloads of resumes are requested.
Figure #3, which follows this FRN, shows a model of this approach
to measure program performance.
2. Measurement Approach for Facilitated Self-Help Strategies
Under the facilitated self-help service strategy, staff assist
customers to use self-help tools and resources. The services provided
include:
the self-service activities noted above;
access to computer hardware, software, telephones, office
equipment and other physical resources in a Labor Exchange office; and
staff assistance in using any of the above services or
resources.
The approach to measurement of these services would be a
combination of usage and customers' satisfaction. Usage would be
measured as a simple count of the number of persons using resources
rooms. This could be collected using swipe cards, tallies, sign-in
sheets, and/or automated session counts. Customers using self-help
services should not be included in outcome measurement until or unless
they are provided more intensive staff-assisted service. Customer
satisfaction would be used to look at the dimensions of service:
accessibility, quality, timeliness, and security. This could be done by
surveys that would not be reported nationally. These surveys would be
locally developed, controlled, and used for continuous improvement
projects at the local level.
3. Measurement Approach for Staff-Assisted Strategies
Services delivered using a staff-assisted service strategy
represent a value-added service. Measuring the performance of these
value-added services requires addressing their employment-related
outcomes for job seekers and employers. Additionally, measures should
be adopted which can be used to assess the overall effectiveness and
impact of the public labor exchange system.

D. Initial Comments on the Draft for Discussion Issues Paper

The United States Employment Service in cooperation with the
Interstate Conference of Employment Security Agencies (ICESA) released
the issues paper, entitled, ``Draft for Discussion--America's Labor
Exchange Performance Measures,'' in September, 1997. It provided a
review of the rationale and a suggested conceptual framework for
establishing a set of key performance measures of labor exchange
services. Included was a discussion of a variety of alternative
measures and the identification of 10 potential performance measures of
staff-assisted job seeker and employer services. In addition,
performance measures were suggested for the electronic labor exchange
which counts holdings, usage, and transactions. Also included were two
system measures. One was a cost-to-benefit measure that looks at unit
cost per outcome, and the other was a measure of the impact of service
on Unemployment Insurance benefits compensated.
State Employment Security Agency Administrators were asked to
provide written reactions to the suggested measures by October 31,
1997. The ``Draft for Discussion'' issue paper was also made available
on the Internet on the USDOL-supported ICESA Workforce ATM website
(http://www.icesa.org/national/docs/LABOREX.HTML). This approach
provided an opportunity for the public labor exchange system to have
direct input to the initial development of the performance measures
that are the subject of this FRN.
Based on the comments received from States, and subsequent
additional discussion and consideration by the workgroup, the number of
job seeker measures has been reduced and several improvements have been
made to other measures. In some cases, where different approaches were
suggested and there was no clear-cut decision as to which approach was
the better choice, more than one option has been included in this FRN
for comment.
Current system redesign efforts to enhance and integrate the
features of America's Job Bank and America's Talent Bank systems will
make it possible to collect and aggregate some data electronically
without additional State data collection or transmission effort. It is
the Department's intention to actively pursue electronic data
collection approaches to the extent feasible.

IV. Definitions

A. Job Search Assistance (JSA): Includes services currently defined
in the ETA 9002 Data Preparation Handbook NO. 406 under the ``Received
Some Reportable Service'' categories and in items 12 through 42. The
service activities included are: job search workshops, job finding
clubs, resume assistance, providing specific labor market information,
job search plan development, job matching, job development, referral to
jobs, vocational guidance, assessment interviews, testing, vocational
counseling, federal bonding, and referral to other services

[[Page 32570]]

(including skills training, educational services and supportive
services).
B. Facilitated Self-help: Is a staff-assisted service in which
staff facilitates customer access to a variety of in-office self-help
job finding tools and resources. Support staff provides limited
assistance on an as needed basis. Most job seekers require limited
assistance. Normally, job-seeking customers are not required to
register for service before they are allowed access to self-help tools
or resource areas, rooms or centers.
C. Self-help Tools: Include, but are not limited to, resource
rooms, automated labor exchange system devices, Internet access, fax
machines, telephones, photocopiers, personal computers, word
processors, career and labor market information, and reference
materials.
D. Staff Assisted Labor Exchange Services: Include job search
assistance, job development assistance and job referrals, and can be
characterized as service which is conducted one-on-one or in small
groups.
E. Job Listing Services (JLS): Include activities performed on
behalf of employers to assist them in filling their workforce needs.
JLS includes but is not limited to services and activities such as job
order taking, job order maintenance, referral follow-up and interview
support, together with connecting activities including searching job
seeker databases, transmitting resumes, and marketing job orders to the
job seeker (applicant) pool, etc.
F. Referral Related Action: Includes the following services and
activities:
(1) Referral of qualified candidates, or
(2) Contact with the employer to review potential referrals, to
develop a candidate recruitment strategy or adjust features of the job
order which prevent identification of candidates for referral, i.e.,
salary, experience requirements, etc.
G. Business Assistance Service (BAS): Includes, but is not limited
to, providing employer education seminars, job and task analysis,
providing local labor market information, and referral to other
workforce development or economic development services or agencies.
There are a number of employer service activities which are
excluded from the definition of BAS. They are: periodic mass mailings
and routine promotional mailings to increase employer job listings,
providing labor market information via the Internet and employer
contacts for the provision of other core services, e.g., job listing,
job matching, initial screening or referral services.
H. Entered Employment: The unduplicated count of job seekers
(applicants) who enter employment by job placement or obtained
employment. See page II-12 of ETA 9002 Data Preparation Handbook, ET
Handbook NO. 406.
(1) Job Placement: The hiring of a job seeker by a private or
public employer after referral to a job by the Agency staff, or
collocated or out-stationed staff in cooperation with the Agency,
provided that the following conditions were fulfilled:
(a) Prepared a job order prior to referral, except in the case of a
job development contact on behalf of a specific job seeker,
(b) Made prior referral arrangement with the employer,
(c) Referred an individual who was not designated by the employer,
and
(d) Verified from a reliable source that the job seeker had entered
work, and the placement was recorded in the agency data base.
See page II-12 of ETA 9002 Data Preparation Handbook ET Handbook
NO. 406.
(2) Obtained Employment: Individuals who secure employment within
the current quarter or the next completed quarter following the last
staff-assisted job search assistance service that was partially funded
by the Agency, such as:
(a) Participating in job search activities,
(b) Accepting a position resulting from the use of an agency-
sponsored automated labor exchange,
(c) Receiving employment counseling or testing or development of an
employability plan,
(d) Receiving bonding assistance,
(e) Terminating from a skill training program to which a job seeker
was referred by the Agency, or
(f) Receiving tax credit voucher, and receipt of verification from
a reliable source, preferably the employer.
See page II-13 of ETA 9002 Data Preparation Handbook ETA NO. 406.
Entered Employment can be counted in the current quarter or the next
completed quarter following the receipt of the last staff-assisted
service (a maximum of 180 days).
I. Entered Employment Rate (EER): Is the percentage of job-seekers
securing employment after receiving Staff-Assisted Services.
J. Cost per Entered Employment (CPEE): Is the cost of achieving the
positive outcome of entry to employment following the provision of
service. The CPEE is determined by dividing the total of those who
entered employment by the total funding received for the federal
Wagner-Peyser Act, and Veteran Services grants and State appropriations
used for the Labor Exchange function during a program year (July 1-June
30).

V. Proposed Labor Exchange Performance Measures

Specific questions have been developed regarding these performance
measures for which public comment is sought. Questions can be found at
the end of sections A., C., and D. below, and they are numbered
sequentially, 1 through 16. Commentators need not repeat questions as
part of their response, but addressing the specific questions by number
would be most helpful to this effort.

A. For Self-Service Strategies

Performance measures will be output measures based on holdings,
usage and transactions (much like a library) until customer-friendly,
non-invasive sign-on and low cost follow-up approaches are developed to
identify job-seekers in the electronic labor exchange.
In developing these performance measures, the needs and interests
of legislators, policy makers, program managers, budget planners,
analysts, employers and other investors were considered. These output
measures may be used for strategic planning, program management,
continuous improvement and research. This information combined with
customer satisfaction surveys and feedback could assist in the design
of system improvements to meet customer needs.
It is DOL's expectation that national data reporting will be
produced by the AJB/ATB system, and will be aggregated and reported
electronically. It is also DOL's expectation that these performance
measures will not require additional state data collection.
1. Holdings: This measure is a count of the number of employer job
orders and the number of job seeker resumes in the AJB/ATB system.
Continued growth in the number of job orders and resumes in the AJB/ATB
system provides an indication of customer satisfaction and perceived
value by the customers.
2. Usage: This measure is a count of user sessions on the AJB/ATB
system. A user is an individual who accesses the AJB/ATB system for any
purpose. Some users of the system supply personal identification
numbers and passwords; others are anonymous. A user session represents
each single continuous access. This is different than a ``hit'', which
measures the number of Internet server accesses (ie. computer to
computer communications). Under this measure, we will count registered
employers, registered job seekers and anonymous sessions. Enhancements
to

[[Page 32571]]

the AJB/ATB system are underway to provide better information on
anonymous sessions. This output measure will provide a gauge of how
many customers are using these electronic labor exchange services over
some defined period of time and whether the usage is growing, remaining
static or declining.
3. Transactions: This is an output measure of a higher level of
interaction with the AJB/ATB system beyond a review of the information
resources. For example, among the variety of transactions which could
be collected electronically, two stand out. They are Job Seeker
Referral Requests and Employer Resume Downloads.
``Job Seeker Referral Requests'' measures the number of job order
referral instruction screens that are viewed by job seekers on AJB.
This output measure is a count of the number of times job order
referral instruction screens are viewed by job seekers on AJB after
viewing a job order description. This measure identifies the number of
users who have requested specific job referral information and are
likely to respond to the employer's job opportunity. There are two ways
in which this occurs: unsuppressed job orders and suppressed job
orders. In unsuppressed job orders the employer contact information is
broadcast and not hidden from the job seeker. In this case, the job
seeker self refers. In suppressed job orders the employer contact
information is not viewable or accessible through computers and the job
seeker must be referred to the employer by the local labor exchange
office holding the job order.
``Resumes Downloaded'' is a count of job seeker resumes that
employers have selected from a list of job seekers, who met an employer
defined search criteria, to obtain job seeker contact information.
Questions
The Department specifically invites interested persons to provide
comments, data, information and views concerning the following:
Q1. Do these proposed measures adequately represent an employer's
or job seeker's satisfaction with the Electronic Labor Exchange?
Q2. How would you rate the importance of the three types of self-
service outputs (holdings, usage, and transactions)? Which would be
most or least important for strategic planning, program management,
etc.?
Q3. What other ways would you suggest for measurement of self-help
services?
Q4. How do you currently or how would you suggest the national
system collect customer satisfaction data and information?
Q5. Will the proposed Self-Service Electronic Labor Exchange
measures enhance State quality improvement initiatives?
Q6. What specifically defined period of time would be most
appropriate for self-service performance measurement: monthly,
quarterly, or annually?

B. For Facilitated Self-Help Strategies

Two Facilitated Self-help measures are being proposed: the number
of users and customer satisfaction. These measures are intended to
determine the extent to which customers value facilitated self-help
services. The measures address this by answering two questions: do
customers use the service and are customers satisfied with the
services?
1. Number of Users of Self-Help Service
The Number of Users of Self-Help Service is an output measure of
the number of customers who access the self-help resources which
includes the assistance of a knowledgeable staff member assigned to
facilitate customer access to and use of the self-help tools. It is the
intent that facilitated self-help provides service to a large number of
job seekers with a minimum investment of staff time.
The accumulation of a count of users can be as rudimentary as an
office stroke tally or a sign-in log. It would not be necessary to
track which self-help tools are used, although valuable information
needed for the continuous improvement of facilitated self-help programs
might be gained from a periodic assessment of which tools are most
popular or useful to job seekers.
2. Customer Satisfaction
Customer Satisfaction can be measured by conducting periodic formal
telephone or mail surveys or by collecting customer in-person feedback
using a structured approach at specified intervals. The measure could
capture the frequency of use of self-help resources. Customers normally
are not required to register to use the self-help tools in a labor
exchange office or one-stop career center. These measures recognize the
customer's use of and satisfaction with self-help resources.
Facilitated Self-Help is a widely used and valid form of service.
Self-Help service is consistent with the reality of shrinking
resources, and relates directly to customers' demand. This measure can
contribute to continuous improvement with input from the customer.

C. For Staff-Assisted Service Strategies

Measures of Job Seeker Customer Services
Staff-Assisted Labor Exchange Services are the core activities of
the modern Labor Exchange Office. The purpose of these performance
measures is to determine the outcomes, effectiveness, and system impact
of Staff-Assisted job seeker service. Performance will be evaluated
through a combination of interrelated performance measures. These
measures will be useful to legislators, employers, policy makers,
agency administrators and program managers since they provide a tool
for managing programs, baseline data for continuous quality
improvement, and feedback that will allow States to respond to customer
needs. These quantitative measures should be supplemented by
qualitative measures of customer satisfaction and are not intended to
be the sole measures of satisfaction.
1. Entered Employment Rate (EER): The EER is the percentage of job
seekers securing employment after receiving staff-assisted labor
exchange services divided by the total number of job seekers who
received staff-assisted labor exchange services. This measure uses as
its denominator only those customers who receive staff-assisted
service(s) and not the total number of applicants registered in the
labor exchange office database. This performance measure looks at the
effectiveness of the Job Search Assistance services that have
employment as the expected outcome. This measure encourages an
increased level of job order and job seeker follow-up, tracking and
employer feedback. Service and entered employment data would be
collected and reported by staff to the labor exchange reporting system.
Unemployment Insurance wage records can be an additional data source
used to conduct job seeker follow-up on entry to employment.
[Optional Measure] Job Development Entered Employment Rate: Is an
additional and optional measure of Job Search Assistance service. Job
Development (JD) has been an effective tool to help individuals who
have barriers to employment or difficulty in finding employment to find
jobs. With the expected expansion in the role of the Labor Exchange in
Welfare-to-Work programs, it has been suggested that a JD Entered
Employment Rate would be valuable for program management and continuous
improvement.
Job Development Entered Employment Rate is the percentage of job
seekers who entered employment after receiving Job Development referral

[[Page 32572]]

compared to the total number of job seekers who received JD referral.
An outcome measurement of the pro-active solicitation of a job
opportunity for an individual or group, this measure assesses the
effectiveness of job development efforts in contributing to job seeker
entry to employment. Job development is defined as development of a job
opening for a job seeker or group of job seekers through direct contact
with potential employers when no suitable job openings are currently
listed. Job development contacts are currently reported within the
category of ``Received Reportable Service'' in the ETA 9002 report.
This performance measure is intended to create an incentive for
staff to use their considerable knowledge of the labor market and
employers' needs to increase the number and effectiveness of job
development contacts and referrals on behalf of job seekers.
Measures of Employer Customer Services
The variety of staff-assisted services that are made available to
employers can be grouped into three categories: Job Listing Services,
Job Matching, and Business Assistance service.
The purpose of the proposed performance measures is to determine
the extent to which employers' needs are being fulfilled in terms of
the ``Timeliness,'' ``Quality,'' and ``Impact'' of our work. Staff-
assisted employer service performance should be evaluated by this
combination of interrelated performance measures. The individual
measures should not be viewed in isolation and these measures should be
supplemented by qualitative measures of employer satisfaction.
The proposed measures provide an indirect indication of employer
customer satisfaction. They are not intended to be the sole measures of
satisfaction. For example, the Return (Repeat) Business measure may be
a good measure of employer satisfaction at the macro (state or
national) level. However, federal contractors and others who are
required to list their jobs with the Agency may or may not be satisfied
employer customers.
It is recognized that services provided to employers are not
currently reported to the USDOL and may not now be collected by the
labor exchange agency. These employer service measures may require
States to develop new administrative reporting or program record
keeping systems.
2. Job Listing Return Business Rate: The number of employers who
list more than one job order divided by the total number of employers
who use the Staff-Assisted Job Listing Services during a reporting
year. This outcome measure of repeat business serves as a means to
determine employer satisfaction with the job listing service.
The intent of this measure is to provide an incentive for service
delivery staff to improve customer service and relationships including
quick response time, quality referrals, and employer follow-up
contacts. Most of the data needed for the Return Business measure is
readily available with the possible exception that some States do not
capture employer identification numbers on job orders. This measure
would require that employer identification information be collected and
reported for all job orders in the agency reporting and Job Bank
system.
Some States have expressed a preference for measuring job openings
rather than job orders. The focus of this measure is employer
satisfaction, not the volume of job openings or the number of job
orders the labor exchange receives or an employer market penetration
rate. Repeat business is a better indicator of employer satisfaction,
and this measure is used in the private sector as an important
indicator of success in service sector business enterprises.
Some States and labor exchange staff were also concerned with the
effect of employer direct job order entry into AJB and employer access
to resumes on ATB and its impact on the number of job orders they would
receive. In the past the number of job orders secured by a local office
and listed in the State job bank has been a key local office
performance measure in some States. Again, this measure represents the
percent of Return Business which results from the employer's
satisfaction with the Staff-Assisted service. Job orders entered
directly by employers are captured in the electronic labor exchange
measures of AJB and ATB holdings and are not lost to the state. An
unintended consequence of this measure may be a counter-productive
competition between a State's staff-assisted service component and the
State's electronic labor exchange component. From a local office
viewpoint, this measure may motivate staff members to discourage
employers use of the job order self entry service options in AJB to
assure that there is a high rate of repeat job listing business
credited to the staff assisted service component. Enhancements to the
AJB/ATB systems and report generators are currently underway as
mentioned above. This labor exchange system performance issue will be
addressed in future AJB/ATB versions. The Department would appreciate
comments and suggestions for dealing with this potential consequence of
measuring Repeat Employer Business in the staff-assisted service
component of the labor exchange.
Other States commented on the impact of having a large number of
small employers or adverse economic conditions which would reduce the
opportunity for return business during a reporting period as short as a
year. DOL acknowledges the concern and would encourage comments which
are based on a review of State Employment Service, Unemployment
Insurance or Tax administrative data which demonstrate this phenomenon.
DOL would also be interested to learn if this is a result of an agency
policy to target small employers who have limited workforce replacement
or expansion needs.
Finally, even if the suggested defect in this performance measure
is accurate, the Return Business measure would be useful for strategic
planning, performance management and continuous improvement efforts.
This measure is intended to be used by States for national performance
reporting and state program management as opposed to being used for
State-to-State comparisons.
3. Business Assistance Service Return Business: In addition to Job
Listing and Job Matching services, there are a number of other types of
Staff-Assisted labor market information and services which employers
require. These were defined in section IV. G. above. The provision of
Business Assistance Services (BAS) is intended as a means of
stimulating increased employer job listings and employer support for
the Labor Exchange and Workforce Development System. The BAS Return
Business Rate is the number of employers who utilize more than one BAS
service divided by the total number of employers who use at least one
Job Listing, Job Matching or Business Assistance staff-assisted
service. The intent is to measure the level of employer customer
satisfaction with BAS provided and to be an incentive to increase
service to an important customer.
4. Referral Response Time: Is an outcome measure of Job Matching
Service (JMS) effectiveness, which combines the interests of job-
seekers and employers. The Job Order Response Time measure is the
percentage of job orders for which referral-related action took place
within three business days compared to the total number of job orders
listed. This performance measure provides an incentive for quick
response

[[Page 32573]]

to job orders by job-seeker referral or by direct referral-related
contact with employers. Three business days have been identified as the
generally accepted standard for referral and/or follow-up contact with
employers who use the Job Listing Service (JLS).
Some States indicated concerns regarding their ability to track
employer follow-up contacts. It is recognized that current systems may
have to be adjusted to capture this information. Although it is
understood that not all job orders can have a quality referral made
within three days, this measure encourages rapid response to the
employer's need by the provision of other employer services such as,
contact with the employer to review potential referrals, to develop a
candidate recruitment strategy and/or to adjust features of the job
order which prevent identification of candidates for referral, i.e.,
salary, experience requirements, etc.
5. Average Time Lapse to Successful Referral: This outcome measure
will provide a picture of how quickly labor exchange agencies respond
to job orders with referral of a qualified candidate for employment.
The performance measure is the average time lapse (in days) from the
date an employer's job order is listed to the date of referral of the
first agency-referred individual hired by that employer. The underlying
presumption is that when the employer decides to hire one of the labor
exchange referrals, the agency has met the employer's candidate
qualification need.
Not included in this calculation are job orders where the employer
hires from another source. This removes from the equation situations
where, although qualified candidates are referred, the employer decides
to hire an equally qualified candidate from another source.
This measure focuses on the time lapse to the date of job-seeker
referral. An employer's hiring practices which may delay the offer of
employment for many days or weeks does not affect the outcome of this
measure. This measure provides a snapshot of the agency's referral
practices in terms of responsiveness with a quality feature. It is
intended to balance the rapid referral measure and mitigates the
possibility that fast, but poor quality, job-seeker referrals will be
the reaction to a rapid response performance measure.
Job Order Fill Rate: Based on State comment, an additional measure
of the Employer Satisfaction is proposed. The Job Order Fill Rate is
the number of job orders for which a placement is made divided by the
total number of suppressed job orders received within the reporting
year. A job order for which the employer name, address, and other
identification or contact information is hidden from the job seekers'
view is considered to be a suppressed job order.
Employers have the option of listing a job order in a suppressed or
unsuppressed mode depending on the level of initial screening, and/or
referral control service the employer wish the labor exchange to exert.
Frequently employers are guided in the decision to suppress contact
information based on the local availablity of candidates for employment
who meet the skill, background and experience required to be successful
candidates for employment by labor exchange staff.
This outcome measure will provide a picture of how effectively
agencies or States respond to these job listings with qualified
candidates for employment. In this measure, the fact that the employer
hires one of the job seekers referred by the agency is verification
that the candidate was qualified for the job and that the employer was
satisfied with the service provided.
The intent of this measure is to provide an incentive to meet and/
or exceed employer customer expectations by referral of qualified
candidates for employment. Not including in the denominator of this
performance measure job orders which are listed in an unsuppressed or
broadcast fashion avoids the possible unintended consequence that the
labor exchange fill rate performance is measured against job orders
where staff-assisted service is neither needed nor wanted by the
businesses that want to broadcast available employment opportunities.
The required data are currently being collected in many States and are
available.
In some States, a Job Opening Fill Rate performance measure has
been used for many years as the approach to gauge the efficiency of
local Labor Exchange offices and operations. In other States, the Job
Opening Fill Rate has been discontinued because it has proven to be a
disincentive to securing employer job orders in occupational areas
where local management and staff would find it difficult to identify
local and immediately available candidates. By including in the
performance measure denominator only those job orders and openings
which have the employer contact information suppressed, there will be
less of an incentive to withhold job orders from the system or to
artificially reduce the number of job openings. Based on labor market
information, and in consultation with the employer, job orders that are
not likely to be filled by a local candidate can be listed in an
unsuppressed fashion and would therefore not count in the Fill Rate
measure denominator.
Questions
The Department specifically invites interested persons to provide
comments, data, information and views concerning the following:
Q7. Are there other services which your State or agency provides
which have entry to employment as a goal which should be included in a
list of Job Search Assistance services?
Q8. Veteran service and other target group service programs
continue to provide a higher level of one-on-one job-seeker service and
job development assistance. Is the level of the job development
assistance activity in your State or agency significant enough to
require measurement of the Job Development Entered Employment Rate
separately and apart from the Staff-Assisted Entered Employment Rate
(EER)?
Q9. What would be the impact of a requirement to collect or have
the facilities necessary to associate an employer ID number with a job
order?
Q10. Is the proposed three business day standard for measuring
referral-related follow-up contact with employers consistent with your
agency or State approach?
Q11. The Job Order Fill Rate measure is intended to provide an
indication of employer satisfaction with staff-assisted labor exchange
services. Are the unintended consequences of a fill rate performance
measure serious enough to eliminate it from consideration as a program
performance measure?
Q12. Are there other services and/or service delivery approaches
which should be included in the mix of services suggested under the
Business Assistance Service definition?
Q13. Will these proposed Staff-Assisted Labor Exchange service
measures enhance State continuous quality improvement initiatives?

D. System Measures

1. Cost Per Entered Employment (CPEE)
This is an outcome measure of the efficiency of the labor exchange
function. The CPEE is calculated by dividing the total number of
entered employment counts by the total funds allocated to the labor
exchange function. Funding sources would include Wagner-Peyser Act,
Veteran Employment and Training Service (VETS) allocations to States,
and supplemental State funding provided for the labor exchange
function. This measure will provide the context for the public labor
exchange systems to

[[Page 32574]]

compare its key outcome--entered employment--against other workforce
development components. For example, a 40% EER that costs only $300 per
entered employment may seem more favorable when compared to an 80% EER
when each EE costs $5000.
Another intent of the measure is to promote efficiency by providing
management with cost-of-operation information. This information could
be used for strategic planning, allocation and distribution of scarce
resources, as well as continuous quality improvement efforts.
Administrative data sources are available to produce this system
measure, and includes SESA administrative records and labor exchange
job-seeker service records. In many States, Unemployment Insurance wage
record data can be used to improve entered employment information.
Funding data can be captured from the annual grant allocation
documents for Wagner-Peyser and Veteran's Services. The information on
State funding for the labor exchange function would have to be
collected by canvassing the States to determine the level of additional
State support. The cost should be calculated based on the Wagner-Peyser
Program Year basis, which is consistent with other employment and
training program funding cycles. The Veterans' program grants would
have to be prorated since they are allocated based on a Federal Fiscal
Year. A similar proration would be required for State funding which is
not allocated on a program year basis.
The CPEE measure is easy to understand and is generally accepted.
These data are also useful for program management and the resource
allocation process.
2. Duration of Benefits Compensated
Employers are the primary customers of Workforce Development
programs nationwide. To meet employers' workforce needs, public labor
exchange agencies focus on job-seekers who possess a labor market
attachment and marketable skills. In addition, Wagner-Peyser Act
funding is provided to the public labor exchange, in part, to
administer the work test for the State Unemployment Insurance
compensation system and to provide job search and placement services
for claimants. The effect and/or impact of services directed toward UI
benefit customers has many dimensions which can be significantly
affected by movement in the economy and local business conditions.
However, there is a general belief that labor exchange service can have
a direct and measurable impact on the duration of benefits.
Two options are proposed to measure the impact of staff-assisted
service on unemployment insurance benefit customers:
The first option is a measure of duration of benefits compensated
for claimants who receive services compared to prior year(s)''
duration. This measure can be adjusted based on economic conditions.
This measure seeks to determine the impact of staff-assisted labor
exchange services on the duration of benefits for claimants who
received services. Data can be collected from Unemployment Insurance
average duration data records (UI claims first payments/weeks
compensated), labor exchange agency records, and local office data
reporting. USDOL Unemployment Actuarial Unit data on business cycles
and economic conditions could also be useful in explaining the impact
of job-seeker services on claimants. Benchmark data would be available
from previous State reporting.
An easy-to-understand measure of the success of labor exchange
services, this measure can be adjusted to consider State and local
economic conditions. Most of the required data is easily accessible.
This duration-of-benefits model could also be adapted to other groups
such as Welfare-to-Work customers and Veterans, etc., and could be
expressed as a reduction in the number of income transfer payments, as
a trust fund savings, or as a part of a return on investment statement.
The second option is a measure of the impact of Labor Exchange
Services on the Duration of Benefits Compensated for UI Claimants
Required to Search for Work (Work-Test Claimants) who receive Staff-
Assisted Labor Exchange services and enter employment compared to the
average duration-of-benefits compensated to all claimants who are
required to search for work.
This outcome measure also seeks to determine the impact of staff-
assisted labor exchange service on the duration of benefits claimants.
In this option, the group to be studied will consist of UI claimants
who: (1) were not exempt from an active work search; (2) received
staff-assisted labor exchange job-seeker services; and (3) entered
employment.
Staff-Assisted Labor Exchange services for claimant job-seekers
includes job search assistance (JSA) and a variety of facilitated self-
help services enumerated in the definitions above in section IV. A, B,
and C. Since the benefit customer job-seeker's identity is known, they
are usually registered in the labor exchange system.
A claimant who is required to search for work is a job-seeker who,
as a condition of receiving Unemployment Insurance benefits, is
required to perform an active work search. This criterion excludes UI
benefit customers who are on a temporary layoff, required to secure
employment through a union agent, partially unemployed and receive
benefits under an approved partial benefit program, or exempt from work
search requirements due to enrollment in an approved training program.
This performance measure provides an incentive to deliver high
quality JSA and other service to UI job seekers, to begin the provision
of service early in the claim, to aggressively follow-up on job
referrals, to track UI job-seekers' follow-through on work search
plans, and to determine when the benefit claimant became re-employed.
Administrative data are available for development of this measure
of impact from Unemployment Insurance Wage Records (claims filed and
average duration data), ES records (referral and service data elements
in the current ETA 9002 compliant reporting systems and the Employment
Security Systems Institute systems), and local office data reporting.
In addition, federal and State benchmark databases which can be used to
measure the State against its previous accomplishments are available.
The measure can be used to show the effectiveness of service when a
reduction in duration can be shown, and would be useful to legislators,
employers, and policy makers. The impact or reduction in the duration
of benefits performance measure could be:
(a) translated into a reduction in the number of weeks compensated
and a calculation of Unemployment Insurance Trust Fund savings using
the State's average benefit rate, and
(b) adapted for other target groups including Welfare-to-Work
customers, Veterans, etc. In this case the result could be expressed as
a reduction in the number of monthly income transfer payments, as a
dollar savings in benefits using average benefit rate data, or as part
of a return on investment statement.
Questions
The Department specifically invites interested persons to provide
comments, data, information and views concerning the following:
Q14. Can a measure of UI Benefit duration provide a meaningful
measure of the impact of labor exchange services on those claimants who
receive staff-assisted services?
Q15. Would the measurement, and comparison of the rate of UI claim

[[Page 32575]]

exhaustion be a better system measure of the labor exchange staff-
assisted service to UI benefit customers?
Q16. Considering the measures proposed in this FRN, is there an
optimal mix of performance measures which your State or agency would
suggest?
Finally, it should be noted that the Workforce Development
legislation currently pending before the Congress (H.R. 1385 and S.
1186) provides core performance measures that include, in addition to
entered employment, retention in employment six months after entry and
increases in earnings. Under the proposed legislation, these measures
would apply to participants in the Workforce Development activities
provided through the One-stop System, including labor exchange
activities.
We would appreciate receiving views on the appropriateness of these
additional measures for measuring public labor exchange programs.

Paperwork Reduction Act

The notice issued here is not subject to the requirements of the
Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.) Because it
contains no ``collection of information'' as defined in 44 U.S.C.
3502(3).

Signed at Washington, D.C., this 8th day of June 1998.
Raymond J. Uhalde,
Acting Assistant Secretary for Employment and Training.

BILLING CODE 4510-30-P

[[Page 32576]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.016

[[Page 32577]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.017

[[Page 32578]]

[GRAPHIC] [TIFF OMITTED] TN12JN98.018

[FR Doc. 98-15754 Filed 6-11-98; 8:45 am]
BILLING CODE 4510-30-C

---

Source: Frix Law Library, https://www.frixlaw.com/law-library/documents/fr%3A98-15754. Public record. Not legal advice.