RECENT EFFORTS TO MAXIMIZE BENEFITS FROM THE STATISTICS OF INCOME

RECENT EFFORTS TO MAXIMIZE BENEFITS FROM THE STATISTICS OF INCOME

ADVISORY PANEL

Presented at the 2003 American Statistical Association Meetings

Tom Petska and Beth Kilss

Statistics of Income (SOI) Division, U.S. Internal Revenue Service

P.O. Box 2608

Washington, District of Columbia, 20013-2608

Key words: Advisory panel; administrative records;

statistical operations; microsimulation modeling.

The Internal Revenue Service's Statistics of Income

(SOI) Division has had, for over 15 years, a

Consultants’ Panel whose membership consists of a

distinguished group of individuals from academia,

nonprofit organizations, State and local government,

and the private sector. The Panel has always had a keen

interest in helping SOI fulfill its mission by assisting

SOI staff to improve its overall performance and in

providing guidance and advice to make SOI ever more

efficient, forward thinking, and responsive to its many

customers in and outside of the public sector. In

addition, the Panel has served as a management

sounding board on issues, including strategic planning,

data dissemination, and project prioritization.

For many years, this assistance was primarily as a

result of periodic meetings in which SOI staff presented

ongoing plans and operations to Panel members and

invited guests from the public to solicit feedback,

guidance, and direction. While these efforts were

beneficial, both Panel members and SOI staff agreed

that greater involvement in the core operations of SOI

could be mutually beneficial. This paper is a progress

report on how SOI has solicited greater involvement

from its Panel members, what has been accomplished to

date, and what approaches and initiatives are being

planned for the future.

Background Information

The SOI function goes back to the enactment of the

modern income tax in 1913. In the 1916 Act, it was

written that "the Secretary (of the Treasury) shall

prepare and publish not less than annually statistics

reasonably available with respect to the operations of

the internal revenue laws." Despite many revisions to

the tax law, the original requirement of that Act

continues to this day.

The mission of the SOI program is to collect and

process data so that they become meaningful

information and to disseminate this information to

customers and users. The SOI Division conducts for the

Internal Revenue Service and Treasury Department

studies on the operations of tax laws with respect to

individuals,

corporations,

partnerships,

sole

proprietorships, estates, nonprofit organizations, and

trusts, as well as specialized studies covering both

inbound and outbound international activities.

The SOI Division has produced studies and

published reports for over 85 years. However, around

1980, with the advent of modern computing and

microsimulation modeling by policy and revenue

estimation functions at Treasury and elsewhere, it

became apparent that SOI had not kept up. In the first

half of the 1980's, under the direction of then Director

Fritz Scheuren, a major overhaul of SOI's

methodologies and key business processes began. By

mid-decade, several developments and accomplishments were of note:

1. A renewed emphasis on quality that had come to the

Internal Revenue Service was closely embraced

within SOI.

2. SOI began to attract many new technical staff who

could help lead the retooling of projects.

3. SOI began to develop its own "minicomputer

network," replacing reliance on IRS mainframe

technologies where statistical programs were a low

priority.

4. An SOI "Consultants Advisory Group" was formed

to help guide and direct these efforts.

Minicomputers were placed in SOI's headquarters

in Washington, as well as in two key field sites, Ogden,

Utah and Cincinnati, Ohio. Pilot projects began to

develop a one-pass approach to the complex data

editing operations that were a substantial improvement

over the multi-iterative approach used on the IRS

mainframe computer systems. Each year, SOI staff

made regular and continuous improvements to the

systems, so that, over time, SOI's data processing and

analytical capabilities became quite sophisticated.

Early Involvement by the Panel

The Consultants’ Panel evolved from discussions

and involvement with the late Joe Pechman, a scholar at

The Brookings Institution, who saw the large public

value in greater access to and dissemination of tax

return data. In the early 1960's, SOI had developed its

first public-use microdata file, a non-identifiable subset

of the annual sample of individual tax returns, at

Pechman's urging, as an invaluable tool for tax policy

analysts outside of the Government who did not have

access to SOI's rich data files. The success of the

public-use file and the potential gain for the policy

analyst community prompted him to host periodic

meetings at Brookings and eventually suggest the

formation of a formal advisory board in the mid-1980's.

The formation of an advisory group by IRS’s

Statistics of Income Division was not unique to the

Federal statistical agency community. Most of the

major Federal statistical agencies have a long history of

using one or more advisory groups as a mechanism for

inviting the participation of private citizens in their

decision-making processes. [1] In 1986, the SOI

Division was a relative latecomer to this arena and

chose a less formal arrangement by forming an ad hoc

group as opposed to an advisory committee operating

under the provisions of the Federal Advisory

Committee Act (FACA; Public Law 92-463, 92nd

Congress, House of Representatives 4383, October 6,

1972). From the beginning, the SOI Division obtained

advice from the individual members of its advisory

group rather than from the group as a whole.

The SOI Consultants Panel formally began in the

spring of 1986 with a general mission of use as a

sounding board and a source of ideas and innovations.

Planning sessions between Joe Pechman and then SOI

Director Fritz Scheuren in preparation for the first

meeting established the scope and character of the

Panel. Initially, they determined that the main purpose

of the Panel was to help shape the SOI program so its

products would be given wider use in the research

(academic/business/policymaking) community.

The

spring was chosen for the first meeting to coincide with

the completion of SOI’s multiyear planning process,

and it focused on the current SOI program with special

emphasis on future directions and changes. A followup meeting was suggested for the fall of that year

because it would serve as a checkpoint on the degree to

which SOI could incorporate changes into its plans in

preparation for the next multiyear plan. It was also

thought that a spring 1987 meeting would provide the

forum for reviewing the new plans resulting from the

above process and for deciding on the periodicity of

meetings thereafter. Thus was born, 17 years ago, a

framework for SOI to gain more systematic input about

how well it was doing as an organization and how it

might improve service to its customers that has become

an integral part of the SOI culture.

Seventeen years ago, at the fall 1986 meeting, SOI

concluded the session by giving all Panel members and

participants an opportunity to identify those issues they

felt the Division should be most concerned with. Many

of the areas mentioned then were, and still are, a major

focus of the SOI program, perhaps underscoring their

significance, and yet also indicating SOI’s need to

continually work to improve what it does best. Some of

the issues raised at that session that still resonate today

include: risk of disclosure for microdata; public access

and confidentiality issues; archiving and documenting

historical files; the implications of electronic filing on

the SOI program; and the development of more

longitudinal data.

It is also true that, while many

topics have been repeated on the agendas over the

years, there has also been much variety introduced into

the sessions. Some noteworthy examples include

conducting offsite meetings at two IRS service centers;

inviting guest speakers; and organizing the session as a

workshop. Many Panel meetings were organized

around particular themes, some focused on

technological innovations, while others included online

demonstrations of SOI's computer systems.

The focus of the Panel meetings has included tax

policy data needs, statistical disclosure research,

computer modernization, microsimulation modeling,

tax reform, and individual and corporation data. Early

Panel meetings in the late 1980’s and early 1990’s

regularly included updates on the SOI Division’s

individual, corporation, partnership, foreign, and

special studies programs. These meetings frequently

included presentations by Panel members on such

topics as State tax statistics, household surveys with tax

data, analyzing SOI panel data, individual tax model

research, and economic statistics initiatives. Later,

such topics as the earned income tax credit, tax gap, the

Survey of Consumer Finances, use of SOI data in

emerging tax issues, data sharing legislation, and data

warehousing were covered.

Panel Feedback and Input

While each Panel meeting agenda throughout the

years comprised different, specific topics, which were

interesting and educational, discussions of SOI

programs or systems between SOI staff and Panel

members were often the most revealing and beneficial.

Though SOI periodically received much praise for the

quality, usefulness, and importance of its data and the

professionalism and caliber of its staff, it also

recognized that it needed to do more, in some part

because of the feedback and input from Panel meeting

discussions. As was hoped when the Consultants Panel

was originally formed, ideas for improvements were

presented, suggestions were made in a neutral

environment, and general underlying themes kept

recurring that eventually pinpointed a number of

program shortcomings. One result was that SOI

immediately undertook initiatives to address program

limitations and deficiencies. [2] Former Director Fritz

Scheuren and current Director Tom Petska, who was at

the time Chief of the Division’s Coordination and

Publications Staff, developed a list of items from these

initiatives to present to Panel members for

consideration and comment.

These items were

discussed as a “research and improvement agenda” at

one of the Panel meetings in the early nineties.

Discussions focused on tradeoffs among improvement

priorities, and Panel members were polled for their own

individual rankings. Five initiatives include the needs

for greater program timeliness, improved data

consistency, better tracking of demographic changes,

preservation of historical information, and public

access. These five are summarized below.

•

•

•

Timeliness--The fact that users never have

enough current information from tax returns is

an inherent weakness of the SOI program.

Timeliness of SOI studies has been a focus for

improvement and one in which some success

has been achieved. In all major SOI studies,

there is an ongoing commitment to complete

statistical processing within a minimum time

after the close of the sampling period.

Delivery dates have improved as a result.

Preliminary data are also provided as early as

possible.

Data

Consistency--Problems

of

data

consistency are of two general types, statistical

and conceptual. Despite extensive validity

testing, inconsistent or erroneous data still

escape undetected for a variety of reasons in

some SOI data files. Efforts continue to rid

these out of the system. Improving the

conceptual clarity and year-to-year consistency

of the content of tax and information returns is

also a problem that has no easy solution.

Where possible, efforts have been made to

ensure consistency in time series data.

Tracking

Demographic

Changes--The

redesign of the individual program at the

request of Treasury’s Office of Tax Analysis

(OTA) underscored the need to improve

longitudinality in SOI studies. Transactions

such as capital asset realizations, that can have

multiyear ramifications, can only be examined

by means of a panel data base. A similar need

for greater longitudinality also applied to

business sector studies.

Tax reforms,

particularly those affecting individual and

corporate tax rates, have increased the

occurrence of changes of legal form, such as

switching from a corporation to a limited

partnership. Developing panel data in the

individual and corporate areas has been a

major focus of SOI work over the past 10

years.

•

Preservation of Historical Information-Although current efforts are focused on better

meeting current and future customer needs,

SOI has become “keeper” of an abundance of

tax information documents in a variety of

media. Much of this information, though

cumbersome to use, is irreplaceable.

However, as new technologies become

available, the cost of moving this information

into more user-friendly formats will drop

considerably. A difficult decision has been

and continues to be how many current

resources should be diverted from present

work to safeguard this historical information.

•

Public Access--Tax returns are protected by

law from public scrutiny, and strict procedures

govern the handling of returns and computer

tape files containing such information. Even

after specific identifiers (e.g., name, address,

and Social Security number) are removed, the

remaining tax return data are usually still

confidential. While SOI’s primary customers

are authorized to receive detailed tax return

microdata files, other users may have only

summary tabulations. Public-use microdata

files of individual tax data have been produced

regularly since 1960 and are the only source of

certain information. An ongoing issue for SOI

has been how to make more tax microdata

publicly available to researchers outside of

Government. This will continue to be studied

in both the individual and corporate areas.

These items are all crucial to the growth,

development, and success of SOI if it is to be

considered a world-class statistical organization. Panel

members’ opinions on these topics during Panel

meeting discussions over the years were certainly one

of the factors that helped shape SOI’s thinking and

decision-making as the Division sought to make

continual improvements to its programs.

Panel Membership

With rotating membership, the Consultants’ Panel

has met virtually every year since 1986. The 10-15

members of the Panel represent academia, the corporate

world, economic research centers, State governments,

and nonprofit "think tanks." Attendees at Panel

meetings include the members themselves, SOI staff,

and invited guests. These include members of the

Treasury’s Office of Tax Analysis (OTA); the

Congressional Joint Committee on Taxation (JCT); the

Congressional Budget Office (CBO); the General

Accounting Office (GAO); the Census Bureau; the

Federal Reserve Board; and others from research

organizations and academia. The daylong meetings are

usually held at The Brookings Institution in the spring

or fall. SOI reimburses Panel members only for travel

and per diem; so, their advice and guidance are largely

pro bono.

The Panel was originally chaired by the late Joe

Pechman of Brookings and consisted of 12 additional

members. Of those 12, the 4 who remain to this day are:

• Martin David, the University of Wisconsin and The

Urban Institute

• Dan Feenberg, the National Bureau of Economic

Research

• Gene Steuerle, The Urban Institute

• Bob Strauss, Carnegie Mellon University

In addition to the above, the current Panel

membership has added the following members, all of

whom have served at least 5-10 years:

•

•

•

•

•

•

•

•

•

•

Bill Gale, The Brookings Institution and Panel

Chair

Steve Caldwell, Cornell University

Virginia Hodgkinson, Georgetown University

Tom Neubig, Ernst & Young

George Plesko, MIT

Joel Slemrod, University of Michigan

Lin Smith, PricewaterhouseCoopers

Phil Spilberg, California Franchise Tax Board

Jenny Wahl, Carlton College

Sally Wallace, Georgia State University

Benefits from the SOI Consultants’ Panel

Over the years, the SOI Consultants’ Panel has

become a critical part of the communication process

between SOI and its customers. While other statistical

agencies, like the Census Bureau and the National

Center for Health Statistics, hold major user

conferences on a regular basis to receive input from

their customers, SOI has chosen this small-scale and

relatively inexpensive approach to keep its customers

informed.

There have been many benefits to SOI from the

Panel. These include:

• The Panel provides an opportunity to tap into an

extensive knowledge base of tax experts, some of

whom are regular SOI data users.

• SOI staff members have given presentations to the

Panel on technological and methodological

•

•

•

•

•

improvements in SOI programs or quality initiatives

that affect SOI projects. These occasions have been

valuable learning experiences for staff members and

resulted in specific suggestions, which have led to

further improvements.

A continuous theme from the Consultants’ Panel has

been the need for more timely and electronically

available data. In the early 1990’s, this led to the

development of the SOI Electronic Bulletin Board,

the forerunner of the current Tax Stats on the IRS

website.

Demonstrations of online systems have led to

improved understanding by users of how SOI data

are processed.

Discussions of statistical innovations by SOI staff

have resulted in valuable comments that led to

further improvements in SOI methodology.

Panel members also strongly advocated the need for

developing metadata systems, which more fully

document a study’s processes from start to finish.

Input from microsimulation modeling experts has

helped SOI to provide better data for its tax policy

analysts at Treasury and the Joint Committee on

Taxation.

New Directions

All Panel members believe it is important to have

public-use data on the functioning of the tax system and

have given time and energy to ensure that SOI

continually improves its capabilities to make available

timely, high quality data from tax and information

returns. Under the tutelage of new SOI Director Tom

Petska, the Panel once again meets biannually, and

members have been asked to get more involved in areas

of SOI modernization.

As noted above, the SOI Division has produced

annual Public-Use Files (PUF’s) since the early 1960's,

and, while there has been periodic and anecdotal

feedback from PUF users on how SOI could best

suppress the data to minimize analytical pursuits, the

Division never had a formal PUF users’ group. In the

spring of 2001, a group was formed and, after 2 years,

has become an unqualified success. PUF data users

have welcomed the opportunity to contribute to overall

plans for disclosure suppressions.

The PUF users’ group has six members from the

user community, two of whom are Panel members. The

success of the PUF Users’ Group as a way to improve

communications with users, to obtain users’ advice, and

to revise data files in a way most useful to data users is

an excellent model for forming similar subgroups from

the SOI Consultants’ Panel membership. As a result,

SOI Director Petska decided to seek additional Panel

involvement and assistance to streamline SOI

operations in four additional areas. His expectation was

that every Panel member will become a member of one

of these subgroups and help SOI explore possibilities

for systematic improvements in its key operations.

These areas are:

1.

2.

3.

4.

content and frequency of all SOI publications,

examine how to make them more useful, look

at methods of advertising and disseminating,

and look at what is not being published that

perhaps should be, e.g., new types of Bulletin

articles. A standing committee of senior SOI

staff, working with three Panel members, has

been formed to help anticipate these needs and

make data more useful to a wider audience of

researchers and practitioners. Expanding the

regular statistical content of publicly available

data in publications and/or the website would

make SOI data more useful to a broader

audience, and also eliminate needs for "ad

hoc" data requests, which can be disruptive.

Modernizing SOI's website to efficiently

disseminate data;

Guiding research in estates and gift taxation

and personal wealth;

Improving SOI's publications and tables; and

Advising on how to improve training and

employee development.

Each of these areas, and the roles for assistance

from the Panel members, are described below.

1.

Web modernization team.--In the fall of

2002, an inhouse team of SOI’s Internet

"visionaries" was commissioned to scope out

the best capabilities and Internet features and a

new look for SOI’s website, “Tax Stats." This

team sought support from members of the

Consultants’ Panel as resources in making

improvements to the website. One initial task

was to visit the 60+ websites listed in

FEDSTATS, the Federal agencies' primary

source point for statistics, to scope out best

practices and then broaden the search. The

SOI goal is to implement the group’s

proposals by redesigning the SOI website.

This team currently consists of nine SOI staff

members and three additional Panel members

who are familiar with tools, capabilities, and

features of state-of-the-art websites to help this

effort achieve SOI’s goal of making Tax Stats

the best website in its class.

2.

Evaluation of SOI table content and

publications.--SOI has a long history of

publishing since its original mandate in 1916.

Today, the Division publishes the quarterly

Statistics of Income Bulletin, the annual

Individual and Corporation complete reports,

the annual Corporation Source Book, the

annual report in the Methodology series, and

the annual IRS Data Book. In addition, SOI

publishes periodic compendiums and, most

recently, the proceedings for the newly

established annual IRS Research Conference.

A tremendous amount of time and effort goes

into publishing these reports, but considerably

less time has been spent evaluating the

content, frequency, and dissemination of the

publications.

Some of the tasks that a

subgroup plans to undertake are: review

3.

Research in estates, gift, and wealth.--The

focus of research in the estate, gift, and wealth

areas, including SOI support of the Federal

Reserve Board’s Survey of Consumer

Finances (SCF), is closely tied to the needs of

the Office of Tax Analysis and the Joint

Committee on Taxation.

For the SCF, a

contractual agreement between the Federal

Reserve Board and SOI regulates the use of

administrative data and protects individuals

from disclosure of their financial and tax data.

However, it is beneficial to review the scope

and direction, as well as the item content, of

these areas of research. One Panel member

already works with SOI staff members on the

estate, gift, and wealth team. Three additional

Panel members have recently joined this group

to look at the recent body of work in these

areas and help provide insights to SOI on the

focus of this work.

The group is also

interested in exploring innovative ways to

make these data more valuable and more

available, not only to Treasury and the Joint

Committee, but also to outside users.

4.

Teaching

and

training

SOI

staff.-Periodically, SOI has hosted invited speakers

describing the importance of SOI data and

how they use them in forecasting and

economic or policy analysis. The two areas

that have done this more systematically are

SOI’s new employee orientations and

infrequent formal training classes. Concerning

new employee organization, some years ago, a

series of a dozen orientation briefings was

developed for all new employees that

concluded with presentations by SOI's

principal external customers at Treasury, the

Joint Committee on Taxation, and the Bureau

of Economic Analysis.

Although new

employees may not have been ready to grasp

the complexities of the work of these agencies,

other SOI employees asked to attend. Also in

the early 1990's, a 10-part course in Public

Finance was designed and taught by invited

senior public policy analysts (including some

Panel members) to lecture on their work in

public finance practice and how it relates to

theory. What this group seeks is to add Panel

members to a new inhouse staff, initially to

periodically brief SOI on policy analysis using

tax and other microdata. Next, working with

SOI staff, training needs in tax law, policy

analysis, the Federal statistical system, and

statistical project management will be

assessed, and inhouse training modules and

short courses will be developed.

With the four new subgroups, long-term plans

are to encourage frequent, periodic meetings, as

needed, of the subgroups but to host semiannual

meetings for the entire Panel. As requested by

some members, periodic reports from the

subgroups will be distributed to all Panel members

at least once between the semiannual meetings.

The general Panel meetings have traditionally been

open to the public and widely announced.

However, as the workings of the small subgroups

progress, some thought is being given to restricting

the general meeting, which is open to the public, to

once per year so that more attention can be given to

improvements to SOI's internal operations and policies.

Summary and Conclusions

The SOI Consultants’ Panel is a capable and

energetic group of distinguished tax scholars, policy

analysts, academics, and researchers in the public and

private sectors who have generously offered their

assistance to improve SOI operations. Not to accept

their offer of assistance would be a travesty. This paper

is an interim report on how this work has been

structured, with an expectation of tangible benefits in

the not-too-distant future. As it learns of these initial

attempts, SOI plans to refocus the talents of the Panel

members to other aspects of SOI operations.

Notes and References

[1] Eldridge, Marie D., “The Status of Advisory

Committees to the Federal Statistical Agencies,” The

American Statistician, May 1990, Volume 44, Number

2, pp. 154-162.

[2] Petska, Tom, “Statistics on Federal Taxation: The

Statistics of Income Program of the IRS,” Turning

Administrative Systems Into Information Systems:

1994, Internal Revenue Service, 1995.