Customer Satisfaction Initiatives at IRS’s Statistics of Income:
Agency decision
Ask Donna
What actually matters in this document.
Text
Customer Satisfaction Initiatives at IRS’s Statistics of Income:
Using Surveys To Improve Customer Service
Ruth Schwartz and Beth Kilss, Internal Revenue Service, Statistics of Income Division
Key Words:
Customer satisfaction, survey results
Introduction
IRS’s Statistics of Income (SOI) Division conducts statistical
studies on the operations of tax laws and publishes annual
reports, including the quarterly SOI Bulletin, which includes
statistics produced from tax and information returns. SOI’s
Statistical Information Services (SIS) office responds to
thousands of data and information requests annually by
providing SOI data along with technical assistance. To ensure
that customer needs are being met through the SIS office and
through its flagship publication, SOI has been measuring
customer satisfaction for both via customer satisfaction
surveys. These surveys are part of SOI’s commitment to use
survey results to improve customer service. This paper will
focus on three aspects of these surveys: the process by which
we surveyed our customers, the findings from the surveys, and
the steps we are taking to use the results to further improve
our products and services.
In the first section of the paper, background information
on the SOI Division and its SIS office will be presented. The
second section will describe the methodology used to survey
SIS customers, present selected findings from the past 4 years
of surveys, and describe how SOI is using these results to
identify areas for improvement. Similarly, the third section
will describe the methodology, present a summary of the
findings, and briefly discuss some of the steps that SOI staff
are taking to improve the SOI Bulletin. Finally, next steps to
improve SOI products and services in response to survey
findings will be discussed.
Background
Congress created the Statistics of Income Division 90 years
ago in the Revenue Act of 1916, some 3 years after the
enactment of the modern income tax in 1913. Since that time,
the Internal Revenue Code has included virtually the same
language mandating the preparation of statistics. Section 6108
of the Code currently states that “…the Secretary (of the
Treasury) shall prepare and publish not less than annually
statistics reasonably available with respect to the operations of
the internal revenue laws, including classifications of
taxpayers and of income, the amounts claimed or allowed as
deductions, exemptions, and credits and other facts deemed
pertinent and valuable.”
SOI’s mission is to collect, analyze, and disseminate
information on Federal taxation for the Office of Tax
Analysis, Congressional committees, the Internal Revenue
Service in its administration of the tax laws, other
organizations engaged in economic and financial analysis, and
the general public. Its mission is similar to that of other
Federal statistical agencies--that is, to collect and process data
so that they become useful and meaningful information.
However, SOI collects data from tax returns rather than
through surveys, as do most other statistical agencies. These
data are processed and provided to customers in the form of
tabulations or microdata files. Although the IRS uses SOI
data, the primary uses for SOI data are outside of IRS, in
policy analyses designed to study the effects of new or
proposed tax laws and in evaluating the functioning of the
U.S. economy.
SOI Products and Services
Throughout its long history, SOI’s main emphasis has been
individual and corporation income tax information. SOI
began publishing data with the 1916 Statistics of Income,
which reported individual and corporation statistics.
Beginning in 1936, for Tax Year 1934, individual and
corporation income taxes are each reported separately in
annual “complete” reports (Individual Income Tax Returns and
Corporation Income Tax Returns, respectively). The annual
Corporation Source Book provides detailed balance sheet,
income statement, and tax information for major and minor
industry sectors by asset size. Over the years, SOI has
increased its studies and publications to meet the needs of its
customers. Introduced in 1981, the SOI flagship quarterly
Statistics of Income Bulletin presents the most recent data and
related articles on completed studies and a historical section
featuring time series data on a variety of tax-related subjects.
SOI also periodically publishes compendiums of research
on nonprofit organizations, estate taxation, and personal
wealth.
Research articles presented at professional
conferences, namely the American Statistical Association and
the National Tax Association, are published annually or
biannually in the methodology report series, Special Studies in
Federal Tax Statistics. Beginning with the 1998 issue, SOI
took over publishing the IRS Data Book, a fiscal year report
that presents statistical data on the administration of the U.S.
tax system.
SOI produces the following microdata files: Individual
Public-Use Files; Exempt Organizations Records; and Private
Foundations (and Charitable Trusts) records, all of which are
available for a fee. Before release of the Individual PublicUse microdata, SOI follows security guidelines and edits the
files to protect the confidentiality of individual taxpayers to
prevent disclosure of taxpayer information. Tax returns for
both the exempt organizations and private foundations are
publicly available. Because of their size, these products are
available on a CD-ROM or magnetic tape directly from SOI.
Exempt organization microdata files have recently been
released to the public via the World Wide Web
(www.irs.gov/taxstats).
Public awareness of SOI products and easy access to them
have gradually increased over the years. The establishment of
the Statistical Information Services office that responds to data
and information requests has helped raise the visibility of SOI
products. With the introduction of the IRS World Wide Web
10 years ago, SOI’s products became more widely used. They
may be found at: www.irs.gov/taxstats. Tax Stats includes
statistics for individuals, businesses, charitable and exempt
organizations, IRS operations, budget, compliance, and a
variety of other topics. Currently, over 6,000 files reside on
Tax Stats, and this number continues to increase.
Statistical Information Services
The Statistical Information Services (SIS) office was
established in 1989 as part of efforts to streamline the SOI
organization. From the beginning, the SIS mission was
straightforward: Provide accurate and timely data along with
excellent customer support and technical guidance. Although
the number of customers and variety of requests have changed
since then, the SIS staff still strives to fulfill this mission after
17 years.
When the SIS office was set up, a telephone, paper reports
and publications, index cards with contact information, and a
fax machine were its primary tools. Word spread quickly, and,
soon, the SIS office was inundated with requests, many of
whose answers were readily available from published data.
When customer requests involved data unavailable from SOI,
the SIS staff made every effort to fulfill requests by providing
information or contacts from other sources. In the early years
of SIS operations, 4,000 to 5,000 information requests were
received annually. Over the years, the tools have been greatly
improved, and more data are readily available directly to the
public. An electronic management system--the Response
Processing System (RPS)--tracks customer information and
details of data requests. While the number of information
requests has leveled off with the availability of data on Tax
Stats, the complexity of information requests has increased
significantly. Many of these requests require extensive
research, some supported by SOI subject-matter analysts.
Over 2,400 information requests were received in
Calendar Year 2005 from a broad range of customers. The
customers are as widely varied as the information they
request, from a private citizen requesting data on car
dealerships to a Congressional request for alternative
minimum tax data. Consultants and researchers were the
largest group with 23.5 percent of the requests. Academia and
the Internal Revenue Service were the second and third largest
groups with 13.5 percent and 12.9 percent, respectively. In
Calendar Year 2005, most information requests (50.4 percent)
were received by phone, followed closely by 48.2 percent
received by e-mail. The SIS office also receives information
requests via fax, letters, and walk-in customers.
SIS Customer Satisfaction Survey
How is the SIS office meeting its goal of providing accurate
and timely data along with excellent customer support and
technical guidance? Although the SIS office has received
positive feedback from many of its customers over the years,
is this the complete picture? What about the many SIS
customers, especially one-time customers, who do not provide
any feedback? In 2003, at the suggestion of SOI Director
Tom Petska, the SIS office administered its first survey to
measure customer satisfaction. Prior to the SIS survey, SOI
surveyed its primary customers (Treasury’s Office of Tax
Analysis, the Congressional Joint Committee on Taxation, and
the Department of Commerce’s Bureau of Economic
Analysis). The SIS survey was an expansion of SOI’s efforts
to measure customer satisfaction and to use customer input to
improve service.
Administering the Survey
SOI mathematical statistician Kevin Cecco and, later, Diane
Milleville, in close consultation with the SIS staff, designed
the SIS surveys. Following the Office of Management and
Budget’s approval, the first SIS survey was administered in
2003. After assisting the customer with an inquiry, an SIS
staff member provided a survey by e-mail or fax and asked for
the customer to complete the survey related to the customer’s
most recent inquiry.
For the first survey in 2003, the survey recipients were
selected randomly from the daily roster of calls and e-mails.
The SIS office planned to survey one of every four customers
from January through July 2003. However, the target number
of customers surveyed was not reached in July, and the survey
was extended an additional month.
Over the years, changes were made to improve the survey
administration process. Diane Milleville and Information
Technology Specialist Elizabeth Nelson, who provides RPS
technical support, both helped improve the process. Surveys
were imbedded in an e-mail, thus eliminating the additional
step of downloading the survey file. Every customer was sent
a survey, eliminating difficulties with the random selection
process. Customers surveyed were tracked in RPS, which
eliminated the need for SIS staff to manually track them.
Beginning with the 2004 survey, response options were
revised to bring the SIS survey in line with a set of measures
used by SOI’s parent organization, Research, Analysis, and
Statistics (RAS). Known as “balanced measures,” these
criteria were designed to measure how well RAS meets its
goals. To maintain consistent measures throughout all
divisions of RAS, including SOI, some SIS survey questions
and response options were changed to include these measures.
Findings
Table 1 highlights response rates for the 4 years the SIS
survey was administered. Initially, the SIS office’s goal was
to achieve a response rate of 50 percent. SIS planned to survey
approximately 400 customers with the expectation that it
would receive 200 responses. Although SIS fell short of
distributing 400 surveys by 28 percent, it was quite satisfied
with the 49-percent response rate. However, after the first
survey in 2003, the response rate dropped 7 percentage points
in 2004, but has increased to 44 percent in 2006. The number
of Government surveys sent to customers has increased over
the years, and this may also contribute to the declining
response rate. Although SIS would like to have a higher
response rate, it is pleased with its results to date. However, it
will continue efforts to improve its survey instrument and its
methods for administering it.
Table 1--Response Rates for SIS Survey, 2003-2006
Surveys
Number of
Response
Year
distributed
respondents
rate
2003
288
142
49%
2004
425
181
43%
2005
300
125
42%
2006
271
119
44%
Table 2 presents the respondents by job function for each
of the 4 years the survey was administered. For 2003, 2004,
and 2006, the top 4 categories – consultant/research,
State/local government, academic, and IRS employee
(excluding those classified as “other”) accounted for over 57
percent of survey respondents. For 2005, some 3 of the top 4
categories were the same; Federal Government replaced
State/local government as the fourth category. Collectively,
these accounted for 53.2 percent of survey respondents. In an
effort to improve the SIS customer job function categories,
some changes were made during the 4 years the survey was
administered. In 2004, the nonprofit category was added. In
2006, the library, marketing, and realtor categories were
substituted for the corporation category which was eliminated.
There were some differences noted between job functions
reported by SIS survey respondents and the general population
of SIS customers. SIS compared responses for job function
reported by survey respondents and recorded by SIS staff in
the Response Processing System (RPS) during the time period
in which the SIS surveys were administered. Overall, the
differences were generally small for most job categories. An
exception was the private citizen category, which ranged from
12 to 2 percentage points higher (for 2003 and 2006,
respectively) in RPS than in survey responses. These
differences may be a function of respondents’ selfclassification versus classification by an SIS staff member.
Table 2--Percentage Distribution of SIS Survey
Respondents by Job Function, 2003-2006
Year
Job
function
2003
2004
2005
2006
Total
100.0 100.0 100.0
100.0
Consultant/Research
19.4
17.8
15.3
17.1
State/Local Government
14.4
13.9
8.1
10.1
Academic
13.7
13.3
13.7
15.1
IRS employee
10.1
12.8
12.9
14.2
Media
7.9
5.6
5.7
5.0
Corporation
7.2
8.3
10.5
n.a.
Federal Government
5.8
7.2
11.3
7.5
Private citizen
4.3
6.7
2.4
3.4
Accounting firm
2.9
1.7
3.2
5.0
Association/Society
2.2
0.6
1.6
0.0
Congress
0.7
1.1
0.8
2.6
Law firm
--
2.8
0.8
1.7
Nonprofit
n.a.
4.4
4.8
5.9
Library
n.a.
n.a.
n.a.
5.9
Marketing
n.a.
n.a.
n.a.
1.7
Realtor
n.a.
n.a.
n.a.
1.7
Other
11.5
3.9
8.9
2.5
Tax Preparation/
n.a. -- not available
The first survey was designed with 17 questions in 2003.
Over the 4 years, some questions were removed, while others
were added. Overall, the number of questions decreased to a
total of 12 for the 2006 survey (see Appendix). Survey
questions focusing on 3 issues are discussed below.
Table 3 presents the customer’s expectation of timeliness
for receiving a response to an information request in the 2003
survey and actual timeliness in response to questions for the
2004-2006 surveys. Note that the 2003 question is different
from the question included in the 2004-2006 surveys. The
2003 question asks when the customer expected to receive a
response, but the 2004-2006 question asks when a response
was received. Response options for all 4 years are the same.
By changing the wording of the question, SIS was able to
obtain more useful information from its customers. The
expected response time (in the 2003 survey) was significantly
greater than the actual response time (in the 2004 survey.)
Some 36 percent expected a response on the same business
day in the 2003 survey. However, over 70 percent actually
received their responses on the same business day (in the 2004
survey). For 2004 through 2006, a response was received in 3
business days or less 93 percent to 96 percent of the time.
SIS compared the response time for survey respondents to
the response time recorded in RPS by SIS staff using the time
period that SIS surveys were administered in 2004-2006.
Response time of 1 day or less reported by survey respondents
ranged from 74.2 percent to 62.3 percent (for 2005 and 2006
respectively). In contrast, the response time of 1 day or less
reported in RPS was 93.8 percent or higher for 2004-2006.
The SIS staff generally responds to customers within 1
business day as indicated in RPS. However, a completed
request including additional research may take 2-3 days. This
is indicated by 26.1 percent to 30.7 percent of survey
respondents reporting a response time of 2-3 business days.
The response time gap between survey responses and RPS
may be the difference between making an initial contact and
delivering the completed information to the customer.
Table 3--Response Timeliness for SIS, 2003-2006
Percentage of respondents indicating . . .
Survey
Response
In
question
options
2003
When did you
expect to
receive a
response?
Same day
36.0
2-3 business days
52.5
4-5 business days
8.6
6 or more
business days
2.9
Percentage of respondents indicating . . .
Survey
Response
In
question
options
2004 2005
When
did you
receive a
response?
Same day
2-3 business days
4-5 business days
6 or more
business days
2006
70.6
26.1
1.7
74.2
23.4
2.4
62.3
30.7
3.5
1.7
--
3.5
Table 4 presents the issue of meeting customer needs. In
2004, the question and the response options were changed to
reflect the RAS balanced measures. The 2003 question asked
if SOI’s product(s)/data satisfied customer needs. The 20042006 question asks if the product(s) or services(s) provided
met customer needs. The major difference between the 2003
question and the 2004-2006 question is the response options.
In the 2003 survey, there is no option for a “middle ground”
between the “disagree options” and the “agree options.”
Instead, a “not applicable” option is listed at the end after
“strongly agree.” Beginning with the 2004 survey, a “not
sure/neither” option is available between the “disagree
options” and the “agree options.” During the 4 years of the
surveys, the percentage of respondents who agreed or strongly
agreed that their needs were met ranged from 76.5 percent in
2004 to 82.5 percent in 2005.
Response
options
In
2003
Strongly disagree
5.1
Disagree
8.0
Agree
30.4
Strongly agree
51.4
Not applicable
5.1
Percentage of respondents indicating . . .
Survey
Response
In
question
options
2004 2005
SOI's
product(s)/data
satisfied your
needs.
The product(s)
or service(s)
provided met
your needs.
Strongly disagree
Disagree
Not sure/neither
Agree
Strongly agree
6.9
6.9
9.7
33.1
43.4
5.0
5.0
7.5
29.2
53.3
Survey
Response
In
question
options
2003
Rate your overall
Very low
0.7
satisfaction
Low
1.4
with your
Average
10.1
most recent
High
34.8
data request.
Very high
52.9
Percentage of respondents indicating . . .
Survey
question
Rate your overall
satisfaction
with your
most recent
data request.
Table 4--SIS Met Customer Needs, 2003-2006
Percentage of respondents indicating . . .
Survey
question
Table 5--Overall Satisfaction With SIS, 2003-2006
Percentage of respondents indicating . . .
2006
5.3
3.5
12.3
33.3
45.6
Table 5 presents customers’ overall satisfaction with the most
recent response they received from SIS. For all 4 years, the
question was the same, but, beginning with the 2004 survey,
the response options were changed to reflect RAS balanced
measures. Therefore, responses are not comparable between
2003 and the 2004-2006 responses. However, for all 4 years,
the satisfaction rate remained high. Respondents who were
satisfied or very satisfied ranged from 85.9 percent in 2004 to
91.6 percent in 2005.
The surveys each year also included open-ended
questions asking for further explanations, recommendations,
and suggestions for improving service to SIS customers. The
information gleaned from responses to these open-ended
questions has been exceptionally useful. Several respondents
suggested adding the missing years in SOI historical tables,
published in the Statistics of Income Bulletin and also released
on Tax Stats. In these historical tables, the most current 5
years were shown, and, for earlier years, only every fifth year
was shown. Data classified by locality are SIS’s most
Response
options
Totally
dissatisfied
Dissatisfied
Neither
Satisfied
Totally
satisfied
2004
In
2005
2006
0.6
3.5
9.9
41.5
3.4
0.8
4.2
41.2
1.7
2.6
7.8
33.9
44.4
50.4
53.9
frequently requested products. SOI, in conjunction with the
Census Bureau, produces county-to-county and State-to-State
migration data, along with county income data. SOI also
produces Zip Code data. Not surprisingly, respondents
requested more locality data. Some respondents, for example,
requested earned income tax credit and alternative minimum
tax data by county or Zip Code and migration data classified
by occupation. Respondents also requested that locality data
or the Corporation Source Book be made available on Tax
Stats. These products have been available on a reimbursable
basis from SOI.
Changes Planned or Implemented
Based on the input received from SIS customers, the SIS
office has made some changes over the past 3 years. The SIS
office conducted a benchmarking trip to the SIS’s counterpart
at the U.S. Department of Transportation and is looking into
other fact-finding trips. After the first survey was conducted,
the SIS office worked with an Information Technology
Specialist to more effectively track customer requests and
information about its customers.
SOI has also made improvements to its products and
services by eliminating breaks in time series data for many of
its tables. In selected SOI Bulletin historical tables, data for
sequential years are published as space allows. On Tax Stats
where no space limitation exists, SOI is looking into adding
more years of historical data by inserting data for missing
years. SOI has also begun adding more data to Tax Stats.
This year, SOI added the 2000-2003 issues of the Corporation
Source Book.
SOI Bulletin Survey
The SOI Division’s long history of publishing stems from its
original mandate in 1916. Over the years, the number of
publications and the amount of time and effort to publish them
have grown, but considerably less time has been spent
evaluating the content, frequency, and dissemination of the
publications. Three years ago, these tasks were the charge for
a new workgroup that involved senior SOI staff and 3
members of SOI’s Advisory Panel [1]. Initially, this group
undertook to review the content and frequency of all SOI
publications; examine how it could make them more useful;
look at methods of advertising and disseminating; and look at
what it is not publishing that perhaps it should.
Ultimately, the workgroup’s efforts turned to improving
the quality of SOI’s most visible publication--the quarterly
SOI Bulletin--and the efficiency of the Bulletin production
process. Two methods were used--focus groups and a
customer satisfaction survey.
The focus groups were
conducted to learn how authors and reviewers perceive the
writing and review process, and to solicit ideas for changes in
the writing and review process. The customer satisfaction
survey was administered to better understand how SOI
customers use the Bulletin, how satisfied they are with the
contents, how useful the various features of the Bulletin are to
them, and how it should be improved. The remainder of this
section of the paper will be devoted to the Bulletin itself,
describing the survey process, summarizing the key findings,
and, finally, telling how SOI is using the survey results to
improve the publication.
About the SOI Bulletin
Twenty-five years ago, in the summer of 1981, the first issue
of the Statistics of Income Bulletin was published. It was
initially created as the vehicle for disseminating more limited
data on topics formerly covered by separate reports, as well as
to provide the results of the growing number of special
projects. The first SOI Bulletin was 46 pages and included
just 3 articles--on individual income tax returns, sole
proprietorship returns, and partnership returns. Recently, SOI
Division published the 100th Bulletin (Spring 2006, Volume
25, Number 4), which included 6 articles; 23 selected
historical and other data tables; sections on sampling
methodology, projects and contacts, and products and
services; and an index of selected previously published
articles. SOI is currently working on the first issue of its 26th
year (Summer 2006, Volume 26, Number 1). The average
size of the report for 2005 was 310 pages.
Today’s Bulletin is issued quarterly, in March, June,
September, and December and provides the earliest published
annual financial statistics obtained from the various types of
tax and information returns filed, as well as information from
periodic or special analytical studies of particular interest to
students of the U.S. tax system, tax policymakers, and tax
administrators. It also includes personal income and tax data
by State and historical data for selected types of taxpayers, in
addition to data on tax collections and refunds and on other
tax-related items. Much work goes into producing each issue
of the Bulletin, but it was not clear whether it was meeting
customers’ needs. Thus, a survey was designed to collect
critical information on how customers felt about the Bulletin.
Administering the Survey
Once again, SOI Division mathematical statisticians Kevin
Cecco and Diane Milleville were called upon to assist in
developing the survey. The result was a relatively brief and
visually engaging, 15-question customer survey, which was
subsequently cleared for use by the Office of Management and
Budget. Following OMB’s approval, the survey was then
administered to SOI Bulletin customers in several ways.
The survey was sent directly via e-mail to SOI’s main
customers at the Department of Treasury’s Office of Tax
Analysis, the Congress’s Joint Committee on Taxation, and the
Commerce Department’s Bureau of Economic Analysis, as
well as to all members of SOI’s Advisory Panel. The survey
was also included in the Summer 2004 and Fall 2004 issues of
the SOI Bulletin for customers to remove, fill out, and either email or fax back to SOI. As a further outreach to potential SOI
Bulletin customers, an SOI Advisory Panel member facilitated
the dissemination of the survey via the Federation of Tax
Administrators (FTA) list serve in January 2005.
Following a reasonable amount of time after publishing
the Fall 2004 Bulletin and time allowed for FTA members to
reply, the responses were compiled and analyzed. In all, 52
surveys were returned. The majority of respondents were
from the groups SOI targeted outside of the printed
publication. Only 9 respondents filled out the survey from the
Bulletin itself. To put these numbers in perspective, it should
be noted that, for the Fall and Summer issues that year,
approximately 2,000 copies of each were printed. Of these,
about 400 copies were sent to internal IRS and Treasury
Department offices, about 1,250 copies were provided to the
Government Printing Office (GPO) for subscribers and the
Federal Depository Libraries, and about 350 copies were for
the SOI Division for internal purposes. Because just 52
responses were received, a major concern was that responses
might not be representative of all users, meaning this
information should probably not be the basis for any final
decision concerning the Bulletin. Also, it was not possible to
conduct a nonresponse analysis, because the majority of the
Bulletin copies are distributed by the GPO, and SOI does not
know who the customers are. In addition, SOI decided not to
continue to include the survey in subsequent issues of the
Bulletin for several reasons--1) the responses were likely to be
low again; 2) the OMB approval process was required for each
issue of the Bulletin, and, with a low response rate, it would be
more difficult to justify including it in the report; and 3) the
OMB approval process had just become much longer, taking
about 5 weeks instead of 2 weeks. Nevertheless, SOI did have
the results from 52 surveys to evaluate, and, after consulting
with the mathematical statisticians advising us on this effort,
they recommended that SOI work with the results it has and
use another vehicle to focus on a particular part of the Bulletin,
e.g., another focus group, should SOI decide to solicit additional
customer feedback. The findings are presented below.
Findings
Type of respondents. Over one-third of the respondents (36
percent) were affiliated with State and local governments.
Another 18 percent indicated a Federal Government
affiliation, while 17 percent had a Congressional affiliation.
Nearly one-third of all responses came from members of the
FTA list serve.
Use of other SOI products. The three most heavily used SOI
products other than the SOI Bulletin were the Corporation
Source Book, the IRS Data Book, and the Individual complete
report--used by 40 percent-50 percent of all respondents. A
little over one-third of respondents also indicated they used the
Corporation complete report.
About one-fourth of all
respondents use Special Studies in Federal Tax Statistics,
public-use microdata files, and special tabulations. Twenty
percent or less said they use other SOI products.
How respondents receive the Bulletin. Half of all respondents
receive the Bulletin through a subscription. Another 20
percent receive it directly from the SOI Division.
Frequency of use. Of the 49 who responded to how frequently
they use the Bulletin, 37 (about 76 percent) use it 4 times a
year. Only 8 percent use it once a year.
Overall satisfaction. Of the 49 who responded, 86 percent
were satisfied or totally satisfied with the SOI Bulletin; only 2
respondents were dissatisfied, while 5 were neither satisfied
nor dissatisfied.
in both print and on the Web. And of 19 respondents who
answered the question about how often to publish the
historical tables, 11 (or about 58 percent) felt that the
historical section should appear in all SOI Bulletin issues.
Verbatims
The survey also included the following open-ended questions
in order to gain additional information about how the
information in the Bulletin is being used and to seek
recommendations and suggestions for improvements. The
following summarizes the responses SOI received to the openended questions from the survey:
•
About 60 percent of respondents chose to reply.
Verbatim responses covered a number of areas of
uses. A few respondents stated that they use the
Bulletin for “quick look-up of tabulations” or to look
up the most recent data on a topic. One respondent
identified him/herself as a “scholar and educator with
deep interest in the Federal tax system” who reads the
Bulletin for “keeping up” responsibilities. Another
uses the Bulletin as a resource for responding to
media inquiries. The most recurring themes centered
around the Bulletin as a source of data for research
and for the historical series data. About a third of the
answers indicated that the statistics were used for
research, revenue estimation, or tax modeling
purposes. Another 20 percent were mainly interested
specifically in the historical data series that is
included in each issue.
Use of specific features. Of the 8 features listed (from the
Bulletin Board column in the front of the report through the
index on the inside back cover), and checking all that apply,
the Selected Historical and Other Data section was by far the
most frequently used--90 percent of survey respondents,
compared to 67 percent who said they use the featured articles
and 38 percent who use the data releases. An equal number
(about 25 percent of respondents) use each of the remaining
features, except for the Bulletin Board, which less than 8
percent indicated they use.
Suggestions for change. When asked to check boxes
regarding possible changes to the Bulletin, nearly half of all
respondents indicated they would like to see more articles on
topics of current interest. They also indicated an interest in
shorter articles focused on key findings (nearly 37 percent).
About one-fourth of respondents said they would like more
details on methodologies and samples. For the response
“Other,” 8 survey respondents offered varied suggestions,
such as adding links to data and explanatory material on the
Web, including more longitudinal data, and reporting medians
as well as averages and measures of variability.
•
Where to publish historical tables. Nearly 70 percent of those
who use the historical tables felt that they should be published
If you use the Selected Historical and Other Data section
of the SOI Bulletin, which tables do you use, do you find
them useful, do you find the accompanying footnotes
useful, and how would you improve this section?
About half of the 90 percent of survey respondents who
indicated that they use the historical data also told
which tables they use of the 23-table section. The
majority of those use 7 or more tables in the section, and
some specifically stated that they use the annual State
data, a 53-page table titled “Table 2--Individual Income
and Tax Data by State and Size of Adjusted Gross
Income.” About 20 percent of those who use the
historical data also answered the question about whether
they find the tables useful. Several stated they found
them useful as a quick reference, while others stated
they were difficult to find on the Web. Only 1 person
responded to the question about the footnotes, finding
them marginally useful because of the limited number
of years available. Suggested improvements ranged
from only publishing the series once a year to adding
more details on the State table, to including many more
years of data, to more detailed data by State.
How to publish sections: print, Web, or both. This question
dealt with the component parts of an article or data release and
asked respondents whether they preferred the parts to be
provided in print only, posted to the Web only, or to be
available in both places. About two-thirds of respondents
preferred that the tables be provided in both mediums; nearly
half or more than half of respondents indicated that they
preferred most parts of an article to be published in print and
on the Web.
Use of Selected Historical and Other Data section. When
asked if they used the Selected Historical and Other Data
section, some 90 percent said yes. Of those who said yes, over
93 percent said the tables are useful, and over 84 percent said
the footnotes were useful. Of the 2 respondents who answered
no to this question, 1 provided additional comments,
indicating that publishing the historical tables in every issue
was not necessary.
What is your primary use of the SOI Bulletin?
•
If you could change one thing about the SOI Bulletin,
what would it be?
Nearly one-third of respondents chose to weigh in on
this question, and the responses offered a few themes
for SOI to consider--namely, a more detailed index in
order to locate earlier, related articles; more topical,
interesting articles as some are rather dull; providing
links to related, technical documentation on the Web;
and making Bulletin tables electronically useable on
the Web.
•
Please provide any additional comments and/or
suggestions you may have concerning the SOI Bulletin.
Ten responses were received to this question, about
20 percent of those who responded to the survey. No
2 comments were the same, but 1 area for
improvement suggested in several responses was in
length of articles. There appears to be more interest
in the figures, graphs, and tables. Some asked SOI to
consider producing a leaner Bulletin, with more
interesting writing.
Next Steps
Although the number of responses to the SOI Bulletin Survey
was less than had been hoped for, SOI feels that the results are
a strong indication that it is doing a good job of producing the
SOI Bulletin. It is a useful resource for looking up data on a
specific tax-related topic. The historical data are very useful
and an important reason why people use the Bulletin.
However, it is also clear that there is room for improvement in
a number of areas--in improving the writing, e.g., preparing
shorter articles focused on key findings and preparing more
articles on topics of current interest. Many customers are also
interested in more details on methodologies and samples. And
another message that came through is an interest in more
consecutive years of historical data.
These results, along with the results from focus groups
with Bulletin authors and technical reviewers, are being used
to focus SOI efforts on specific areas of improvement.
Recently, SOI has been working with some of the members of
SOI’s Web Modernization Team with the goal of improving
the process of producing and posting tables to the Tax Stats
Web site, which should also improve the process of producing
Bulletin articles. One outcome in streamlining this part of the
Bulletin production process is that we are making data
available earlier on Tax Stats. The Tax Stats Web Team is
also working with a contractor on a dynamic tables prototype
that will allow users to make their own tables from previously
tabulated SOI data. Currently, this is a prototype that allows
users to make tables from 2 years of Corporation Source Book
data. The prototype will run for 4 months, after which SOI
will evaluate feedback, costs, etc., to determine how this will
fit into SOI’s data dissemination strategy.
SOI also plans to address Bulletin content issues.
Working more closely with managers, authors might want to
refresh their articles by shortening them, by becoming more
familiar with relevant tax and economic literature, by
soliciting ideas from senior staff from Treasury’s Office of
Tax Analysis and other customers, and by co-authoring
articles with senior staff or outside experts. SOI will seek to
assist authors in accessing the tax and economic literature by
establishing an electronic index of the SOI library and
arranging a briefing on electronic research from a sister
organization in IRS. SOI will also assemble a collection of
examples of good Bulletin articles and other descriptive papers
to aid newer authors.
SOI will continue to work on improvements to the
Bulletin, as evidenced by current efforts to get consensus from
our senior managers on a plan to improve the Bulletin
production process, followed by incremental improvements in
content and quality of the articles and tables. In so doing, SOI
is committed to responding to the recommendations and
suggestions of customers.
Summary and Conclusion
As discussed, the Statistics of Income Division is using surveys to
improve the methods of conducting business, with the emphasis on
providing top-quality service to its customers. The SIS Survey
questions dealt with communication, characteristics of staff, opinions
of products, and overall satisfaction. When surveying SOI Bulletin
customers, questions dealt with characteristics of the customer and
their use of this publication, content issues, suggestions for
improvement, and overall satisfaction. Administering surveys and
examining the findings over the past several years have shown SOI
how well it is doing in improving products and services and have
helped guide efforts to make improvements in these areas. For both
the SOI Bulletin and SIS surveys, specific suggestions included in
verbatims related to SOI current products have been particularly
useful. The Statistical Information Services office has definitely
benefited from the surveys over the past 3 years. The SIS survey has
helped maintain focus on the SIS goal of outstanding customer
service. To continue to improve its service, the SIS made a
benchmarking trip and is looking into other fact-finding trips. The
SIS office also made enhancements to its electronic tracking
system (RPS) to more effectively track requests as well as
information about its customers. Overall, the responses
received from the SOI Bulletin Survey have been useful in
helping direct current efforts to improve the Bulletin. For
example, it is clear that SOI customers want to continue to
have Historical and other data tables available in both the
printed publication and on SOI’s Tax Stats Web site. SOI
staff are currently working on guidelines for making tables
more usable for customers who intend to download and work
with the data SOI provides. In addition, SOI is working on
improving the publication process itself as well as desktop
publishing tools to improve the layout process. It also intends
to work with subject-matter experts and mathematical
statisticians on content issues, e.g., including more articles on
topics of current interest and more information about the
statistical significance of reported trends, especially when the
reported changes are small in magnitude.
Measuring customer satisfaction will continue to be a
major priority for SOI. A commitment to collecting and
evaluating customer satisfaction data will ensure that SOI does
not lose its focus on critical issues that impact its customers.
An emphasis on collecting customer satisfaction data will
reinforce the SOI culture of providing outstanding service to
customers. As is evident from the data presented in this paper,
SOI has done a good job of exceeding the expectations of its
customers. However, SOI should not rest on its successes, but
rather work even harder to ensure that it meets or exceeds
customer expectations.
Reference
[1] “Recent Efforts To Maximize Benefits From the Statistics
of Income Advisory Panel,” by Tom Petska and Beth Kilss,
Special Studies in Federal Tax Statistics: 2003, Internal
Revenue Service, pp. 87-93, 2004.
Appendix -- SIS Survey Questions, 2003-2006
Survey question
Year question included in SIS
survey
2003
2004
2005
2006
Which of the following best describes your function?
X
How did you initially learn about the SOI SIS office?
X
How did you initially learn about the SIS office?
How often do you contact our office?
X
X
X
X
X
X
X
X
X
X
How often do you contact the SIS office?
How did you contact us?
X
Was the first contact with SIS with a (1) person; (2) voice message
Was the voice message (1) informative; (2) user-friendly; (3) okay as is; (4)
needs improvement by _______ .
Did we satisfy your data request? (If only partially or not at all, please
explain why in the space provided below.)
X
X
X
Did the SIS satisfy your data request?
Did the SIS satisfy your data request? (If only partially or not at all, please
explain why in the space provided below.)
When did you expect to receive a response from us?
X
X
X
X
When did you receive a response?
X
X
When did you receive a response regarding your most recent data request?
X
How did we respond to your data request?
X
Our staff was focused on determining and satisfying your needs.
X
The SIS staff was focused on determining and satisfying your needs.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Please rate your overall satisfaction with your most recent data request.
If you could change one thing about your experience with the SIS office,
what would it be?
X
X
X
X
Please list any other Web sites that you use to gather statistical information.
Please provide comments and/or suggestions on ways we may better serve
your data needs.
X
SOI's product(s)/data satisfied your needs.
X
The product(s) or services (s) provided met your needs.
SOI's product(s)/data was received timely.
X
How often do you retrieve data from the SOI Tax Stats Web site?
X
The SOI Tax Stats Web site is user-friendly.
X
The SOI Tax Stats Web site is user-friendly. Why or why not?
The Tax Stats Web site would be more useful if SOI considered the
following (1) adding more data; (2) deleting data; (3) adding links to other
data; (4) having a sophisticated search engine; (5) allowing "create your
own" tables; (6) adding more viewable tables; (7) other.
X
The information from the SOI Tax Stats Web site met your needs.
If you could change one thing about the SOI Tax Stats Web site, what
would it be?
How would you prefer to receive products/files from SOI?
If given the opportunity, would you be interested in receiving notice of
future data/product releases from SOI?
What types of new products/data releases would you be most interested in
receiving?
X
X
X
X
X
X
X
X
This is a copy of a public record, reproduced as it was published. It is not legal advice, and it may not be the version a court would rely on. Check the official source before you cite it.