Local Research Partnerships for Early Head Start Programs: Availability of Funds and Request for Applications
Federal RegisterDec 21, 1995
Ask Donna
What actually matters in this document.
Text
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Administration on Children and Families
Local Research Partnerships for Early Head Start Programs:
Availability of Funds and Request for Applications
AGENCY: Head Start Bureau, Administration on Children, Youth and
Families (ACYF), Administration for Children and Families (ACF).
ACTION: Availability of funds and request for applications to conduct
research in Early Head Start programs.
-----------------------------------------------------------------------
SUMMARY: The Head Start Bureau, Administration on Children, Youth and
Families has recently awarded grants to provide comprehensive services
to families with infants and toddlers. A cross-site evaluation of a
subsample of the total 68 Early Head Start programs will be performed
by Mathematica Policy Research Institute which was designated as the
National Early Head Start Evaluation contractor. Additional site-
specific research will be conducted by research partners who reside in
or near the same subset of Early Head Start programs and will attempt
to determine the interrelationships of child, family, program and
community variables and program outcomes (local research). This
announcement describes the requirements to be met by applicants seeking
to conduct the local research.
DATES: The closing time and date for receipt of applications is 5 p.m.
(Eastern Time Zone) February 20, 1996. Applications received after 5
p.m. will be classified as late.
ADDRESSES: Mail applications to: Early Head Start Local Research,
Department of Health and Human Services, ACF/Division of Discretionary
Grants, 6th floor, 370 L'Enfant Promenade, SW, Washington, DC 20447,
Mail Stop 6c-462, Attn: Application for Early Head Start Local
Research.
[[Page 66277]]
Hand Delivered, Courier or Overnight Delivery applications are
accepted during the normal working hours of 8 a.m. to 5 p.m., Monday
through Friday, on or prior to the established closing date at: Program
Announcement: ACYF/HS, Administration for Children and Families,
Division of Discretionary Grants, ACF Mailroom, 2nd Floor Loading Dock,
Aerospace Center, 901 D Street, SW, Washington, DC 20024.
FOR FURTHER INFORMATION CONTACT: The ACYF Operations Center, Technical
Assistance Team (1-800-351-2293), is available to answer questions
regarding application requirements and to refer you to the appropriate
contact person in ACYF for programmatic questions.
In order to determine the number of expert reviewers that will be
necessary, if you are going to submit an application, please send a
post card with or call in the following information: the name, address,
and telephone and fax number of the contact person and the name of the
organization four weeks prior to the submission deadline date to:
Administration on Children, Youth and Families Operations Center,
Ellsworth Associates, Inc., 3030 Clarendon Blvd., Suite 240, Arlington,
VA 22201, (1-800-351-2293).
If you decide to submit after the notification date, you may still
submit a proposal.
Part I. General Information
A. Table of Contents
This announcement is divided into four parts, plus appendices:
Part I provides information on the purpose of the local research
effort and a discussion of issues particularly relevant to the local
research under this announcement.
A. Table of Contents
B. Definitions
C. Purpose
D. Background
E. Local Research Studies
Part II contains key information such as eligible applicants,
project periods, special conditions and other information.
A. Statutory Authority
B. Eligible Applicants
C. Special Conditions
D. Cooperative Agreements
E. Project Duration and Federal Share
Part III presents the criteria upon which the proposals will be
reviewed and evaluated.
A. Criteria
B. Review Process
Part IV contains information for preparing the fiscal year
1996 application.
A. Availability of Forms
B. Proposal limits
C. Check List for a Complete Application
D. Due Date
E. Paperwork Reduction Act of 1980
F. Required Notification of State Single Point of Contact
Appendix A contains a list of the Early Head Start grantees.
Appendix B-1 contains the Request for Proposal for contract to
conduct the evaluation of Early Head Start as originally published and
now funded.
Appendix B-2 contains the tentative measures proposed for the
cross-site evaluation.
Appendix C includes the relevant forms necessary for completing the
application.
B. Definitions
Research Partner: The initial university or non-profit organization
designated in the Early Head Start grantee's proposal or a university
or non-profit organization which formed a partnership with an Early
Head Start grantee for the purpose of conducting the research under
this announcement after the Early Head Start grant was awarded.
Cooperative Agreement: A cooperative agreement is a funding
mechanism which allows substantial Federal involvement in the
activities undertaken with Federal financial support. Details of the
responsibilities, relationships, and governance of the cooperative
agreement will be spelled out in the terms and conditions of the award.
The specific responsibilities of the Federal staff and grantee staff
are tentatively listed in Part II-D and will be agreed upon prior to
the award of each cooperative agreement.
C. Purpose
The purpose of this announcement is to invite universities and non-
profit organizations who agree to be the research partners of Early
Head Start program grantees to submit proposals for competitive
Cooperative Agreements to (1) conduct local research studies on issues
related to Early Head Start which will enrich and expand the National
Early Head Start Research and Evaluation Study and benefit the field,
and (2) establish the foundation for a possible longitudinal study of
the mediating and moderating influences on the developmental progress
of Early Head Start and Head Start children and families.
D. Background
On March 17, 1995, ACF announced the availability of funds on a
competitive basis for Early Head Start Programs. Sixty-eight applicants
were successful and became Early Head Start grantees on the effective
date of September 30, 1995. (Applicants should be familiar with this
document in order to prepare a responsive proposal. Copies of this
announcement are available from the Technical Assistance Team at (1-800
351-2293.) Along with the development of the program specifications for
Early Head Start, ACYF designed a set of research and evaluation
initiatives to establish the efficacy of the Early Head Start program
and to contribute new knowledge to the field on factors which influence
the developmental progress of low-income infant and toddlers and their
families. The plan for the Early Head Start research and evaluation
activities is based on the premise that the first set of Early Head
Start programs are prototypes of the Early Head Start concept of state-
of-the-art services for families with infants and toddlers. They will
operate during a period which will almost assuredly see major social
reforms and reconfigurations in services including welfare, health and
child care. Therefore, the lessons learned and the models that will be
developed will shape the direction of services for families with
infants and toddlers well into the 21st century. The plan features (1)
a dynamic and iterative formative evaluation process, designed to be
used in subsequent Early Head Start programs, that will serve as the
instrument for continuous program improvement; (2) an impact evaluation
to determine whether and under what conditions program prototypes were
effective, and (3) an integrated research base consisting of local
research studies as well as the cross-site study for generating further
hypotheses around a broad array of potential development and service
issues and possibilities; and (4) an intended longitudinal study of
both Early Head Start and Head Start.
On May 19, 1995, the first phase of the competitive award process
for the conduct of the research and evaluation activities was initiated
as a Request for Proposal for a national contractor to perform the
cross-site evaluation of Early Head Start (Appendix B-1). The contract
resulting from the competition for the national contractor was awarded
to Mathematica Policy Research, Inc. This announcement is the second
phase, in which cooperative agreements will be awarded on a competitive
basis to the research partners of Early Head Start grantees to conduct
local research studies.
[[Page 66278]]
E. Local Research Studies
1. Local Research Under This Announcement
a. Role of Local Researchers
Under the total Early Head Start Research and Evaluation effort
local researchers will have two significant roles:
(1) Under this announcement they will conduct research relevant to
the issues addressed in section E.2.b. below; and (2) under a
subcontract to the national contractor they will be responsible for the
collection of data for the cross-site study. The local researchers will
form a consortium with the other local researchers and the national
contractor to insure that all the parts of this study form a cohesive
whole. A Technical Review Panel will by appointed by ACYF to review all
the research and evaluation efforts as a whole and provide additional
input. (See Appendix B-1 for a more detailed description.) In order to
ensure the minimum of intrusion for the Early Head Start programs and
to ensure a cohesive study, no applicants will be considered for an
award under this announcement unless they agree to serve as
subcontractors to the national contractor.
b. Concepts
Local research studies are intended to supplement, complement and
enrich the research that will be conducted in the cross-site study.
(See Appendix B-1 The Statement of Work for the Cross-site Evaluation
and Appendix B-2 for a list of the tentative measures proposed for the
cross-site evaluation.) With full access to the cross-site data
collected in their respective sites, local investigators will have an
opportunity to explore mediating events or the theoretical pathways
that explain the results that are obtained. In addition, local research
provides an opportunity to identify outcomes, that because of data
constraints, are not explored in the cross-site study or are specific
to an individual site. It also expands the possibilities for multiple
measures of the same construct. Another advantage of local research is
the enhanced opportunity for the use of observational, ethnographic,
case study and other qualitative approaches that inform our
understanding of how the program functions and explain the particular
outcomes that are achieved.
Four outcome domains and specific outcomes under each were
preliminarily identified by the Advisory Committee on Services for
Families with Infants and Toddlers for Early Head Start. Although no
one program is expected to be equally successful across all outcomes,
these outcomes were identified by the Committee as particularly
important for continued child, family and program development.
Child: Health and physical development; social competency; secure
attachments with parents and other caregivers; language and cognitive
development; resiliency factors; benefits to siblings.
Family: Attitudes towards parenting; parent-child interaction;
reduction in teenage pregnancy and positive birth outcomes; having a
medical home; parenting, employability and progress towards self-
sufficiency; training and education; housing; physical and mental
health; substance abuse; home environment; safety; involvement in the
Early Head Start program; knowledge of child development; child
guidance beliefs and practices.
Community: Collaboration among agencies serving children and
families; seamlessness in referrals and actual service provision;
quality of services for children and families; increase in services for
infants and toddlers; safety.
Staff: Staff-parent/child relationships; staff continuity; staff
professional development; staff compensation; staff physical and mental
health; staff qualifications; and staffing patterns.
The major question for the local studies is ``What mediates and
moderates positive child and family development within the context of
the specific Early Head Start program and the local community?'' Each
of the local research studies may focus on variables within one of the
four outcome domains listed above. Positive child and family
development are the ultimate objectives of Early Head Start, and thus,
must have a prominent focus. However, well-designed local research
studies which focus on particular staff or community outcomes will be
considered if their relationship to the well-being of children and
families can be theoretically linked through the existing literature
and investigated within the time frame of the five-year cooperative
agreement. Investigators focusing on the same outcome domain may find
additional opportunities for cooperative research. Depending on the
questions for the local research, investigators may choose or not
choose to incorporate the control group, which will be part of the
cross-site evaluation, in the local research study.
Within the framework of the Early Head Start program design, each
site represents a unique model based upon the needs, values, resources
and cultural climate of its community. Therefore, within the array of
possible outcomes, it is highly likely that each program will place
different emphases among them and work toward additional objectives
that are unique to the particular local site. It is therefore important
for the local research studies to identify site-specific outcomes which
are not explored in the cross-site study and to study intra-site
differential impacts and the reasons for them. The local studies will
enhance the cross-site analysis by the provision of additional
explanatory material for inter-site differences and by the
identification of additional effects of Early Head Start programs. The
first data collection point for the child's developmental status,
attachment, mother/child interaction and other child and family
measures for the cross-site analysis will be around the time of the
child's first birthday. ( See Appendix B-2 for a list of tentative
measures.) If applicants see a need for earlier data collection for
their local research studies, they may propose such data collection
using the same or other measures as part of the local research data
collection and analyses.
c. Study Parameters
--Design
The program sites whose local research partners receive awards
under this announcement will be sites in which both local research and
the national cross-site evaluation are conducted. However, if less than
12 proposals receive an acceptable rating, additional sites may be
selected to participate only in the national cross-site evaluation to
ensure 12 sites for the cross-site effort. The sites with local
researchers will become the potential sites to continue on with the
follow-up longitudinal studies. For the cross-site evaluation, all 12
sites, whether they are additionally local research sites or not, will
be required to participate in random assignment of those families who
have applied to the Early Head Start program and in which there is a
pregnant woman or a child under one year of age. Such families will be
randomly assigned to either the program or control group under a system
designed by the national contractor with participation from the local
researchers. The Early Head Start program must agree to fully cooperate
with the random assignment as a condition for the research partner to
receive an award under this announcement.
As noted above, applicants are not required to utilize the control
group in their local research designs unless the proposed research
questions require such a design. However, since the cost
[[Page 66279]]
of the data collection on a number of child and family measures for the
program and control group will be covered by the national contractor,
the applicant may wish to consider adding an additional sample, such as
a random sample of the Early Head Start eligible population, or other
types of scientifically sound samples. These samples could contribute
valuable information to the Early Head Start research and evaluation
effort and would considerably strengthen longitudinal follow-up
efforts.
--Sample
In order to be considered for an Early Head Start research
cooperative agreement, the applicants must be able to guarantee that
their Early Head Start program partners have the ability to recruit a
minimum of 150 families meeting the designated criteria for the random
assignment pool. Specifically, the families to be recruited must
include a child who is born between June 1, 1995 and June 30,1997 and
must not have had a child enrolled in Head Start and PCC (Parent and
Child Centers) within the last 12 months, or, in the case of CCDP
(Comprehensive Child Development Program), the last five years. Neither
may the families to be recruited have been enrolled in any other
Federal, State or local program with similar comprehensive services for
the last 12 months. Exceptions to these requirements will be considered
on a site by site, or family basis after the research sites have been
selected. (Note: Enrollment in other programs is defined as
participating for a minimum of three months.) The families in the
random assignment pool, as the term implies, will be randomly assigned
to either the program or the control group. (A minimum of 75 in each.)
Therefore the Early Head Start program must have the ability to enroll
a minimum of 75 families who meet the research requirements during the
research recruitment period of March 1, 1996 and June 1, 1998 (27
months). The families must be enrolled some time during the mother's
pregnancy or before the child is one year of age. The Early Head Start
programs will be continuously enrolling families during the course of
their operation. Therefore, the research sample will be an additive
sample rather than a cohort sample. However, no family will be
recruited into the research sample if the child is born before June 1,
1995 or after June 30, 1997. If a research sample family leaves the
program during the 27 month research recruitment period, replacement of
families can only be made within the parameters stated above.
Although there is a 27 month recruitment and enrollment period for
the research sample, sites may wish to use all or part of that period
to recruit the requisite sample in accordance with what works best for
their program. (For example, some programs may not be ready to recruit
or enroll families by March and other programs may wish to enroll the
majority of their research sample families as early as possible.)
However, to ensure that the site will reach the requisite sample size,
the earliest possible enrollment of the full research sample is
encouraged. In addition, any site which anticipates that it can secure
a sample of over 75 program families and 75 control families over the
recruitment period that meet the research criteria, may enroll other
families, in excess of the 75 families, which do not meet the research
criteria. These additional families will not be included in the
research sample. Programs are encouraged, however, to achieve the
largest research sample possible, up to 125 families each for the
program and comparison group. Larger samples would be a major advantage
for any future longitudinal research.
2. Considerations for the Longitudinal Studies
Longitudinal studies beyond the five years of the Early Head Start
research and evaluation effort are outside the scope of the present
announcement. However, it is ACYF's intent to engage in such
longitudinal studies, given availability of future funds and the
feasibility of such efforts in five years time. It will be necessary to
lay the groundwork for such studies from the beginning of the Early
Head Start research and evaluation effort in order to ensure that early
data necessary for the longer effort is collected.
Although the longitudinal studies are related to and embedded in
the Early Head Start Research and Evaluation Study, they have a number
of sufficiently unique considerations to warrant a separate discussion.
a. Eligibility--It is anticipated that universities and non-profit
organizations which receive Early Head Start research grants under this
announcement will be potential candidates for follow-on longitudinal
research grants. The exact number of grants that will be awarded for
the follow-on longitudinal studies will depend on the availability of
funds and other criteria such as the size of the sample left at the
site, the quality of the research conducted to date, and the level of
program implementation.
b. Studies--The longitudinal follow-ons can be conceptualized as
two studies which serve different purposes.
Longitudinal Studies of Early Head Start
Longitudinal studies of Early Head Start will address the
contributions of earlier intervention to the child and family's later
development.
Longitudinal Studies of Head Start
Since Early Head Start programs are required to establish formal
linkages with local Head Start programs in order to provide for the
continuity of services for children and families, the Early Head Start
research sites provide a unique opportunity for the conduct of
longitudinal studies of Head Start. Presently, there are no existing
studies of Head Start where the early service patterns and experiences
of children and families either enrolling in Head Start or serving as
comparisons or controls are known to the extent that they will be known
in Early Head Start. That data will be available, at least for part of
the Head Start population, in the Early Head Start research sites by
the time Longitudinal Studies of Head Start are underway. In addition,
the studies can make progress in addressing the question of whom among
the Head Start population Head Start serves.
Design and sampling issues for both studies will need careful
consideration.
Part II Program Information and Requirements
A. Statutory Authority
The Head Start Act, as amended, 42 U.S.C. 9801 et seq.
B. Eligible Applicants
Universities and other non-profit institutions which have been
designated by the Early Head Start grantees listed in Appendix A as
their research partner for the purposes of the impact evaluation.
A research partner may be the institution identified in the Early
Head Start grantee's proposal or a new or additional research partner
that the Early Head Start grantee has selected for the purposes of
conducting the research under this announcement.
Note: Only one university or non-profit institution per each
Early Head Start grantee may apply. An applicant must be certified
by the Early Head Start grantee as the designated research partner.
In addition, if a university or non-profit institution applies on
behalf of one or more investigators as the research partner of an
Early Head Start grantee, the university or non-profit institution
may only apply as the partner of any other Early Head Start grantee
if applying in behalf of different investigators.
[[Page 66280]]
C. Additional Special Requirements
1. In order to be accepted for review, applications must contain a
letter from the Program Director of the Early Head Start program
certifying that the applicant is the designated research partner of
that program.
2. The proposed local research study must not overlay additional
interventions for children, families or staff which are designed by the
local research partner for research purposes beyond the existing Early
Head Start intervention designed by the program for that site. (For
example, the local research may not investigate research hypotheses
that would require the assignment of families enrolled in the program
to treatment and control groups.
3. Applicants must agree to enter a subcontractual or other
arrangement with the national contractor for the purposes of collecting
the data for the cross-site study and for site-specific analysis of the
cross-site data. The subcontract with the national contractor will be
in addition to the funds received under this announcement and will
primarily consist of providing input to the cross-site design;
supervision of cross-site data collection at the local site; ensuring
quality control; and site specific data analyses of the cross-site
data.
4. Applicants must agree to work in a consortium with the other
local researchers and the national contractor in order to produce an
integrated set of studies.
5. Applicants must present their proposal to and receive approval
from the Early Head Start program policy councils (or other appropriate
policy group) prior to submission.
6. Successful applicants must form a local advisory committee
consisting of staff and parents of the Early Head Start program, other
community agencies and researchers with expertise in areas relevant to
the local research.
7. Applicants' Early Head Start program partner must be able to
recruit and enroll the required number and types of families as
described in Part I, section E-1-c above.
8. The principal investigator and at least one other key research
team member must attend a minimum of one two-day meeting of the local
researchers in Washington, DC in addition to the two-day meetings with
the national contractor and the Technical Review Panel. A third day
will be provided at the national contractor meetings in order for the
local researchers to meet on the issues and coordination of the local
research projects. Successful applicants must also plan to attend Head
Start's Third National Research Conference in Washington, DC June 20-
23, 1996. The applicant will be responsible for all travel expenses
related to these meetings. These travel expenses may be included in the
applicant's budget.
9. Since the research will be conducted at the Early Head Start
program site, applicants must use their off-campus research rates for
indirect costs. If the applicant is a non-profit organization, the
applicant is limited to an indirect cost rate of no more than 15
percent.
10. In submitting an application, the applicant understands that
the data resulting from the local research is the property of the ACYF.
Therefore, a copy of the raw data set with accompanying documentation
must be submitted to the Government in a manner and frequency that will
be specified in consultation with the consortium during the first year
of the cooperative agreement. It is not the intention of the Government
to inhibit or restrict presentations and publications of the results of
the local research by the grantee beyond any publishing restrictions
that will be agreed upon by the Consortium and ACYF.
12. The applicant must provide all required assurances and
certifications including a Protection of Human Subject Assurance as
specified in the policy described on the HHS Form 596 (attached in
Appendix A).
D. Cooperative Agreements
ACYF is utilizing a cooperative agreement mechanism to support
local research as a means of ensuring close cooperation and
coordination between and among local researchers, Early Head Start
programs and the National contractor. Together, these three entities
form the research team. Although the three entities have equal status
on the research team, each has an area of primary responsibility: (1)
The Early Head Start program has primary responsibility for the design
and implementation of program services and activities; (2) the National
contractor has primary responsibility for the cross-site study; and (3)
the local researcher has primary responsibility for the local research
study. In applying for a cooperative agreement under this announcement,
the applicant pledges close cooperation and coordination with the other
research partners.
1. Responsibilities of the Grantee
The Grantee
Conducts a local research study which enhances, enriches
or expands the cross-site data and focuses on one of the four Early
Head Start outcome domains.
Designs and conducts the preliminary research for the
Longitudinal Study of Early Head Start and the Longitudinal Study of
Head Start.
Participates as a member of the consortium of local
researchers and the national contractor.
Conducts local analyses and interpretations of the cross-
site data.
Agrees to enter a subcontract or other financial
arrangement with the national contractor for purposes of collecting
data for the cross-site study.
Agrees to work as a member of the research team consisting
of the Early Head Start program, the national contractor and the local
researcher.
2. Responsibilities of the Federal Staff
Federal Staff
Provide guidance in the development of the final study
design.
Participate as members of the national consortium or any
policy, steering or other working groups established at the consortium
level to facilitate accomplishment of the project goals.
Facilitate communication among consortium members, Early
Head Start grantees and the Federal staff.
Provide logistical support to facilitate meetings of the
local researchers.
E. Project Duration and Federal Share
1. Project Duration
Awards, on a competitive basis, are for a project period of five
years. Continuation applications beyond the first 12 month budget
period, but within the five-year project period, will be entertained in
subsequent years on a non-competitive basis, subject to availability of
funds, satisfactory progress and a determination that continued funding
is in the best interests of the Government.
2. Federal Share of Project Costs
Federal share of project costs shall not exceed $150,000 for the
first 12-month budget period inclusive of indirect costs and shall not
exceed $150,000 for the second and third 12-month budget period. The
Federal share of the fourth and fifth budget period shall be negotiated
prior to the fourth and/or fifth year of funding.
3. Matching Requirements
There is no matching requirement; however, applicants must apply
their indirect cost rate for off-campus
[[Page 66281]]
research or no more than 15 percent for non-profit research
institutions.
4. Anticipated Number of Projects to be Funded
It is anticipated that 12 projects will be funded.
Part III Evaluation Criteria
The criteria presented below will be applied by the reviewers to
the applicants submission in order to select the successful applicants.
ACYF has prepared a document entitled ``Helpful Tips for Preparing a
Successful Research Grant Application.'' This document can be obtained
from the Technical Assistance Team at (1-800-351-2293).
A. Criteria
1. Objectives and Significance 25 points
The extent to which the objectives of the local research
are important and relevant to the overall Early Head Start Research and
Evaluation effort.
The extent to which the local research study makes a
significant contribution to the overall study and to the broader field.
The extent to which the related literature review supports
the study objectives, the questions to be addressed or the hypotheses
to be tested.
The extent to which the questions that will be addressed
or the hypotheses that will be tested are sufficient for meeting the
stated objectives.
2. Approach 40 points
The extent to which the planned approach reflects
sufficient input from and partnership with the Early Head Start
program.
The extent to which the research design is appropriate and
sufficient for addressing the questions of the study.
The extent to which the planned approach allows for the
identification and differentiation of site-specific outcomes.
The extent to which the planned research includes
quantitative and qualitative methods.
The extent to which the planned measures and analyses both
reflect knowledge and use of state-of-the-art measures and analytic
techniques and advance the state-of-the-art.
The adequacy of the anticipated research sample size for
the requirements of the cross-site study and for the local research
study.
The extent to which the site in which the research will be
conducted has a recruitment and enrollment strategy that meets the
requirements set forth in the design section of the announcement.
The extent to which planned site activities are sufficient
preparation for potential longitudinal studies.
The extent to which the applicant's proposals for
resolution of the data collection issues as a result of the two types
of data collection are realistic and feasible.
The applicant has provided all required assurances.
The reasonableness of the budget for the work proposed.
3. Staffing 35 points
The extent to which the principal investigator and other
key research staff possess the research expertise necessary to conduct
the local research including infant/toddler and family development; the
application of advanced statistical analysis for quantitative and
qualitative data; and the use of quantitative and qualitative methods
as demonstrated by the technical portions of the applications and the
information contained in their vitae.
The extent to which the proposed staff reflect an
understanding of and sensitivity to the issues of working in a
community setting and in partnership with program staff and parents.
The extent to which the proposed staff reflect a multi-
disciplinary team.
The adequacy of the time devoted to this project by the
principal investigator and other key staff in order to ensure a high
level of professional input and attention.
The extent to which the staffing is sufficient for
conducting the local research and the data collection and site analysis
of the cross-site evaluation.
B. The Review Process
Applications received by the due date will be reviewed and scored
competitively. Experts in the field, generally persons from outside the
Federal government, will use the evaluation criteria listed in Part III
of this announcement to review and score the applications. The results
of this review are a primary factor in making funding decisions. ACYF
may also solicit comments from ACF Regional Office staff and other
Federal agencies. These comments, along with those of the expert
reviewers, will be considered in making funding decisions. In selecting
successful applicants, consideration may be given to achieving an
equitable distribution among geographic regions of the country and
other considerations necessary to achieve, to the greatest extent
possible, a research and evaluation sample that is representative of
all Early Head Start programs.
Part IV Instructions for Submitting Applications
A. Availability of Forms
Eligible applicants interested in applying for funds must submit a
complete application including the required forms included at the end
of this program announcement in Appendix C. In order to be considered
for a grant under this announcement, an application must be submitted
on the Standard Form 424 (approved by the Office of Management and
Budget under Control Number 0348-0043). A copy has been provided. Each
application must be signed by an individual authorized to act for the
applicant and to assume responsibility for the obligations imposed by
the terms and conditions of the grant award. Applicants requesting
financial assistance for non-construction projects must file the
Standard Form 424B, ``Assurances: Non-Construction Programs'' (approved
by the Office of Management and Budget under control number 0348-0340).
Applicants must sign and return the Standard Form 424B with their
application. Applicants must provide a certification concerning
lobbying. Prior to receiving an award in excess of $100,000, applicants
shall furnish an executed copy of the lobbying certification (approved
by the Office of Management and Budget under control number 0348-0046).
Applicants must sign and return the certification with their
application.
Applicants must make the appropriate certification of their
compliance with the Drug-Free Workplace Act of 1988. By signing and
submitting the application, applicants are providing the certification
and need not mail back the certification with the application.
Applicants must make the appropriate certification that they are
not presently debarred, suspended or otherwise ineligible for award. By
signing and submitting the application, applicants are providing the
certification and need not mail back the certification with the
application.
Applicants must also understand that they will be held accountable
for the smoking prohibition included within Pub.L. 103-227, Part C
Environmental Tobacco Smoke (also known as The Pro-Children's Act of
1994). A copy of the Federal Register notice which implements the
smoking prohibition is included with the forms. By signing and
submitting the application, applicants are providing the certification
and need not mail back the certification with the application.
All applicants for research projects must provide a Protection of
Human
[[Page 66282]]
Subjects Assurance as specified in the policy described on the HHS Form
596 (approved by the Office of Management and Budget under control
number 0925-0137) in Appendix C. If there is a question regarding the
applicability of this assurance, contact the Office for Protection from
Research Risks of the National Institutes of Health at (301)-496-7041.
Those applying for or currently conducting research projects are
further advised of the availability of a Certificate of Confidentiality
through the National Institute of Mental Health of the Department of
Health and Human Services. To obtain more information and to apply for
a Certificate of Confidentiality, contact the Division of Extramural
Activities of the National Institute of Mental Health at (301) 443-
4673.
B. Proposal Limits
The proposal should be double-spaced and single-sided on 8\1/
2\'' x 11'' plain white paper, with 1'' margins on all sides. Use only
a standard size font such as 10 or 12 pitch throughout the
announcement. All pages of the narrative (including appendices,
resumes, charts, references/footnotes, tables, maps and exhibits) must
be sequentially numbered, beginning on the first page after the budget
justification as page number one. Applicants should not submit
reproductions of larger sized paper that is reduced to meet the size
requirement. Applicants are requested not to send pamphlets, brochures,
or other printed material along with their applications as these pose
copying difficulties. These materials, if submitted, will not be
included in the review process. In addition, applicants must not submit
any additional letters of endorsement beyond any that may be required.
The length of the narrative section, including appendices, should
not exceed 60 pages. Anything over 60 pages will be removed and not
considered by the reviewers. Applicants are encouraged to submit
curriculum vita using ``Biographical Sketch'' forms used by some
government agencies.
Please note that applicants that do not comply with the
requirements in the section on ``Eligible Applicants'' will not be
included in the review process.
C. Checklist for a Complete Application
The checklist below is for your use to ensure that the application
package has been properly prepared.
--One original, signed and dated application plus two copies.
--Attachments/Appendices, when included, should be used only to provide
supporting documentation such as resumes, and letters of agreement/
support.
--A complete application consists of the following items in this order:
(1) Application for Federal Assistance (SF 424, REV. 4-88);
(2) Budget information--Non-Construction Programs (SF424A&B REV.4-
88);
(3) Budget Justification, including subcontract agency budgets;
(4) Letter from the Director of the Early Head Start program
certifying that the applicant is the designated research partner of the
respective program;
(5) Application Narrative and Appendices (not to exceed 60 pages);
(6) Proof of non-profit status. Any non-profit organization
submitting an application must submit proof of its non-profit status in
its application at the time of submission. The non-profit organization
can accomplish this by providing a copy of the applicant's listing in
the Internal Revenue Service's (IRS) most recent list of tax-exempt
organizations described in Section 501(c)(3) of the IRS code or by
providing a copy of the currently valid IRS tax exemption certificate,
or by providing a copy of the articles of incorporation bearing the
seal of incorporation of the State in which the corporation or
association is domiciled.
(7) Assurances Non-Construction Programs;
(8) Certification Regarding Lobbying;
(9) Where appropriate, a completed SPOC certification with the date
of SPOC contact entered in line 16, page 1 of the SF 424, REV.4-88;
(10) Certification of Protection of Human Subjects.
D. Due Date for the Receipt of Applications
1. Deadline: Mailed applications shall be considered as meeting an
announced deadline if they are received on or before the deadline time
and date at the U.S. Department of Health and Human Services,
Administration for Children and Families, Division of Discretionary
Grants, 370 L'Enfant Promenade, SW., Mail Stop 6c-462, Washington, DC
20447, Attention: Early Head Start Local Research, Applicants are
responsible for mailing applications well in advance, when using all
mail services, to ensure that the applications are received on or
before the deadline time and date.
Applications handcarried by applicants, applicant couriers, or by
overnight/express mail couriers shall be considered as meeting an
announced deadline if they are received on or before the deadline date,
between the hours of 8 a.m. and 5 p.m., at the U.S. Department of
Health and Human Services, Administration for Children and Families,
Division of Discretionary Grants, ACF Mailroom, 2nd Floor Loading Dock,
Aerospace Center, 901 D Street, SW., Washington, DC 20024, between
Monday and Friday (excluding Federal Holidays). (Applicants are
cautioned that express/overnight mail services do not always deliver as
agreed.) ACF cannot accomodate transmission of applications by fax.
Therefore, applications faxed to ACF will not be accepted regardless of
date or time of submission and time of receipt.
2. Late applications: Applications which do not meet the criteria
above are considered late applications. ACF shall notify each late
applicant that its application will not be considered in the current
competition.
3. Extension of deadlines: ACF may extend the deadline for all
applicants because of acts of God such as floods, hurricanes, etc.,
widespread disruption of the mails or when it is anticipated that many
of the applications will caome from rural or remote areas. However, if
ACF does not extend the deadline for all applicants, it may not waive
or extend the deadline for any applicants.
E. Paperwork Reduction Act of 1980
Under the Paperwork Reduction Act of 1980, Public Law 96-511, the
Department is required to submit to OMB for review and approval any
reporting and record keeping requirements in regulations including
program announcements. This program announcement does not contain
information collection requirements beyond those approved under OMB
Control Numbers 0348-0043, 0348-0044, 0348-0046 and 0925-0137.
F. Required Notification of the State Single Point of Contact
This program is covered under Executive Order 12372,
``Intergovernmental Review of Federal Programs,'' and 45 CFR part 100,
``Intergovernmental Review of Department of Health and Human Services
Program and Activities.'' Under the Order, States may design their own
processes for reviewing and commenting on proposed Federal assistance
under covered programs.
* All States and Territories except Alaska, Colorado, Connecticut,
Hawaii, Idaho, Kansas, Louisiana, Massachusetts, Michigan, Minnesota,
Montana, Nebraska, Oklahoma, Oregon, Pennsylvania, South Dakota,
Tennessee, Virginia, Washington, American Samoa and Palau have elected
to participate in the Executive Order process and have
[[Page 66283]]
established Single Points of Contact (SPOCs). Applicants from these
twenty-one jurisdictions need take no action regarding E.O. 12372.
Applicants for projects to be administered by Federally-recognized
Indian Tribes are also exempt from the requirements of E.O. 12372.
Otherwise, applicants should contact their SPOCs as soon as possible to
alert them of the prospective applications and receive any necessary
instructions. Applicants must submit any required material to the SPOCs
as soon as possible so that the program office can obtain and review
SPOC comments as part of the award process. It is imperative that the
applicant submit all required materials, if any, to the SPOC and
indicate the date of this submittal (or the date of contact if no
submittal is required) on the Standard Form 424, item 16a.
Under 45 CFR 100.8(a)(2), a SPOC has 60 days from the application
deadline to comment on proposed new or competing continuation awards.
SPOCs are encouraged to eliminate the submission of routine
endorsements as official recommendations.
Additionally, SPOCs are requested to clearly differentiate between
mere advisory comments and those official State process recommendations
which may trigger the ``accommodate or explain'' rule.
When comments are submitted directly to ACF, they should be
addressed to: Department of Health and Human Services, Administration
for Children and Families, Division of Discretionary Grants, 370
L'Enfant Promenade, SW., Washington, DC 20447. A list of the Single
Points of Contact for each State and Territory is included as an
Appendix to this announcement.
Dated: December 14, 1995.
Olivia A. Golden,
Commissioner, Administration on Children, Youth and Families.
Appendix A--List of Early Head Start Grantees
Alaska
Rural CAP Child Development, Karen King, P.O. Box 200908, Anchorage, AK
99520-0908, Telephone: (907) 279-2511, Fax: (907) 279-6343, E-mail:
None
Arizona
Southwest Human Development Ginger Ward, 202 E. Earll, Suite 140,
Phoenix, AZ 85012, Telephone: (602) 266-5976, Fax: (602) 274-8952, E-
mail: [email protected]
Arkansas
Child Development Inc., JoAnn Williams, P.O. Box 2110, Russellville, AR
72811, Telephone: (501) 968-6493, Fax: (501) 968-7825, E-mail:
[email protected]
California
The Children First, Manuel Castellanos Jr., Venice Family Clinic, 604
Rose Avenue, Venice, CA 90291, Telephone: (310) 314-7320 x670, Fax:
(310) 314-7641, E-mail: None
Northcoast Children's Services (NCS), Siddiq Kilkenny, P.O. Box 1165,
Arcata, CA 95521, Telephone: (707) 822-7206, Fax: (707) 822-7962, E-
mail: None
Sacramento Employment Training Agency (SETA), Head Start, Catherine
Goins, 3750 Rosin Court, Suite 100, Sacramento, CA 95834, Telephone:
(916) 263-5342, Fax: (916) 263-3779, E-mail: None
Colorado
Clayton Mile High Family Futures Project, Mitzi Kennedy/Adele Phelan,
3801 Martin Luther King Jr. Blvd., Denver, CO 80205, Telephone: (303)
355-2008, Fax: (303) 331-0248, E-mail: None
Community Partnership for Child Development, Terry Schwartz, 2132 E.
Bijou, Colorado Springs, CO 80909, Telephone: (719) 635-1536 x217, Fax:
(719) 634-8086, E-mail: Later date
Family Star, Lereen Castellano/Alicia Sheridan, 1331 E. 33rd Avenue,
Denver, CO 80205, Telephone: (303) 295-7711, Fax: (303) 295-0958, E-
mail: None
District of Columbia
Edward C. Mazique Parent Child Center, Cynthia Faust, 1719-13th Street,
NW, Washington, DC 20009, Telephone: (202) 462-3375, Fax: (202) 939-
8696, E-mail: None
United Cerebral Palsy of Washington and Northern VA (UCP), Stanley L.
Pryor, 3135 Eighth Street, NE, Washington, DC 20017, Telephone: (202)
269-1500, Fax: (202) 526-0519, E-mail: [email protected]
Florida
Alachua County School District, Donna Omer, School Board of Alachua
County, 620 East University Avenue, Gainesville, FL 32601, Telephone:
(904) 955-7605, Fax: (904) 955-6700, E-mail: None
Metro Dade Community Action Agency, Regina M. Grace, 395 NW. 1st
Street, Miami, FL 33128, Telephone: (305) 347-4640, Fax: (305) 372-
8745, E-mail: None
Georgia
Berry Chattooga Early Development Center, Nancy Daniel, 702 South
Congress Street, Summerville, GA 30747, Telephone: (706) 857-1651, Fax:
(706) 857-6610, E-mail: None
Clark Atlanta University Head Start, Linda Hassan, 350 Autumn Lane,
SW., Atlanta, GA 30314, Telephone: (404) 696-9585 x104, Fax: (404) 696-
9524, E-mail: None
Georgia Early Head Start Network, Donna Overcash, Save the Children
Child Care Support Ctr., 1447 Peachtree Street, NE, Suite 700, Atlanta,
GA 30309, Telephone: (404) 885-1578, Fax: (404) 874-7427, E-mail:
[email protected]
Illinois
City of Chicago, Dept. of Human Services, Frank McGehee, 510 North
Peshtigo Court, 8th Floor, Chicago, IL 60611, Telephone: (312) 744-
0251, Fax: (312) 744-7530, E-mail: None
The Ounce of Prevention Fund, Portia Kennel, 188 W. Randolph Street,
#2200, Chicago, IL 60601, Telephone: (312) 853-6080, Fax: (312) 853-
3337, E-mail: None
Wabash Area Development, Inc., Donna Emmons, 100 N. Latham, Enfield, IL
62835, Telephone: (618) 963-2387, Fax: (618) 963-2525, E-mail: None
Indiana
Healthy Beginnings, Hamilton Center, Anita Lascelles, 620 8th Avenue,
Terre Haute, IN 47804, Telephone: (812) 231-8335, Fax: (812) 232-8228,
E-mail: None
Iowa
Mid-Iowa Community Action, Susan Fessler, 1001 South 18th Street,
Marshalltown, IA 50158, Telephone: (515) 752-7162, Fax: (515) 752-9724,
E-mail: None
Upper Des Moines Opportunity, Inc., Mary Jo Madvig, P.O. 519, 101
Robbins Avenue, Graettinger, IA 51342-0519, Telephone: (712) 859-3885,
Fax: (712) 859-3892, E-mail: None
Kansas
Head Start Parent & Child Center, Glenda Wilcox, 931 South St. Francis,
Wichita, KS 67211, Telephone: (316) 267-8314, Fax: (316) 267-7185, E-
mail: None
Project EAGLE of the University of Kansas Medical Center, Martha
Staker, Gateway Centre Tower II, Suite 1001, 4th & State Avenue, Kansas
City, KS 66101, Telephone: (913) 281-2648, Fax: (913) 281-2680, E-mail:
None
[[Page 66284]]
Salina USD #305, Korey Powell-Hensley, 700 Jupiter, Salina, KS 67401,
Telephone: (913) 826-4868, Fax: (913) 826-4867, E-mail: None
Kentucky
Breckinridge-Grayson Programs, Inc., Cleo Lowery, P.O. Box 63,
Leitchfield, KY 42755, Telephone: (502) 259-4054, Fax: (502) 259-4055
E-mail: None
Murray Head Start, Judy Whitten, 208 S. 13th Street, Murray, KY 42074,
Telephone: (502) 753-6031, Fax: (502) 759-4906, E-mail: None
Maryland
The Family Services Agency, Inc., Mary C. Jackson, 640 E. Diamond
Avenue, Suite A, Gaithersburg, MD 20877, Telephone: (301) 840-2000
x205, Fax: (301) 840-9621, E-mail: None
Friends of the Family, Inc., Linda R. Gaither, 1001 Eastern Avenue--2nd
Floor, Baltimore, MD 21202-4364, Telephone: (410) 659-7701, Fax: (410)
783-0814, E-mail: None
Michigan
Region II Community Action Agency, Martha York, Center for Family, 817
W. High Street, Jackson, MI 49203, Telephone: (517) 784-2895, Fax:
(517) 788-5998; 784-9226, E-mail: None
Mississippi
Friends of Children of Mississippi, Inc., Cathy Gaston/Marvin Hogan,
4880 McWillie Drive, Jackson, MS 39206, Telephone: (601) 362-1541, Fax:
(601) 362-1613, E-mail: None
Missouri
Human Development Corporation, Lois A. Harris, 929 North Spring Avenue,
St. Louis, MO 63108, Telephone: (314) 652-5100 x285, Fax: (314) 652-
0813, E-mail: None
KCMC Child Development Corporation, Shirley Stubbs-Gillette, 2104 East
18th, Kansas City, MO 64127, Telephone: (816) 474-3751 x603, Fax: (816)
474-1818, E-mail: None
Nebraska
Central Nebraska Community Services, Suzan Obermiller, P.O. Box 509,
Loup City, NE 68853, Telephone: (308) 745-0780, Fax: (308) 745-0824, E-
mail: None
New Hampshire
Community Action Program Belknap-Merrimack Counties, Inc., Rebecca
Johnson, P.O. Box 1016, Concord, NH 03302-1016, Telephone: (603) 225-
3295, Fax: (603) 228-1898, E-mail: None
New Jersey
Babyland Nursery, Inc., Mary Smith/Martin Schneider, 755 South Orange
Avenue Newark, NJ 07106, Telephone: (201) 399-3400, Fax: (201) 399-
2076, E-mail: None
NORWESCAP Head Start Administration, Linda Kane, 481 Memorial Parkway,
Phillipsburg, NJ 08865, Telephone: (908) 454-8830, Fax: (908) 859-0729,
E-mail: None
New York
The Astor Home for Children, Elizabeth Colkin, 50 Delafield Street,
Poughkeepsie, NY 12601, Telephone: (914) 452-4167, Fax: (914) 452-0718,
E-mail: None
Chautauqua Opportunities, Inc. Head Start, Grace Knaak, Municipal
Bldg--5th Floor, 200 E. Third Street, Jamestown, NY 14701, Telephone:
(716) 661-9430 Fax: (716) 661-9436 E-mail: [email protected]
Educational Alliance, Marion Lazar, 197 East Broadway, New York, NY
10002, Telephone: (212) 475-6200 x6200, Fax: (212) 982-0932, E-mail:
None
Parent & Child Center, Coleen A. Meehan, 175 Hudson Street, Syracuse,
NY 13204, Telephone: (315) 470-3324, Fax: (315) 474-6863, E-mail: None
Project Chance Early Head Start, Bart O'Conner, 136 Lawrence Street,
Brooklyn, NY 11201, Telephone: (718) 330-0845, Fax: (718) 330-0846, E-
mail: None
North Carolina
Asheville City Schools Preschool and Family Literacy Center, Robbie H.
Angell 441 Haywood Road, Asheville, NC 28806, Telephone: (704) 255-
5423, Fax: (704) 251-4913, E-mail: None
North Dakota
Little Hoop Community College, Beverly Graywater, P.O. Box 89, Fort
Totten, ND 58335, Telephone: (701) 766-4070, Fax: (701) 766-1357, E-
mail: None
Ohio
Child Focus--Clermont County Head Start, Terrie Hare, 1088 Hospital
Drive, Suite A, Batavia, OH 45103, Telephone: (513) 732-5432, Fax:
(513) 732-5440, E-mail: None
Cincinnati-Hamilton County Community, Action Agency, Verline Dotson,
2904 Woodburn Avenue, Cincinnati, OH 45206, Telephone: (513) 569-1840,
Fax: (513) 569-1251, E-mail: None
Oregon
Southern Oregon Child and Family Council, Inc., Blair Johnson, 505 Oak
Street, P.O. Box 3819, Central Point, OR 97502, Telephone: (503) 664-
4730; 857-9255, Fax: (503) 664-6620, E-mail: Pending
Pennsylvania
Family Foundations, Laurie Mulvey/Heather Fisher, 1811 Boulevard of the
Allies, Pittsburgh, PA 15219, Telephone: (412) 281-3511 Fax: (412) 281-
3254, E-mail: [email protected]
Philadelphia Parent Child Center, Inc., Jewel Morrissette-Ndulula, 2515
Germantown Avenue, Philadelphia, PA 19133, Telephone: (215) 229-1800,
Fax: (215) 229-5860, E-mail: None
Puerto Rico
Aspira Inc. of Puerto Rico, Edme Ruiz Torres, Box 29132, 65th Infantry
Station, Rio Piedras, PR 00929, Telephone: (809) 768-1968, Fax: (809)
257-2725, E-mail: None
New York Foundling Hospital, Zaida Fernandez, P.O. Box 191274, San
Juan, PR 00919-1274, Telephone: (809) 753-9082; 753-1321; 753-9080,
Fax: (809) 763-9209, E-mail: None
South Carolina
District #17 Schools, Anita E. Kieslich, P.O. Box 1180, Sumter, SC
29150, Telephone: (803) 778-6433, Fax: (803) 469-6006, E-mail: None
SHARE Greenville-Pickens Head Start, Rubye H. Jones, 652 Rutherford
Road, Greenville, SC 29609, Telephone: (803) 233-4128, Fax: (803) 233-
4019, E-mail: None
Tennessee
Chattanooga Human Services, Head Start/PCC, Donna Ginn, 2302 Ocoee
Street, Chattanooga, TN 37406, Telephone: (423) 493-9750, Fax: (423)
493-9754, E-mail: None
Tennessee CAREs, Barbara Nye, Tennessee State University, 330 Tenth
Avenue N., Box 141, Nashville, TN 37203, Telephone: (615) 963-7231,
Fax: (615) 963-7214 E-mail: None
Texas
Avance San Antonio Inc., Rebecca C. Cervantez, 2300 W. Commerce, Suite
304, San Antonio, TX 78207, Telephone: (210) 220-1788, Fax: (210) 220-
3795, E-mail: None
Head Start of Greater Dallas, Inc., Rob Massonneau, 1349 Empire
Central, Suite 900, Dallas, TX 75247, Telephone: (214) 634-8704 x484,
Fax: (214) 631-5417, E-mail: None
[[Page 66285]]
Parent/Child Incorporated, Blanche A. Russ-Glover, 1000 West Harriman
Place, San Antonio, TX 78207-7900, Telephone: (210) 226-6232, Fax:
(210) 228-0071, E-mail: None
Texas Migrant Council, Inc., John E. Gonzales, 5102 N. Bartlett Avenue,
P.O. Box 2579, Laredo, TX 78041, Telephone: (210) 722-5174, Fax: (210)
726-1301, E-mail: None
Utah
Bear River Head Start, Glenna Markey, 75 South 400 West, Logan, UT
84321, Telephone: (801) 753-0951, Fax: (801) 753-1101, E-mail: None
Vermont
CVCAC Head Start, Marianne Miller, PO. Box 747, 36 Barre-Montepelier
Road, Barre, VT 05641, Telephone: (802) 479-1053, Fax: (802) 479-5353,
E-mail: None
Early Education Services, Judith Jerald, 218 Canal Street, Brattleboro,
VT 05301, Telephone: (802) 254-3742, Fax: (802) 254-3750, E-mail: None
Washington
Families First, Peg Mazen, P.O. Box 1997, Auburn, WA 98071, Telephone:
(206) 850-2582, Fax: (206) 850-0220, E-mail: None
Port Gamble S'Klallam Tribe, Jaclyn Haight, 31912 Little Boston Road,
NE, Kingston, WA 98346, Telephone: (360) 297-6258, Fax: (360) 297-7097,
E-mail: None
Spokane County Head Start/ECEAP, Washington State Community College
#17, Patt Earley, 4410 North Market, Spokane, WA 99202, Telephone:
(509) 533-8500, Fax: (509) 533-8599, E-mail: None
Washington State Migrant Council, Carlos Trevino, 312 Division,
Grandview, WA 98930, Telephone: (509) 882-5800, Fax: (509) 882-1605,E-
mail: None
West Virginia
Monongalia County Head Start, Marie Alsop/Cheryl Wienke,1433 Dorsey
Avenue,Morgantown, WV 26505, Telephone: (304) 291-9330, Fax: (304) 291-
9324,E-mail: [email protected]
Wisconsin
Renewal Unlimited, Inc. - Head Start of Central Wisconsin, Suzanne
Hoppe, N6510 Hwy. 51 South, Portage, WI 53901-9603, Telephone: (608)
742-5329, Fax: (608) 742-5481, E-mail: None
Appendix B-1--Statement of Work for the Early Head Start Evaluation
Contract
Request for Contract--The Early Head Start Research and Evaluation
Project
I. Background
The Head Start Act, as amended May, 1994 (42 usc 9801 et seg.)
established a new program for families with infants and toddlers within
the framework of Head Start. Section 645A of the Head Start Act,
Programs for Families with Infants and Toddlers states that (a) ``The
Secretary shall make grants in accordance with the provisions of this
section for--(1) programs providing family-centered services for low-
income families with very young children designed to promote the
development of the children and to enable their parents to fulfill
their roles as parents and to move toward self-sufficiency.''
The Department of Health and Human Services calls this new program
Early Head Start. In developing Early Head Start, the Administration on
Children, Youth and Families (ACYF)/Administration for Children and
Families (ACF) engaged in an intensive consultation process to learn
from parents, practitioners, researchers and academics about the state
of the art of quality programming for pregnant women and families with
infants and toddlers. As part of the consultation process, the
Secretary of Health and Human Services formed the Advisory Committee on
Services for Families with Infants and Toddlers. That committee issued
a report in September, 1994, that provides the blueprint for the design
of the Early Head Start program. The ACF issued a Program Announcement
of in March, 1995, and is expected to begin funding programs by
September 30, 1995.
The award of this contract will be followed by a competition among
Early Head Start program research partners to establish a limited
number of local research sites. The first part of this section
describes the overall Early Head Start research and evaluation design,
including activities to be completed both by the national Contractor
and local researchers; the second part details the scope of work for
the national evaluation contract.
The Need for Early Head Start Research and Evaluation
It will be important for the proposed evaluation, mandated by the
Head Start legislation, to build upon the substantial body of knowledge
that exists and to expand upon findings from related studies. The CCDP
evaluation will present results of a rigorous evaluation of an
intensive, comprehensive, multi-service intervention program for
families of infants and toddlers, implemented across a number of
communities nationwide. Additional studies currently underway, such as
the National Institute of Child Health and Human Development (NICHD)
Child Care study and the Healthy Start evaluation will provide findings
and methods that will contribute to the Early Head Start research and
evaluation. However, Early Head Start, while related programmatically
to many predecessors, combines and/or extends elements in previous
programs to present a unique program for evaluation. The Early Head
Start program is individualized; intense; comprehensive; child-service
oriented; two-generational; locally-adapted, utilizing parents in
decision making, and designed to have four levels of effect, on infants
and toddlers, families, communities and staff.
II. The Early Head Start Research and Evaluation Plan
The research and evaluation plan highlights the first Early Head
Start programs as prototypes of the Early Head Start concept. The plan
features (1) a dynamic formative evaluation process, designed to be
used in subsequent Early Head Start programs, that will serve
continuous program improvement; (2) an impact evaluation that will
enable determination of whether and under what conditions program
prototypes were effective, and (3) an integrated research base for
generating further hypotheses around the broad array of potential
program issues and possibilities.
This research and evaluation plan features an integrated local and
national evaluation design with nested levels of program involvement.
Level I, continuous program improvement, is for all sites; Level II,
cross-site impact evaluation and site-specific, related research will
occur at selected sites. The impact evaluation is designed to determine
the attribution of outcomes to the intervention. The research portion
of the study will examine the causally modeled and directional
relationships among specific child, family, program and community
variables and outcomes.
A. Purposes
Elaboration of the purposes and the proposed approach proposed for
each follow:
1. Continuous Program Improvement. The Advisory Committee on
Services for Families with Infants and Toddlers proposed a new role for
program
[[Page 66286]]
evaluation that would be useful to programs seeking to develop to a
standard of high quality. A systematic feedback procedure utilizing
formative evaluation techniques will be developed as a tool for dynamic
program improvement, and as a prototype formative evaluation tool in
the event of Early Head Start program expansion. Thus, the first
purpose of this effort is to support a process for generating and
utilizing program qualitative and quantitative data, including
management information system data, in continuous program improvement.
This feature will be addressed programmatically at all Early Head Start
sites, in most cases with the aid of local research partners. The
national Contractor will provide support for this evaluation function
through development of formative evaluation formats for continuous
improvement.
2. Impact. The Head Start Act and the Advisory Committee on
Services for Families with Infants and Toddlers called for a study of
program effectiveness. A cross-site impact study will be conducted by a
national Contractor in a sample of selected sites. Site-specific
analyses, conducted by local research partners, will identify local
program impacts and elucidate the processes, pathways, and conditions
under which the program had an effect. Cross-site and local studies
will complement each other.
3. Additional Research. The Advisory Committee on Services for
Families with Infants and Toddlers sought to stimulate the research
community to address the many questions we have about how best to
enhance the development of low-income infants and toddlers and their
families under conditions of changing policies and programmatic
variations. The potential for research under the broad umbrella of the
Early Head Start purposes and in connection with impact and continuous
programmatic improvement is great. The Early Head Start research and
evaluation plan thus seeks to bring forth a new generation of solid
research to enhance current understandings of optimal developmental
circumstances for low-income infants/toddlers, their families and
communities. Further, questions related to the program are expected to
be fitted to theoretical frameworks to encompass and extend beyond the
realm of impact evaluation. This research will be conducted primarily
at selected research sites by local research partners
4. Longitudinal Study. Early Head Start presents a unique
opportunity to conduct longitudinal research on Early Head Start, Head
Start and beyond. Thus, the research and evaluation plan emphasizes the
underlying longitudinal nature of the study of Early Head Start, and is
the beginning of a longitudinal study of Early Head Start children and
families.
B. Studies
The specific studies this project is expected to generate and the
approximate sequence are as follows:
Studies to Describe Early Head Start Programs
Studies of Program Quality and Program Implementation
Studies of Program Impact
Studies of Program Variation
Studies Directed Towards Specific Policy Concerns
Studies of Program Impact in a Longitudinal Context
Studies by Local Researchers on Multiple Topics Pertaining to Early
Head Start
C. Questions
A series of preliminary research questions have been developed to
guide the formation of the research and evaluation design.
What are the characteristics of Early Head Start children and
families, communities and staff and programs? What are the origins of
Early Head Start programs? Who attends Early Head Start? How
representative are the children and families who attend Early Head
Start of the Early Head Start-eligible population within their
communities? What types of communities are Early Head Start programs
in? What types of services are delivered? What are the characteristics
and emphases of local programs?
What are the pathways to quality in Early Head Start Programs? How
do programs achieve full implementation? How is quality in Early Head
Start program components defined? What is the quality of Early Head
Start programs and program components? How long does it take to attain
quality in Early Head Start programs? What outcomes are associated with
various aspects of program quality?
Is Early Head Start effective in supporting the development of
children, family, communities and staff? Which Early Head Start
practices maximize benefits for children, families, communities and
staff under what kinds of circumstances? What are the collective and
differentiated impacts of Early Head Start? How does Early Head Start
support development under varying conditions of risk? Are there diffuse
effects of the program? Are there effects that can be attributed to
targeted programs or services in Early Head Start? Are there mediators
between services and outcomes that can be identified? What are the
benefits of Early Head Start that translate into dollars saved?
What child, family, program and community variables contribute to
the optimal development of low-income children in Early Head Start
programs? Which Early Head Start practices maximize benefits for which
children under what conditions? What factors contribute to resiliency
of children in Early Head Start? What factors associated with Early
Head Start contribute to optimizing health, social or cognitive
development? Is targeting specific services for children effective? Are
there strategies that are particularly effective with high-risk
infants? What are the programs that are achieving positive outcomes for
children doing? What are the barriers to attaining positive outcomes
for children in Early Head Start programs?
What Early Head Start factors, community, family and personal
factors contribute to parent and family-level outcomes? What factors,
under what conditions, enhance parenting skills including parent/child
interactions for which parents? What factors contribute to the parents'
ability to make progress toward self-sufficiency? What factors
contribute to the health and well-being of Early Head Start parents?
What factors contribute to male involvement in the lives of Early Head
Start children? What factors contribute to parental involvement for
which parents in the Early Head Start program? What family outcomes are
associated with positive child outcomes, and what are the pathways from
parent to child development in the context of Early Head Start? Are
there targeted strategies that specifically benefit some parents?
What changes in communities occur as a result of the Early Head
Start program? What were the baseline characteristics of Early Head
Start communities at the time programs began? What Early Head Start
practices maximize benefits for which communities under what types of
circumstances? What new collaborations were established? What community
factors supplemented or supplanted the Early Head Start success with
families and children? How strong was the community effect on Early
Head Start programs? Did Early Head Start have a positive effect on
child care services, or on any other services, throughout the
community?
What is the role of staffing in Early Head Start programs? What is
the role of staffing and staff development in creating effective
program processes and
[[Page 66287]]
bringing about positive outcomes for children, families and
communities? What enables staff to create the environments and
relationships that promote infant/toddler and family development? What
factors contribute to staff continuity with children and families? What
role does Early Head Start professional development play in staff
effectiveness?
What are the effects of program variations? What are the
identifiable program variations in Early Head Start? What can be said
about the types of variations and their effects?
What can we learn through Early Head Start to maximize collective
effectiveness of policies and programs that promote the development of
low-income children and their families? What is the role of Early Head
Start for promoting parents' pathways to work? How do comprehensive
Early Head Start services add to the effects of child care on children
and families? What are the barriers and pathways to the successful
integration of children with special needs into Early Head Start? These
questions will be addressed in a report directed towards specific
policy concerns by the national Contractor and the Contractor may be
asked to provide additional special reports around related issues.
How do Early Head Start and Head Start families and comparable
groups who do not participate in Early Head Start develop over time?
What are the developmental trajectories of Early Head Start and
comparison group children and families under varying experiences and
varying degrees of risk?
D. Design
The Early Head Start program is designed as a prototype of an on-
going service program as opposed to an intervention designed by an
investigator for theory building or hypothesis testing. Thus, while
Early Head Start was planned to accommodate evaluation, the evaluation
design primarily has been fitted to the program. This programmatic
emphasis has shaped numerous research and evaluation design elements as
well as the overall two-tier nature of the research and evaluation
plan. All Early Head Start programs will participate in either one or
both levels of the research and evaluation.
Level I: All program sites will participate in formative evaluation
activities which are designed to assist programs in continuous
improvement towards program quality. The Level I evaluation activities
also will be instituted for all subsequent Early Head Start programs
under conditions of program expansion. The following are features of
Level I.
Sites will use data from a uniform management information
system, together with local qualitative and quantitative data in a
formative evaluation process.
In most cases, sites will utilize a local research partner
for this aspect of the evaluation.
The national Contractor will provide standard formats for
the use of these data during the first year of the project, and will
make the characteristics of this format available to sites added in
subsequent years through collaboration with the Training and Technical
Assistance Contractor.
Level II: A sample of sites (12) will participate in Level II
activities focusing on evaluating the effectiveness of the Early Head
Start utilizing approaches to generate both breadth and depth in impact
evaluation . The following are characteristics of Level II activities.
There will be a cross-site impact study, conducted by a
national Contractor, that will be complemented by local studies of
program impact.
The local research partners may apply through a
competitive process for grants to carry out research studies at their
local sites. The sites whose local research partners receive these
grants will become sites for the cross-site impact evaluation. In the
event that there are not sufficient numbers of sites whose local
research partners have submitted an acceptable proposal or if a better
distribution of Early Head Start programs is required than the research
site pool represents, as programs were alerted in the Early Head Start
program announcement, ACYF may select further sites for evaluation.
Data collection at these sites would be conducted directly by the
national Contractor.
The impact evaluation will follow an experimental model.
The program will recruit double the number of families needed to fill
program openings. Families then will be assigned into program and
comparison groups by random assignment, most likely midway through the
first fiscal year. The recruitment and random assignment process will
continue until October 1, 1996, when programs are expected to attain
full enrollment, and, thereafter, as openings occur.
Only those programs that are fully implemented and
operating as the program was designed, with criteria to be determined
as an evaluation task, will be included in the final impact evaluation.
Additional criteria for impact evaluation may be proposed by ACYF as
well. It is anticipated that 12 sites will participate in the final
impact evaluation, however, 15 sites will be selected as preliminary
impact sites to provide an ample pool of sites for impact evaluation.
To fit the service emphasis of the program, subjects will
be continuously recruited into the program to fill program openings as
they occur. That is, this is not a cohort study. Sample sizes will
build over time.
While programs are encouraged to give preference to
subjects who are pregnant and have infants under a year of age, they
may serve children up to three years of age. However, to focus the
sample and to have a potential longitudinal sample of children who
began the program early, the research sample will be comprised only of
pregnant women and families with infants up to one year of age. This
requirement will apply to families recruited at the time the program
begins as well as to replacement families.
Programs will range in size from 75 to 150 families. The
impact evaluation sample, due to continuous recruitment, may exceed 150
program (and an equal number of comparison group) families but shall be
capped at 175 program (and an equal number of comparison group)
families for any one site. Site samples need to include at least 50
program families (with an equal number of comparison families).
In addition to documenting the services received by
program families in the Early Head Start program, it will be necessary
to document the needs and service use by comparison families to
determine if the individualized services provided by the Early Head
Start program had an impact beyond what comparison families received in
their communities. Therefore, the same needs assessment will need to be
conducted on control families and their service usage will need to be
tracked in a manner as similar to the experimental group as possible.
Comparison group families will be given an annotated list
of community services. Pregnant women will receive an initial referral
to prenatal care. All families, program and comparison, participating
in the evaluation study will receive approximately $20/interview period
for their participation in the study. Both the national Contractor and
local researchers are encouraged to solicit material goods to give to
families for completing interviews. These goods may be donated locally
or nationally and could include diapers, infant clothing, or
educational toys.
Program families who drop out will be followed, to the
extent possible. An
[[Page 66288]]
evaluation task will be to develop a plan for tracking comparison and
program group dropout research sample families and for defining the
minimum amount of time that constitutes program involvement.
E. Desired Outcomes
The Early Head Start program targets specific outcome variables in
the four program areas. It will be important for this evaluation to
focus on those outcomes likely to be associated with involvement in
Early Head Start, outcomes that realistically could be expected in a
program of this nature. It will also be important to target potential
interim outcomes, outcomes most likely to be apparent after the first
2-3 years of the study. Preliminary outcomes as proposed by the
Advisory Committee on Services for Families with Infants and Toddlers
include:
Child: Health and physical development; social competency; secure
attachments with parents and other caregivers; language and cognitive
development; resiliency factors; benefits to siblings.
Family: Attitudes towards parenting; parent-child interaction;
reproductive sequelae; having a medical home; parenting employability
and progress towards self-sufficiency; training and education; housing;
physical and mental health; substance abuse; home environment; safety;
involvement in the Early Head Start program; knowledge of child
development; child guidance beliefs and practices.
Community: Collaboration among agencies serving children and
families; seamlessness in referrals and actual service provision;
quality of services for children and families; increase in services for
infants and toddlers; safety.
Staff: Staff-parent/child relationships; staff continuity; staff
professional development; staff compensation; staff physical and mental
health.
F. Data Sources
For purposes of definition and discussion, data are referred to as
either (1) process or (2) outcome data. Process data refer to those
data that document program use and other experiences of families;
outcome data refer to those qualities the program seeks to bring about.
These data may be thought of, respectively, as independent and
dependent variables. It is recognized that some data fit both
definitions and that under different circumstances the same data
element could be either process or product, independent or dependent,
or, mediating, variables.
A family-level management information system will be introduced,
with technical assistance, at the outset of Early Head Start. This
information system, known as the Head Start Family Information System
(HSFIS) will include data elements focused on intake; needs assessment;
use of direct and referred services; family and child health
information and other information related to parent employment,
housing, education and services. Primarily, HSFIS data will be referred
to as process data, however, some of the HSFIS data also can be viewed
as interim outcome and outcome data. Program staff will enter and
utilize HSFIS data. Program staff will enter initial family background
data for all Early Head Start families prior to the random assignment
process. HSFIS data will be available for the evaluation of the Early
Head Start program.
Comparison family services use and health data will be recorded
frequently in a special HSFIS module that will be identical to or
parallel to HSFIS-entered program data. It is proposed that these data
will be entered by a Community Family Coordinator/s, who will form a
relationship with the comparison group families, paralleling to some
extent the context in which HSFIS data for program families will be
entered by program staff. The Community Family Coordinator services
will be subcontracted by the national Contractor, as feasible, through
the local researchers.
Additional process data (qualitative and quantitative) may be
collected locally or nationally as determined by researchers at either
level. This may include process data that cannot be collected through
the HSFIS in comparable ways for the comparison and program families
and will therefore have to be collected using the same, vs. parallel
data collection procedures.
Outcome variables and the measurement for those variables will be
identified and developed for the cross-site evaluation, for both
project and comparison families, by the national Contractor with input
from the local research team. Collection will be contracted through the
national Contractor to the local researchers.
Local research data will be collected by local researchers through
grants made directly from ACYF to the local researcher.
G. The Structure
In this research and evaluation design the primary responsibility
of the national Contractor will be to coordinate and administer the
cross-site evaluation in the 12 impact evaluation sites. This includes
financial and administrative responsibility for all data collected for
the cross-site-evaluation, including both process and outcome data for
both comparison and project families. The national Contractor may use
as much program-collected data (HSFIS) as the Contractor and ACYF deem
appropriate and, as feasible, is encouraged to subcontract other local
data collection to local researchers. The national Contractor has an
additional responsibility to provide a standard format for continuous
programmatic improvement.
The local program research partners will be responsible for
designing and conducting local research in areas relating to the
overall research and evaluation questions. Local researchers will be
funded through individual grants with ACYF to carry out locally-
relevant research. Local researchers will be expected to be reliable
subcontractors for the cross-site project. These same local researchers
will be expected to serve the continuous improvement needs of their
sites and to provide local impact evaluation reports to accompany the
cross-site reports required by this research and evaluation design.
A consortium will provide the mechanism for the coordination
required by the project. A Technical Work Group, meeting with the
consortium, will advise both the cross-site studies and the local
research projects.
H. Challenges to the Evaluation
The unique design for the Early Head Start program and the
requirements for research and evaluation present specific interesting
challenges that both local researchers and the national Contractor will
need to address. These include:
--Program variation within and between sites. The evaluation is
expected to stimulate unique approaches for process data measurement to
meet the challenge of documenting the variety of programs that will be
offered across sites and the individualized nature of services within
sites.
--Changing conditions over time. The evaluation is expected to take
into account a shifting range of ages. The age range will shift from a
less-than-two year range at the outset to nearly five years towards
program completion, due to continuous enrollment. Further, the
evaluation will need to accommodate the transition of children out of
the program at age 3 into Head Start or other programs and to plan for
extending this study longitudinally.
--Decoupling of parent and child data collection. Because Early Head
Start is not a cohort study, the periodicity of child assessments by
child age may
[[Page 66289]]
need to be decoupled from periodicity in assessing parent variables.
--Measuring the true nature of services delivered by Early Head Start
at both the child and family levels. This challenge requires assessing
a number of features associated with services, such as professional
standards of quality, duration, intensity, quality of relationships and
goodness of fit between the program services and individual needs.
--Delineating whether the services received are a function of the
program or of the families' own initiative from other sources in the
community. Since this study is not conducted in the isolation of a
laboratory, families may seek and receive services from other community
service providers. Therefore, it is important to determine what
services families receive outside the program.
--Documenting services received by the comparison group. We cannot
assume a no-treatment comparison group. While some comparison group
data has been collected through parent interview, in other studies
these data are usually collected at fairly long intervals with limited
checking of reliability. The current study requires a careful
documentation of comparison group services for interpretation of study
findings.
--Framing the role of relationships. For very young children the
relationships with parents and caregivers are central to development
and these relationships are often influenced by the relationships
between parents and program staff and the relationships of both to the
community at large. The evaluation is challenged to assess the central
role of relationships in this program.
--Measuring the effect of Early Head Start on communities and the
effect of communities on Early Head Start. Early Head Start is designed
to impact not only children and families but also communities. It is
designed to have a ripple effect on all the programs for young children
in a community. It is an important challenge for the evaluation to
determine how to measure the community at baseline and how to measure
change. Reciprocally, Early Head Start programs are nested in
communities. The evaluation is challenged to reflect the variance of
communities and to document the effect these communities have on Early
Head Start's ability to carry out its purposes.
--Conducting impact evaluation exclusively at fully-implemented
programs. The evaluation is challenged to determine criteria and timing
for assessing full implementation in order to focus the evaluation on
programs that are fulfilling the intent of the Early Head Start
program.
--Determining meaningful effects. It will be important for the national
Contractor and local researchers to go beyond the question of whether
Early Head Start had a simple effect. Researchers are challenged to
conceptualize the Early Head Start data set for use with complex
analytical approaches involving meaningful aggregations and pattern
analyses to account for varying degrees of risk, program variation and
time.
--Fitting national and local evaluations together. The evaluation is
challenged to bring two kinds of knowledge about Early Head Start
together--that gathered across sites and that gathered from in-depth
analyses within sites. A number of premises have been already made
about this feature of the evaluation. For example, it is central to
this project that local researchers and the national Contractor be
equals in evaluating this project. It is presumed that there are
questions that each can answer best from their unique perspectives.
Local researchers are in a position to truly delve into the causes and
effects and pathways to outcomes. They can use in depth and
observational as well as qualitative measures to determine a program's
effect. The local researchers are expected to address ``what's in the
box'' at their site using multiple measures and methods in site-
specific studies. The local researchers will also need to work together
to lay the groundwork for continuation of the study beyond the five-
year funding period. The national evaluation Contractor, on the other
hand, will need to address those questions that cross-site data will
enable answering, including those focused on program variation, and
those requiring large samples for ample cells sizes required for
examination of how the program worked for which children and families
under which conditions. The research and evaluation was designed to
bring forth both types of studies, and both types of studies are
important to the story this evaluation is expected to tell. It will be
necessary for every report to reflect this dual and complementary input
and for the researchers at both levels to affirm the role of the other.
Their task as partners in this evaluation will be to determine at each
step of the project how their two efforts fit together.
--Attrition. Given the five year nature of this program and the
intention to continue to follow the original Early Head Start families,
it will be important for the partners in this project to coordinate to
keep families in the sample. The evaluation is challenged to develop an
array of sample-retention ideas that may range from local solicitation
of gifts, newsletters and birthday cards to relationship-building to
maintain subjects' interest. It will also be necessary to develop a
clear plan for determination of which families who leave the project
will be followed for research purposes under what conditions.
C-2 Scope of Work for the National Contractor
Specifically, the national Contractor will:
1. Provide a description of Early Head Start, from its inception
through Year 1 for the Early Head Start programs, in all sites, with
special emphasis on the 12 research sites, relying primarily on HSFIS
data, but complemented by site profiles from research sites.
2. Conduct a study of program quality and implementation in Early
Head Start (preliminary) impact study sites programs (estimate 12).
This study will be used in the selection of fully-implemented sites for
inclusion in the final impact evaluation, and to present a story of the
development of quality for future Early Head Start programs;
3. Design and carry out an effective cross-site impact evaluation
(estimate 12 sites) that addresses evaluation challenges and determines
whether Early Head Start had an impact on children, families,
communities or staff, and that addresses differential effectiveness by
age of entry, need, sub-population, and by program features, duration
and intensity;
4. Conduct a study of program variation in the impact evaluation
sites (estimate 12 sites) and its effects;
5. Establish an infant sample for future longitudinal study in
impact evaluation sites (estimate 12 sites), to carefully track all
subjects to minimize attrition for the longitudinal study and to
include cross-site analyses of data in a longitudinal context in the
Final Report of this project;
6. Conduct timely analyses and reports (in all years) with Early
Head Start data in the context of critical policy issues, e.g.,
examining the value added of Early Head Start comprehensive services
for children in full-time child care and in the transition from welfare
to work, as requested by
[[Page 66290]]
the ACYF and/or the Technical Work Group; and
7. Prepare an Interim Report in September, 1997, and a Final Report
for this project which fully integrate the cross-site and local
studies, drawing upon the strengths of each.
C-3 Tasks
As part of this Early Head Start evaluation effort, the Contractor
shall access, collect, utilize, analyze and synthesize information
regarding the implementation, operation and effectiveness of Early Head
Start programs.
The work for this contract will be conducted in five sequential 12
month phases and the activities that will be accomplished include the
following:
In Phase I, the Contractor shall:
(1) Participate in an orientation meeting;
(2) Develop a coordination strategy for working with other
Contractors involved with Early Head Start;
(3) Conduct a literature review;
(4) Select a cadre of consultants;
(5) Select a Technical Work Group;
(6) Prepare a revised study design;
(7) Prepare a process data collection plan;
(8) Conduct a feasibility study;
(9) Prepare a logistical proposal for the consortium;
(10) Prepare a site visit protocol;
(11) Convene the consortium;
(12) Prepare a final study design;
(13) Conduct site visits to all impact evaluation sites;
(14) Prepare a protocol for the data collection instruments;
(15) Prepare a data collection and analysis plan;
(16) Prepare an Office of Management and Budget (OMB) clearance
package;
(17) Prepare a revised work plan;
(18) Develop criteria for selection of impact sites.
In Phases II-V, the Contractor shall:
(19) Conduct annual site visits to impact evaluation sites.
In ALL Phases, the Contractor shall:
(20) Conduct cross-site data collection;
(21) Conduct a minimum of two consortium meetings a year in
Washington, DC;
(22) Establish a protocol of all new or additional data collection
instruments and prepare OMB clearance packages for all new or
additional data collection instruments;
(23) Provide timely data entry and return of data disks to sites;
(24) Process and analyze the data collected;
(25) Provide a format for continuous program improvement and
support its use.
In addition, in all phases the Contractor shall prepare
deliverables as necessary for the work completed in each Phase,
including monthly progress reports and in-depth annual progress
reports, and the following reports within an agreed-upon time: ``Report
of Characteristics of Early Head Start Programs,'' ``Pathways to
Quality Study,'' ``Impact Study,'' ``Study of Program Variations,''
``Studies Directed Toward Specific Policy Concerns,'' an Interim
Report, and a Final Report which shall include a synthesis of the
results of the final data analyses, reports of site researchers and a
summary of the five-year project. In all Phase reports, the national
impact study will be supplemented and integrated with the studies from
the local research sites.
A. PHASE I
Task 1--Orientation Meeting With the Federal Project Officer (FPO)
Within one week of the effective date of the contract, the
Contractor shall meet with the Federal Project Officer (FPO), and other
relevant Federal staff to review the background of the project, and the
work to be conducted. The FPO will provide the Contractor with
available copies of the relevant grant proposals for ACYF-funded Early
Head Start grantees. The Contractor shall propose an agenda for the
meeting, indicate who would attend on behalf of the Contractor, list
the types of study design modifications or other problems that would
require FPO decisions at that meeting, and shall provide a project
summary for distribution to ACYF staff. Specific topics to be discussed
at the meeting include: Revisions to the proposed work plan in the
Contractor's proposal; arrangements for maintaining regular contact
with the FPO and relevant Federal staff via the INTERNET network and
other means of communication; arrangements for initial contacts and
ongoing cooperation with program sites; arrangements for information to
be supplied upon selection of research sites and plans for carrying out
the Phase I tasks. The meeting shall provide an opportunity to discuss
any clarifications of the Contractor's proposed approach, the nature of
the project, the schedule of work, and the progress report requirements
and other deliverables. There shall be an additional meeting with the
consortium in the second half of Phase I.
TASK 2--DEVELOP A COORDINATION STRATEGY FOR WORKING WITH OTHER
CONTRACTORS--HSFIS and TRAINING/TECHNICAL ASSISTANCE CONTRACTORS
During all phases of the project, effective coordination with the
Federal staff, Federal Contractors working on related projects and
evaluations and outside stakeholders will be important to the success
of the project. The Contractor shall work with the FPO and other
Federal staff to establish and maintain cooperative working
arrangements and in weeks two through six of Phase I shall establish a
list of tasks and a communication plan for approval by ACYF. It is
particularly important that procedures be coordinated with the HSFIS
Contractor, including procedures for communication and bi-annual
meetings in Washington, DC; procedures for ensuring readiness of
grantees to utilize the HSFIS at the outset of Early Head Start;
feasibility of HSFIS process data collection for project and comparison
groups; procedures for transferring HSFIS reports to the national
evaluation Contractor and for reports to the HSFIS Contractor. It will
also be necessary to coordinate with Training and Technical Assistance
personnel at ACYF and with the Contractor for Early Head Start Training
and Technical Assistance, planning for two yearly meetings with that
Contractor and ACYF staff in Washington, DC. Reports shall be given to
ACYF from each of the meetings.
TASK 3--PREPARE A LITERATURE REVIEW
The Contractor shall conduct a thorough review of the existing
literature on programs and evaluations of services to families with
infants and toddlers, including documents produced by ACYF, foundations
and reports of evaluations, published and not published. To place this
evaluation in a national context, the Contractor shall review and
synthesize relevant research and evaluation findings based on reliable
research methodologies about the effects of services to families with
infants and toddlers. This report shall synthesize findings from
services to families for infants and toddlers and those from services
related to any portion of Early Head Start, present methodological
issues and creative solutions to those issues, identify gaps in the
findings and methodologies and outline how this study will fill those
gaps. This report shall also include the Contractor's recommendations
for adding to or refining the evaluation research questions. The draft
report
[[Page 66291]]
shall be submitted by the beginning of the second month of Phase I and
the final report shall be submitted at the beginning of the third month
of Phase I. The Contractor shall provide ACYF with copies of each
document referenced in the literature review and shall deliver a an IBM
PC-compatible 3-\1/2\ inch diskette. ACYF shall reserve the right to
make the literature review, or parts of the document, available to the
public.
TASK 4--SELECT A CADRE OF CONSULTANTS
The Contractor shall establish a cadre of consultants from relevant
academic, professional, consulting and service-provider communities and
recommend names within two weeks of the contract effective date. The
intent is to have a cadre of professionals available for more intensive
involvement on the design and implementation than is feasible with the
Technical Work Group (TASK 5). Contractor shall provide the names and
vitaes of potential consultants.
The Contractor shall not secure their formal commitment prior to
the award of the contract, and without prior approval of ACYF. The
Contractor shall provide the names and vitae of potential consultants,
including their specific qualifications relevant to this study. Prior
to final approval, the Contractor shall provide a sufficiently detailed
description of the specific work (including total projected hours per
task or subtask to be done by this cadre, and a timeline for its
completion). The Contractor will be responsible for all expenses of
these consultants, including air fare, per diem and honorarium. The
number of persons in the cadre and the quantity of consultation shall
be the decision of the Contractor in cooperation with ACYF. The
Contractor shall propose an estimate of consultant use.
The Contractor shall report on expenditures for professional
consultants as a separate line item in monthly expense vouchers and
shall provide a separate monthly report on activities of consultants.
TASK 5: SELECT AND CONVENE A TECHNICAL WORK GROUP
The Technical Work Group will advise the entire Early Head Start
research and evaluation project, including national impact and local
research activities. Within two weeks following the contract effective
date, the evaluation Contractor shall recommend eighteen experts in
relevant fields, such as: Infant and toddler development; home
visitation; child care; Head Start; parent-child relationships; family
systems; teen parenting; services research; prevention and intervention
research; ethnic diversity and minority issues; health delivery
systems; parent education; mental health; adult education; family
ecology; community development; staff development; assessment of child
development; research methodology; statistics, instrument development,
tests and measurement. The Contractor shall be prepared to make
modifications in the list, as suggested by ACYF, based on additional
and/or alternative candidates who might bring additional strengths to
the Technical Work Group and to complete a group of twelve. All
Technical Work Group members will require the approval of the ACYF
Commissioner. In addition, the Technical Work Group must include
rotating representation from research and program sites. Technical Work
Group meetings will be held in conjunction with national-local research
consortium meetings but there may be additional meetings called by the
Contractor as needed. The Technical Work Group will provide guidance
for this entire project, advising the cross-site evaluation study and
local research, to produce a comprehensive picture of the complex story
of the impact of Early Head Start. Therefore, available time of the
Technical Work Group will need to be appropriated accordingly. A
portion of each Technical Work Group meeting shall be allotted to local
researchers issues.
Phase I meeting schedule: During Phase I of the project the
Contractor shall convene the meetings of the Technical Work Group. With
the exception of the first meeting, these meetings shall be held during
the consortium meetings. Within two months, or earlier if determined
advisable in the orientation session, of Phase I a meeting will be
convened to solicit initial comments and suggestions regarding the
overall scope of the evaluation and issues related to implementing the
set of proposed evaluation activities; to review the draft of the
literature and resource review plan; to review the draft design and
sampling plan and process data collection plans and for consultation in
selection of local research partners. Within six months (with final
determination to be set by the Early Head Start program timetable) but
as early as within four months of Phase I the Technical Work Group will
convene in the consortium to be introduced to local researchers and
their projects; advise local projects; establish representation from
the consortium for the Technical Work Group; establish sampling plans;
plan site visits; and to establish the preliminary data collection
instruments protocol. Nine months following the beginning of Phase I,
members will participate in and report on site visits, including
creating site profiles; consult for process evaluation; recommend a
final design; recommend a data collection instruments protocol for the
cross-site studies; meet with local researchers to plan specifications
for their studies; and review the overall research and evaluation plan
for Early Head Start. All expenses of the Technical Work Group
including honorarium, per diem, travel and lodging to Technical Work
Group or consortium meetings shall be covered by the national
Contractor. Any site-specific consulting done by the Technical Work
Group at site locations, with the exception of that done during site
visits, shall not be the responsibility of the national Contractor.
TASK 6--PREPARE A REVISED STUDY DESIGN
Within four and one half months after the beginning of Phase I, the
Contractor shall prepare a revised design and sampling plan.
Design and Sampling Plan: The Contractor shall develop a design and
sampling plan based on random assignment methodology. The design and
sampling plan also shall specifically include:
A discussion of the issues and approach the Contractor
will use to manage and coordinate with program and research staff, the
recruitment and random assignment of families into project and
comparison groups in 12 (preliminary) evaluation sites, including
discussion of issues pertaining to the implementation of experimental
design in low-income communities, steps that will be taken to ensure
comparability of program and comparison families and how to ensure
minimization of differential response rates and bias, over time;
A discussion of how the Contractor will meet ethical
challenges for comparison families presented by an experimental design,
to be addressed, to some extent by presenting an annotated list of
community services to comparison families, by a referral process for
families that have identified crises, and for pregnant women to receive
a focused initial referral for prenatal services;
A discussion of how the Contractor will resolve challenges
related to the burden on families, compensation, and attrition, and a
discussion of procedures to be put into place to maintain families'
interest, including predetermined plans
[[Page 66292]]
for Contractor to provide a payment of approximately $20/interview
period for project and comparison families; efforts of the national
Contractor or efforts to encourage local sites to leverage additional
material resources (such as diapers, infants clothing or toys); efforts
of a part-time Community Family Coordinator at each site, who will be
subcontracted by the national Contractor, to form relationships with
and collect data from the comparison group families (and to some
extent, program families). (See TASK 7, for elaboration);
A discussion of how the Contractor will address the
challenge of documenting the nature of services received by the program
families, given program variation between and within sites in type of
program delivered, quality, duration, intensity, and goodness of fit
between program and need; and a discussion of how the Contractor will
address the challenge of documenting the types of services comparison
group families received.
A discussion of how the Contractor will meet other
challenges to the design as presented in Section C-1-II-H Challenges to
the Evaluation, on page 19 of this document, and not directly addressed
in any other segment of the Contractor's proposal, including measuring
changing conditions over time; framing the role of relationships;
measuring and determining full program implementation (See also TASK
18);
A discussion of the importance of the Contractor's plan
for determining the representativeness of the Early Head Start sample
in impact evaluation communities, utilizing existing data sources;
A discussion of the implications of the design and how the
Contractor will collect data to place impact evaluation programs in a
community context, including a baseline measure of community
infrastructure; and
A discussion of any other challenges the Contractor
identifies for this evaluation and the Contractor's proposal for
resolving those challenges.
TASK 7--PREPARE A PROCESS EVALUATION PLAN
There are multiple challenges to the process evaluation of the
Early Head Start program that shall be addressed in a process
evaluation plan to accompany the study design (TASK 6) four months
after the beginning of Phase I. (1) In Level I sites the Contractor
shall develop a process report, ``Characteristics of Early Head Start
Programs Report,'' which will be a description of FY '95 and FY '96
programs, using HSFIS data. (2) In Level II sites it will be necessary
to begin documenting the characteristics, needs and the nature of
services for both program and comparison group families, including the
characteristics of programs and communities from the outset of Early
Head Start.
The Contractor shall have overall responsibility for collecting the
process data required for this study, but shall coordinate with local
researchers and program personnel. It is recommended that a part-time
Community Family Coordinator be employed or subcontracted at each site
who will coordinate local process data collection; maintain a
relationship with comparison families; document the service use and
provide emergency services for comparison families; and track families
who have moved or have left the program. The Contractor shall also be
responsible for the collection of:
A baseline intake interview for comparison and program
families in Level II sites. The baseline intake interview may be
completed by program staff using the HSFIS, before random assignment of
recruited families.
A needs assessment for comparison and project groups,
which may be gathered by the program personnel (program) and a
Community Family Coordinator (comparison) families using identical
formats, or utilizing an alternative format proposed by the Contractor.
Establishing comparability of process data between program
and comparison groups, utilizing a program-entered HSFIS data and
parallel comparison group HSFIS data entry. It is anticipated that
collection of service-use and health data will be conducted by the
program personnel (for program families) and that a Community Family
Coordinator will form a relationship with and enter such data for
comparison group families, using a special module of HSFIS. However,
the Community Family Coordinator will interview both project and
comparison families for data for which comparability of parallel entry
cannot be established or the Contractor shall propose an alternative
format.
Additional data collection procedures and a timetable for
process data collection from comparison and project families.
A plan for developing site-specific profiles that will
characterize each of the FY '95 impact evaluation sites. This task may
cross reference with TASK 10, PREPARE A SITE VISIT PROTOCOL. Year 1
site profiles for impact evaluation sites will be jointly authored by
national and local researchers and local program personnel.
An approach that would be used in drawing up a cross-site
descriptive study of the FY '95 and FY '96 Early Head Start programs
utilizing HSFIS data, supplemented by site profiles from research
sites.
Within two months after the beginning of Phase I, the
Contractor shall be prepared to submit an OMB package. See TASK 16,
PREPARE AN OMB CLEARANCE PACKAGE) of process data collection
instruments in the eventuality all or part of a HSFIS evaluation module
is not deemed comprehensive or desirable for process data collection.
An approach that would be used in drawing up a cross-site
descriptive study of FY 95 programs with special emphasis on describing
the 12 research sites, using HSFIS data supplement by site profiles
from research sites.
TASK 8--CONDUCT A FEASIBILITY STUDY IN 3 SITES
Within two months of the beginning of Phase I, the Contractor shall
discuss rationale for and submit a protocol for an evaluation
feasibility study protocol and within three months of Phase I shall
conduct a feasibility study in 3 sites in order to determine if
assumptions about the evaluation design are valid. This study shall
involve site visits which shall have tasks to: determine the status of
the HSFIS in the site; determine program-use needs assessment;
determine viability of entering comparison and project group data with
HSFIS software; determine feasibility of establishing adequate sample
size for experimental design; and estimate the feasibility of measuring
the level and quality of services available in the community for
referral services. A report from this study shall be submitted within
four months of Phase I.
TASK 9-- PREPARE A LOGISTICAL PROPOSAL FOR A NATIONAL-LOCAL RESEARCH
CONSORTIUM
Within three months of Phase I, the Contractor shall be responsible
for proposing a consortium logistics plan which shall be submitted to
ACYF for approval the seventh month after the beginning of Phase I,
following review by the consortium members. This plan shall include the
logistical approach to bi-annual consortium meetings in Washington, DC,
to be attended by the Contractor, the Technical Work Group and local
researchers from impact evaluation sites; a discussion of time-use
divided into equal day-long segments in order to meet the three needs
of the consortium (impact study planning; local research, and Technical
[[Page 66293]]
Work Group consultation); a discussion of areas of the impact study for
which the Contractor will seek input from the local researchers, i.e.,
site visit protocols; data collection instruments; data collection
procedures; workplan; a discussion of the areas for which the national
Contractor, the Technical Work Group and the local researchers will
need to work closely together as partners, i.e., preventing attrition,
integrating the national and local research efforts, publication
issues, and data use. The Contractor is encouraged in preparing these
discussions to review other consortium arrangements such as that
utilized by LONGSCAN. The Contractor will be responsible for logistical
expenses associated with the consortium, as well as for all of the
expenses of the Technical Work Group. Local researchers will cover
their own travel, lodging, registration and other expenses. The
national Contractor shall also provide for honoraria and expenses of
any speakers, if necessary, and subject to prior approval from the FPO.
TASK 10-- PREPARE A SITE VISIT PROTOCOL
The third month of Phase I, the Contractor shall develop a draft
site visit protocol which details procedures for site visits. The
purposes of the site visits will be to review continuous program
improvement evaluation procedures at all FY '95 (and FY '96 research
sites). In impact evaluation sites, additional purposes will be to
establish site profiles, to review staffing for Community Family
Coordinators; to establish relationships with the local researchers and
to understand the local research projects; to establish the procedures
for random assignment, and to establish local procedures for data
collection. The FPO and other ACYF representatives will review the
draft protocol and return it within one week to the Contractor who
shall present a final protocol to ACYF by the fourth month of Phase I.
As part of the protocol development process by the third month of Phase
I the Contractor shall provide the FPO with a draft letter of
introduction for the ACYF Commissioner to send to Early Head Start
sites that will participate in site visits. The letter shall identify
the Contractor, describe the purpose of the project, and inform the
Early Head Start programs about plans for the site visits and specify
other contacts, including community and research representatives. A
letter shall also be provided to the FPO for the researchers at the
sites, identifying their roles in the site visit and describing the
purpose of the visit. Prior to conducting the site visits, the
Contractor shall submit a memorandum to the FPO outlining a schedule
for the visits and an outline of a standardized format for site visit
reports that shall be submitted to the FPO within two weeks after each
visit. Each proposed three-person site visit team shall be comprised
of, but not limited to, representatives of the national Contractor; the
Technical Work Group, and program or research staff from other sites.
ACYF staff may be represented as well.
TASK 11--CONVENE THE CONSORTIUM
Upon selection of research sites, within four months of Phase I
and/or within one month of the selection of research sites, the
Contractor shall convene a meeting of the consortium in Washington, DC,
including ACYF, the national Contractor, local researchers and the
Technical Work Group. The Contractor shall carry out the logistical
plan as proposed previously, dividing the consortium time into thirds
for addressing needs of the cross-site impact evaluation, local
research development and advise for both from the Technical Work Group.
At the initial consortium meeting, the Contractor shall provide
opportunities for identification of each of the local research sites'
research purposes; discuss the logistical plan with the consortium;
establish committees as identified by the logistical plan; establish a
work plan; establish any subcommittees; discuss issues for immediate
and future data collection; review process data to be collected by
HSFIS and otherwise; review sample selection procedures; review the
preliminary site visit protocol; and name site visit teams. The
national Contractor shall communicate about this meeting with ACYF for
a potential joint meeting with program staff. The national Contractor
is responsible for all costs associated with consortium meetings,
including hotel, break out rooms, expenses of Technical Work Group,
except for the direct expenses of the local researchers and federal
staff.
TASK 12--CONDUCT SITE VISITS TO ALL FY '95 EARLY HEAD START IMPACT
EVALUATION SITES
From the fifth through seventh month of Phase I and/or within two
months of the selection of research sites, the Contractor shall begin
site visits as specified in the Site Visit Protocol. A draft report and
sample site profiles shall be submitted to the FPO by the sixth month
of Phase I. Site visit reports and profiles on every Early Head Start
site evaluation site shall be submitted to ACYF by the seventh month of
Phase I. For planning purposes, the Contractor shall allow for site
visits of 2 days in length for each site (with the actual length of the
visits varying somewhat as a function of the size and complexity of the
program, as well as the intended tasks to be accomplished.) All
expenses from the site visits shall be handled through the national
contract.
TASK 13--PREPARE A FINAL DESIGN
It is anticipated that information provided by the Early Head Start
site visits, by the interactions with the local researchers and by the
meetings with the consortium, Technical Work Group and the FPO will
call for changes and clarifications in the evaluation design and
implementation plan. Based on this information the Contractor shall
prepare a draft revised technical evaluation design and analysis report
by the seventh month of Phase I and final plan by the eighth month of
Phase I which consists of the following components:
A. Statement of Evaluation Outcomes
A list of research and policy questions, both general and specific,
that the study shall address. Each specific question shall be logically
connected with the general question to which it relates, as well as
being organized according to the overall conceptual model of the study.
For each specific question the theoretical hypothesis, required
data elements and data source(s) shall be identified.
For each specific question, a discussion of any measurement issues
for obtaining realistic and valid outcomes and the approach to
resolving those measurement issues shall be included.
B. Revised Study Design (See TASK 6)
C. Revised Process Data Collection Plan (See TASK 7)
TASK 14--ESTABLISH A PROTOCOL FOR ALL DATA COLLECTION INSTRUMENTS
By the ninth month following the award of this contract, the
Contractor shall submit to the FPO a complete draft protocol for data
collection instruments for studies for Phase II of the evaluation and a
proposed protocol for data collection instruments for Phases III-V of
this study. It is expected that the Contractor will seek input from
local researchers through the consortium but that final responsibility
for this protocol rests with the Contractor. This protocol will have
multiple sets of data collection instruments (or interview guides). The
first set includes
[[Page 66294]]
instruments to assess quality in site program activities and the second
set will include instruments to measure the programs' impact on
children, families, communities and staff. A third set will include
instruments to determine the variations in programs and may overlap
with other sets. It is anticipated that information shall be gathered
through interviews with parents and staff as well as through
observation of children, parents, home environments, and staff.
The Contractor's approach to measurement, including discussion of
measurement issues, for the several studies of this evaluation shall be
presented. The Contractor shall identify strategies for searching for
measurement instruments, for including measurement instruments utilized
in related studies of infant/toddler development or family services;
and for pilot testing the data collection instruments. It is
anticipated that community-level outcomes may involve the development
of new data collection instruments. It is possible that some of the
quality, variations or policy-related data may be collected by the
national Contractor using cross-site survey methods or qualitative
assessments.
The instruments selected or developed shall be clearly linked to
the conceptual design of the study, services delivered and expected
outcomes. The set of instruments for the quality study shall generate
information in critical areas such as:
Child relationships with caregivers.
Child and parent continuity in relationship with program
providers.
Parent perceptions of, expectations of, and satisfaction
with the program.
Staff perceptions of the quality of their program.
Parent relationships with case managers and other key
Staff.
Goodness of fit between parent/child needs and services
delivered.
Availability, access and quality of services in the
community.
Availability, access and quality of parent education
activities.
Quality of home visitation.
Perceptions of the program by community members.
Child care environment.
The studies of impact shall generate information in critical areas
not contained in the HSFIS or gathered as process data and including:
Child development.
Child security of attachment.
Child risk and resiliency factors.
Home environment.
Child care environments.
Parent-child relationships.
Other caregiver-child relationships.
Parenting attitudes.
Parent knowledge of child development.
Parent attitudes about guidance.
Support for parenting.
Perceptions of conflict and/or violence in the
neighborhood.
Perception of parent involvement activities.
Community collaboration.
Community development of services.
Staff professional development.
Staff-children/family relationships.
Sibling health and development.
For each of the proposed data collection instruments, the
Contractor shall attach an analysis of the instruments with regard to
any prior use in other studies of a similar nature, their psychometric
properties and their acceptance by experts in the field as appropriate
measures. The Contractor shall attach the results from pilot studies of
each of the instruments in the final protocol. The Contractor shall
prepare complete protocols of all instruments, and a training plan for
all data collectors. The Contractor shall revise the instruments plan
based on input from the FPO and the consortium, including the Technical
Work Group, and shall submit final data collection instruments to the
FPO for approval by the end of the tenth month of Phase I.
TASK 15--SUBMIT A DATA COLLECTION AND ANALYSIS PLAN
The Contractor shall prepare a data collection and analysis plan
that links each study question to the data collection instruments,
proposed respondents/data sources and study methods and the final
design and sampling plan (TASK 11). The Contractor shall provide a
graphic that displays this information.
The national Contractor shall be responsible for all the costs of
data collection for all of the national studies as described in this
document, including the cost of compensating families for interviews.
Local researchers will also be encouraged to generate additional
material resources for families. Impact data collection may be, and it
is anticipated in most cases, will be, subcontracted to the local
researcher, in response to the Contractor's call for an application
containing data collection plan and qualifications of staff. The local
researcher shall have first subcontracting opportunity at the first
collection period. However, if, after that, due to a lack of quality or
timeliness in previous data collection; or the local researcher does
not want to subcontract; or there is no local researcher at the site,
the national Contractor may subcontract with other qualified
researchers for local data collection. Subcontracts shall be renewed on
an annual basis.
The Contractor shall develop an overall data collection plan which
clearly outlines timelines for all proposed data collection activities,
including a theoretically-based justification for each proposed data
collection activity. The data collection plan shall include:
A discussion of issues around the timing of data
collection and a proposed timetable for data collection. It is also
anticipated that the impact data shall be collected at multiple points
in time, to correspond with predetermined targets around children's age
and parent and staff length of time in the program. Thus, data
collection activities involving parents and children may need to be
decoupled, leading to the likelihood that data collection in any one
site may be relatively continuous. The Contractor shall discuss a
preferred approach to this issue;
Procedures for contacting and tracking families over time;
A discussion of recommended procedures for the follow-up
of incomplete data;
Theoretical justification, procedures and timelines for
assessment strategies proposed by the Contractor in additional areas
not already mentioned in the presentation related to data collection of
the program process;
Theoretical justification, procedures and timelines for
assessment strategies proposed by the Contractor related to data
collection of child, family, community or staff outcomes;
Theoretical justification, procedures and timelines for
conducting observations and other data collection focused on program
quality or variations;
A discussion of a quality control component which
addresses the training of data collection staff at the local Early Head
Start program sites, continuity of data collection staff and methods
for ascertaining reliability and effectiveness of data collectors;
Procedures for identifying and assessing the quality of
existing data, as well as procedures for negotiating with sites to
access and utilize existing sources of data, particularly as they
pertain to community data for determining the representativeness of the
Early Head Start recruited families; and
Procedures for establishing, maintaining and overseeing
the subcontractors cooperative relationships with the Early Head Start
programs that shall maintain the independence and objectivity required
for a third party
[[Page 66295]]
evaluation, but will allow for the effective management of data
collection activities.
The Contractor shall discuss the data analysis challenge (Section
I-I), including a discussion focused on determining the magnitude of
effect across diffuse program services, and propose solutions to these
challenges. The Contractor shall identify the specific types of data
analyses that will be employed for each phase of data collection and
for each data element, within the context of the revised study design,
including the unit of analysis, possible aggregations and method of
display in the final report.
The draft data collection and analysis plan shall be submitted to
the FPO by the end of the eleventh month after contract effective date.
The FPO will review this plan with other ACYF staff and submit comments
to the Contractor within one week. The Contractor shall make the
required corrections and resubmit the plan in final form to the FPO by
the end of twelfth month after contract effective date.
TASK 16--PREPARE AN OMB CLEARANCE PACKAGE
The Contractor shall develop an Office of Management and Budget
(OMB) clearance package for the study, including all data collection
instruments and transmittal memorandum in accordance with OMB and the
ACF guidelines. The package shall include, but not be limited to, the
following:
A justification and introduction to the study. This
includes a justification of why the study is needed; how, by whom and
for what purpose the information will be used; why existing information
cannot be used and a summary of study components;
Data collection plan. This includes both a description of
and a justification for the study design, including the sample plan;
design of data collection instruments with a question by question
justification; results of pretesting data collection instruments; and a
data analysis plan;
Tabulation and publication plans;
Consultation with outside agencies;
Respondent burden estimate;
Confidentiality statement.
The OMB package shall be submitted to the FPO by the end of the
eleventh month of Phase I. The FPO will provide up to four sets of
comments to the Contractor over a period of three weeks. The Contractor
shall then submit the final OMB package to the FPO by the end of the
twelfth month of Phase II. The Contractor shall allow at least 120 days
for OMB approval.
TASK 17--REVISE THE INITIAL WORK PLAN INCLUDED IN THE PROPOSAL
Based on the progress of work covered by Tasks 1-16, the Contractor
shall produce a revised work plan for each of the remaining Phases of
the contract (Phases II-V), by the twelfth month after the contract
effective date. Key issues to be addressed in both the initial and
revised work plan shall include:
Effective coordination of this project with Federal staff
and designated Contractors, including HSFIS Contractor; Training and
Technical Assistance Coordinator; Early Head Start sites, the
consortium and the Technical Work Group;
Identification of issues to be resolved for data
collection with plans and timelines for how those issues will be
addressed;
Identification of logistical issues in the workings of the
consortium and plans for addressing these issues. A schedule of
consortium meetings;
A proposed protocol of measures with timelines and
identification of data collectors for each data collection point;
Data collection, analyses and reporting plans for later
phases of this project;
A schedule for subsequent site visits;
Any other remaining tasks.
TASK 18--SELECT FINAL CRITERIA FOR INCLUSION IN IMPACT EVALUATION
The Advisory Committee on Services to Families with Infants and
Toddlers recommended establishing criteria for Early Head Start program
evaluation, including the recommendation that no Early Head Start
program shall be evaluated that is not fully implemented. Twelve months
after the beginning of Phase I, the Contractor, with input from the
Technical Work Group, shall submit a draft plan for determining whether
sites demonstrate viability for impact evaluation. This plan shall
include criteria for defining the minimum threshold for program
implementation; standards for demonstrating whether comparability
between comparison and project families was maintained; adequate power,
and any other criterion deemed important to a valid evaluation of
impact, i.e., absence of saturation of Head Start-like services in the
community. As criteria for full implementation, the review shall
include consideration of measures of implementation associated with the
Early Head Performance Standards; Head Start Performance Measures; and
program quality in general. This review and the plan shall be submitted
to ACYF. On approval of the criteria, site visit teams will rate sites,
beginning with research sites, on each criterion; this rating shall
involve a site visit which may be combined with a previously scheduled
site visit or with other planned data collection. Ratings will be
forwarded to ACYF, who will make the final determination of which sites
shall be included in the evaluation. Three months after the beginning
of Phase II or on a modified timetable as proposed by ACYF or the
national Contractor and approved by ACYF, final determination shall be
made by ACYF regarding which and how many sites to include in the final
impact evaluation. As a conservative estimate, the Contractor shall
plan to conduct an impact evaluation at 12 sites.
B. PHASES II-V
TASK 19--CONDUCT ANNUAL SITE VISITS
Within one month of each new Phase, the Contractor shall develop a
revised protocol for annual site visits to, at a minimum, all sites
included in the evaluation. Protocols for the site visits shall be
developed with input from the consortium, including the Technical Work
Group, and be submitted to ACYF for final approval within three months
of each new Phase. Site visits shall follow the approved protocol and
shall include verification of data collection procedures; availability
and use of program data, including HSFIS data, for continuous program
improvement; follow through on research plans, and continued
documentation of the nature of the program. Written reports shall be
submitted to ACYF and the site within three weeks of each visit. The
written report shall include an updated site profile, authored jointly
by the Contractor and the local researcher, where applicable. The
Contractor may be asked to conduct site visits to new Early Head Start
sites for purposes the same as for FY '95 program sites. The proposal
shall include a per-site cost to cover the possibility of additional
Early Head Start program site visits in subsequent years.
C. ALL PHASES
TASK 20--CONDUCT CROSS-SITE DATA COLLECTION
The Contractor shall conduct cross-site data collection for the
national impact studies (for both the project and comparison groups) in
an estimated 12 selected Early Head Start sites, either directly
providing for data collection or by subcontracting with local
evaluators, as determined on an annual basis.
[[Page 66296]]
Upon approval of the OMB Clearance Package, the Contractor shall
conduct the appropriate data collection activities (outlined in the OMB
clearance package) at the selected Early Head Start programs. The
Contractor shall develop a plan to have senior evaluation staff conduct
periodic site visits during data collection periods for the purpose of
monitoring on-site evaluation staff, ensuring quality control and
maintaining good working relationships with local research and program
staff. The Contractor shall develop procedures for monitoring local
staff to make sure they carry out their evaluation responsibilities.
Where appropriate, the Contractor also shall consider the potential
need for the use of security guards to accompany researchers in cases
where their safety is at risk.
As the data collection in this project has a longitudinal nature,
whenever possible, data collectors with demonstrated effectiveness
shall maintain continuity with families; the Contractor shall have
developed compensatory procedures for maintaining reliability of
measurement.
In each site, the Contractor shall continue to design and implement
methods for understanding the services provided to both the treatment
and comparison groups and at the community level. The Contractor shall
continue to build the profile begun at each site to describe the
general character of each program and shall continue to examine methods
for documenting the program. Finally, the Contractor shall continue to
explore methods for understanding the communities' impacts on and from
Early Head Start programs. The Contractor shall work closely with
researchers at local sites in these tasks.
TASK 21--CONDUCT A MINIMUM OF TWO MEETINGS A YEAR WITH THE CONSORTIUM
Within the first three months following the beginning of each new
phase, the Contractor shall convene the consortium, including the local
researchers and the Technical Work Group, in Washington, DC., to
conduct the business of the consortium according to the consortium
workplan. A minimum of two consortium meetings shall be convened each
year, and no six month period shall pass without a consortium meeting.
The possibility of meeting in connection with Early Head Start program
personnel shall be considered for at least one of the two meetings and
ACYF shall guide the decision on whether a program-research meeting is
advised. There shall be support for at least two meetings of each of
the consortium subcommittees per year, as necessary. The Contractor
shall deliver ACYF consortium reports with embedded Technical Work
Group reports within one month following each consortium meeting. The
Contractor shall be responsible for all costs associated with
consortium meetings (See TASK 11), except for the direct costs of local
researchers and the federal staff. If the Contractor coordinates with
program personnel, only the research and evaluation portion of the
costs of the consortium shall be the responsibility of the national
Contractor.
TASK 22--ESTABLISH A PROTOCOL OF ALL MEASURES FOR EACH NEW PHASE OF THE
PROJECT AND PREPARE AND SUBMIT NEW CLEARANCE PACKAGES FOR SUBMISSION TO
THE OFFICE OF MANAGEMENT AND BUDGET (OMB), AS NECESSARY
Within the first three months of each new phase, and as needed, the
Contractor shall review the work plan to determine acceptability of the
protocol for each new phase of measurement and the need to obtain OMB
clearance. Measures added shall be submitted with input from the local
researchers and the Technical Work Group, to ACYF, pilot tested and
approved by the consortium according to procedures developed in the
original protocol before being submitted to OMB. Procedures under TASK
14, TASK 15 and TASK 16 shall be adopted for subsequent measures.
TASK 23--PROVIDE SITES WITH DATA FILES AND SUMMARY REPORTS
The objective of continuous program improvement necessitates the
timely turnaround of all data. Therefore, it will be necessary for
impact data submitted from local sites to the national Contractor to be
cleaned, entered and returned on disk to the local site within three
months of its submission to the national Contractor. The national
Contractor will need to develop procedures for working with sites that
do not maintain quality and timeliness standards within the
subcontracting structure.
Within six months of each Phase, a site-level printout for all
impact evaluation sites shall be generated with sites identified only
by number to maintain confidentiality, presenting the data, as
predetermined in the data analysis plan, in summary form for each site
and summed or averaged, as appropriate, across sites. From these
printouts, sites shall be able to compare their own results with those
of other sites. This report shall include HSFIS data. The national
Contractor shall be required to submit, receive and report to ACYF a
brief assessment of their own activities in the task of timely data
turn around. To this end, initiated by the national Contractor, local
programs will briefly evaluate the Contractor's timeliness and
formatting of returned data. This brief report shall be due to ACYF by
the eleventh month of each Phase, and will be submitted to the ACYF
together with printouts of data returned to sites.
TASK 24--PROVIDE FORMATS FOR CONTINUOUS PROGRAM IMPROVEMENT
All Level I sites are expected to utilize formative evaluation
procedures for continuous program improvement as a component of program
management. Many Level I sites will identify a local university or
research institution partner to assist them in completing this task;
Level II sites will be expected to participate in continuous program
improvement activities in addition to conducting research. The national
Contractor shall be a partner in formative evaluation tasks for
continuous program improvement by: Conducting site visits to all 12
evaluation sites in order to informally assess site preparedness for
continuous program improvement and to provide on-site guidance for
initiating this function at the local level; developing a standard
format for orienting subsequent Early Head Start sites (FY '96 and
beyond) to continuous program improvement activities; providing annual
formative evaluation training, either directly or through the Training
and Technical Assistance Contractor, to all program sites, during
annual program consortium meetings in Washington, DC. or utilizing an
alternative format; participating in bi-annual coordination meetings
with the Training and Technical Assistance Contractor to ascertain that
capacity for this new management function develops in all sites, and
coordinating with the HSFIS Contractor through bi-annual meetings to
assure that HSFIS data are being utilized for continuous program
improvement. The Contractor shall provide a continuous program
improvement report in the twelfth month of each Phase.
TASK 25--ANALYZE THE DATA
The Contractor shall conduct and complete analyses of national
evaluation data on a timetable jointly agreed upon by ACYF and the
Contractor, and based on the
[[Page 66297]]
methodology approved under TASK 15. After preliminary analyses during
each Phase, the Contractor may revise the analytical plan based on the
quality and completeness of the database or on refinements of the
conceptual hypotheses. Procedures for data analysis shall be reviewed
by the Technical Work Group after each Phase of data collection and
analysis. Recommendations for any revisions in the data analysis plan
shall be submitted to the FPO for review and approval. In addition, the
Contractor shall reserve 5% of the budget for analyses requested by
ACYF or the Technical Work Group for analyses focused on policy or not
specified in the contract.
C-4 Deliverables
a. Literature and Research Review, Draft and Final: The Contractor
shall produce a draft and final literature review. The draft report
shall be submitted to the FPO by November 1, 1995. The FPO shall
provide comments within two weeks. The final version of the revised
literature review shall be submitted to the FPO by December 1, 1995.
b. Revised Draft Study Design and Process Data Plan: The Contractor
shall submit a revised study design and sampling plan and process data
plan by January 31, 1996. This shall be an updated version of the
design and sampling plan submitted in the Contractor's Best and Final
proposal and shall reflect input from the Technical Work Group and
ACYF. It shall be in draft form. The process data plan shall reflect
the progress of HSFIS implementation as well.
c. Feasibility Study to Test Assumptions of the Design--Protocol
and Report: By November 31, 1995, the Contractor shall submit a
feasibility protocol, updated from the proposal to reflect knowledge of
sites available for the feasibility study. This shall be a pilot study
of the sampling, design and process evaluation procedures proposed.
Working with ACYF, program sites for this study will be selected and
the Contractor shall report findings by January 31, 1996.
d. Consortium Logistics Plan, Draft and Final: The draft and final
Logistics Plan shall be submitted, respectively, by December 31, 1995
and April 30, 1996. This document shall be an update from procedures
submitted with this contract and shall propose operations procedures
that will guide the coordination of consortium logistics for the
cooperative aspects of this project.
e. Site Visit Draft and Final Protocols, Draft and Reports: After
review by the Technical Work Group, the draft of the site visit
protocol shall be submitted to the FPO by the end of December 1995.
ACYF will make recommendations, and the final site visit protocol shall
be submitted to the FPO for approval by January 31, 1996. By the
beginning of the March 1996, (the week after the first site visit), the
Contractor shall submit a draft site visit report for approval by the
FPO. The draft report shall include a schedule of events, an analysis
of data from interviews and assessments, a summary of any additional
issues raised, and an updated and expanded profile of the program and
its evaluation. The FPO shall provide feedback to clarify expectations
about content and format within two weeks. The final version of the
first site visit report shall be submitted to the FPO for approval by
the end of the March 1996. For each remaining site visit, a draft of
the site visit report shall be submitted to the FPO one week after the
site visit. The final version of each report shall be submitted to the
FPO for approval three weeks after the site visit. The report for the
last of the site visits shall be submitted no later then the end of
April 1996.
f. Study Design, Draft and Final: The study design report shall be
due April 30, 1996, reflecting input from the Technical Work Group, the
consortium, and from site visits. The FPO shall review the design,
obtain comments from other ACYF staff, and provide comments to the
Contractor within two weeks. The Contractor shall then make corrections
to the design and submit a final study design for review and approval
by the FPO. The final study design shall be submitted by May 31, 1996.
g. Data Collection Instruments Protocol, Draft and Final: With
input from the consortium including the Technical Work Group, the
Contractor shall develop, or select existing data collection
instruments to be submitted to the FPO by the end of the June 1996. The
Contractor shall attach an analysis of the instruments with regard to
any prior use in other studies of a similar nature, their psychometric
properties, their acceptance by experts in the field as appropriate
measures, and their performance in pre-tests and field testing. The FPO
shall provide comments to the Contractor within two weeks. The
Contractor shall revise the instruments based on the comments by the
FPO and shall submit final data collection instruments to the FPO for
approval by July 31,1996.
i. Data Collection and Analysis Plan, Draft and Final: With input
from the consortium including the Technical Work Group, the Contractor
shall present a draft data collection and analysis plan to the FPO by
August 31, 1996 that shall be a complete plan for the data collection
for this project and shall present a plan for analysis to answer the
original study questions. The FPO shall review this plan, returning it
to the Contractor for revisions and request its return by the September
30, 1996.
j. Revised Work Plan, Draft and Final: A draft of the Phase I
Revised Work Plan shall be submitted to the FPO for approval by July
31, 1996. The FPO shall provide feedback within one week. The final
version shall be submitted by September 30, 1996. The work plan may be
revised once the data collection is underway to make use of new
information or strategies which emerge over time. Proposed changes
shall be indicated in the monthly technical progress reports and shall
require the prior written approval of the FPO before changes are
implemented.
k. OMB Clearance Package: The draft OMB package shall be submitted
to the FPO by August 31, 1996. The FPO shall provide up to four
separate sets of comments to the Contractor over a period of two weeks.
The Contractor shall then submit the final OMB package to the FPO for
approval by the end of the September 1996. The Contractor shall allow
at least 120 days for OMB approval. An early OMB package shall be
developed within the first several months on a schedule to be
determined by the Contractor and FPO.
l. Report of Site Qualifications for Evaluation: The draft
criteria, finalized criteria and report of sites' qualification for
criteria shall be submitted to ACYF respectively, September 30, 1996,
November 30 and December 31, 1996.
m. Phase II--V Site Visit Protocols, Reports: Following the
schedule established for the Phase I site visit reports, a draft of the
site visit protocol shall be submitted to the FPO by the third month of
each Phase and a draft report shall be submitted one week after the
first site visit. The final version of each report shall be submitted
to the FPO three weeks after the site visit.
n. Monthly and Annual Progress Reports: The Contractor shall
provide brief monthly technical progress reports to the FPO which
clearly indicate the contract tasks which were to be performed in the
prior month, a description of the progress made in completing these
tasks, problems encountered or remaining from the prior month, expected
approach to resolve problems from the prior month, tasks for the
current month, and any budgeting implications or significant concerns
to
[[Page 66298]]
be addressed by the FPO. In addition, the monthly progress reports
shall provide a brief review of the status of the contract budget for
the respective Phase, with separate presentations (by tasks and
subtasks) of the original amount budgeted, funds expended to date,
funds expended in the prior month, and the remaining balance of funds
in the contract. The first two monthly progress reports shall contain a
communication plan which details how all relevant parties shall be
updated regarding project activities. This communication plan shall be
updated as necessary in the monthly progress report.
At the end of each project year, the Contractor shall prepare an
in-depth annual progress report, summarizing the status of the
evaluation cross sites and in each site as well as activities of the
evaluation and the consortium, accomplishments, and problems
encountered during the year. This report shall also include a detailed
plan for activities in each site during the coming year. Within one
month of submitting the annual report for approval, the Contractor
shall provide an in-depth briefing on the progress of the study and
initial findings in Washington, DC, for ACYF staff. Following those
briefings, after receiving input from ACYF staff, the Contractor may be
required to present a similar briefing for a Congressional audience. In
all briefings, the Contractor may be required to collaborate with local
researchers involved with Early Head Start evaluation.
o. New Data Collection Instruments Protocols and OMB Clearance:
Within three months of the beginning of each new phase or as is
necessary, the Contractor shall submit a protocol and OMB clearance for
any new measures to be added to or changed from the originally approved
protocols. The form of these deliverables shall be similar to form
specified above for Data Collection Instruments Protocol, Draft and
Final, and OMB Clearance Package.
p. Consortium and Technical Work Group Reports: Within one month
after each consortium meeting and subcommittee meetings and within one
month of each Technical Work Group meeting a written report shall be
submitted to ACYF. All meetings of these bodies shall be reported in
separate reports, even though Technical Work Group meetings may be
embedded in the consortium meetings.
q. Collaborative Contractor Coordination Reports: One week
following each meeting with the HSFIS or Training and Technical
Assistance Contractor, a report shall be submitted to ACYF and to the
relevant Contractor.
r. Reports of Data Returned to Sites: Timeliness and Usefulness of
Data Turnaround: Reports of data disks returned to sites, site
printouts, and reports of assessments of the Contractor's activities at
local sites shall be submitted to ACYF by August 31st of each Phase.
s. Reports of Activities to Support Continuous Program Improvement:
By September 30th of each Phase, a report shall be submitted
summarizing the Contractor's role in Continuous Program Improvement
activities and progress.
t. Phase Reports: For each Phase, the Contractor shall produce
Draft and Final Report/s that shall incorporate data collected and
analyzed around the intended purposes and plan of the project. These
reports shall be due in draft form August 31 and in final form
September 30 of each Phase, or as determined between ACYF and the
Contractor. Each report shall have attached relevant local researchers'
reports, and provide an overview that integrates national and local
findings. The reports shall be presented in the following approximate
sequence:
``Report of Characteristics of Early Head Start Programs'' which
shall be an analysis of first year HSFIS data together with site
profiles from impact evaluation sites, co-authored by local researchers
and program staff.
``Pathways to Quality Study'' which shall be an analysis of quality
data from sites in describing the various procedures and successes of
programs in attaining program quality. There shall be attached local
studies focused on improving program quality. The national Contractor
shall provide an overview that integrates findings from the national
and local studies.
``Impact Studies'' of this project shall compare program to
comparison groups and also address the question: for which children and
families were there impacts under which conditions? Local research
studies focused on this question shall be attached and the national
Contractor shall provide an overview that integrates findings from the
national and local studies.
The ``Study of Program Variations'' shall first describe, then
examine in depth the site profiles in relation to impact data collected
to examine the questions pertaining to which children and families
benefitted under what conditions of Early Head Start program
variations. Local research reports that address the question shall be
included and integrated.
``Studies Directed Towards Specific Policy Concerns,'' shall
examine potential studies nested in the data set, i.e., analyzing
across sites the added effect of Early Head Start to child care and in
transition from welfare to work.
``Studies of Impact in a Longitudinal Context'' shall be an
analysis of findings in a longitudinal context. Local research reports
that address the question of change over.
u. Interim Report: The Contractor shall produce an Interim Report,
due September 1, 1997, which will summarize findings to date for the
study. This report may require integration with other studies and
evaluations of services for infants and toddlers, such as the CCDP
evaluation.
v. Final National Report: The Contractor shall produce a Final
Report which provides a national assessment of Early Head Start program
implementation and program impacts across the programs examined. This
report shall be comprehensive of the entire 5-year duration of the
project and shall include and integrate findings from local studies,
but maintaining the integrity of the separate studies.
The Report shall draw conclusions about the following issues (as
well as other relevant issues raised during the course of the
evaluation):
(1) Were nationally-defined Early Head Start objectives met?
(2) Were program implementation objectives realized?
(3) To what extent were continuous improvement objectives realized?
(4) To what extent and under what conditions were programs able to
implement quality services?
(5) What short- and long-term impacts did Early Head Start programs
have on children, families, communities and staff?
(6) For which children, families, communities and staff under which
conditions was Early Head Start able to realize its objectives? What
else was learned about child, family, community, staff effects through
Early Head Start?
(7) To what extent did different prototypes of Early Head Start
variation emerge and what kinds of outcomes were associated with
various prototypes?
(8) What was learned through analyses of subgroups in Early Head
Start with additional implications for public policy;
(9) What were the longitudinal effects of Early Head Start under a
variety of conditions, including risk and program variation;
(10) How did the study of Early Head Start programs advance the
methods of program evaluation?
In the Report, the Contractor shall discuss how the contents of
this Report
[[Page 66299]]
relate to any findings and recommendations presented in previous work
produced under this contract. The Contractor also shall provide a
discussion of the findings in relation to the literature in the field.
The discussion of the literature shall be based on a revised version of
the preliminary literature review. These revisions shall take into
account new work in the field as well as information produced by
Contractors under related studies.
The Contractor shall submit a draft outline of the final report to
the FPO by the end of the ninth month before the end of Phase V. The
FPO will have four weeks to review and approve the outline. The outline
shall include a framework for a stand-alone Executive Summary. The
draft final report based on the approved outline shall be submitted by
the end of the tenth month before the end of Phase V. The FPO shall
have four weeks to comment on the report and to obtain comments from
other HHS staff and the Technical Work Group and the consortium. The
Contractor shall plan to revise the draft at least twice based on
comments from the FPO and other ACYF staff prior to submitting the
final report to the FPO for approval. The Contractor shall make a
presentation to Federal staff four weeks after submission of the draft
final report. Issues raised in response to the presentation shall be
considered in preparing the final version of the report. The final
report and a camera ready copy of the final report shall be submitted
to the FPO for approval by the end of the 60th month after contract
effective date. The final report shall include a stand-alone Executive
Summary which must not exceed fifteen pages in length. A copy of the
Report of Evaluation Outcomes and of the Executive Summary shall be
submitted on IBM PC compatible 3.5 inch 1.4 megabyte DS/HD diskettes in
Wordperfect 5.1. In order to accommodate a publishing plan, the
Contractor shall submit line item quotes reflecting the exact costs of
research, writing, editing and copy preparation associated with the
copies of the Final National Report and the Executive Summary
(including one unbound camera-ready copy of each report).
w. Data Files: The Contractor shall produce a public use data
diskette for an IBM PC Compatible 3.5 inch 1.4 MB DS/HD diskette at the
conclusion of each Phase and at the end of the project for purposes of
data archiving. Documentation shall include file and record layout,
data dictionary including coding keys, a dump of the first and last 20
records of the data set and a summary of the processing including edit
conditions and software used for analysis. The file shall contain no
personal identifiers or confidential information.
It is the intent of ACYF that data should be publicly available for
secondary analysis and publication of results as soon as possible
following the completion of the contract. Prior to the end of the five
year project, however, approval of the FPO and consideration by the
consortium shall be required for publications, presentations or other
uses of the data th
This text is long and has been trimmed here. Open the source document for the complete record.
This is a copy of a public record, reproduced as it was published. It is not legal advice, and it may not be the version a court would rely on. Check the official source before you cite it.