Article | Open | Published:
Assessing Quality of Early Care and Education: Lessons Learned from San Francisco’s Gateway to Quality
International Journal of Child Care and Education Policyvolume 6, pages55–70 (2012)
This 10 year overview of a citywide program implemented to improve the quality of childcare and education in San Francisco provides a description of critical issues related to implementation of an assessment process as well as factors that influence success related to initiating and sustaining quality improvement efforts in early care and education (ECE). The population of interest included all licensed childcare programs and family childcare homes in the city and county of San Francisco receiving public funding and those serving families with subsidies. The assessment tool chosen by city childcare leaders was the Environmental Rating Scales (ERS) for preschool, infant toddler and family childcare sites. Quality improvement plans were developed based on the results of the assessment of each classroom or family childcare home, followed by technical assistance targeted at improving the quality of the environment and teaching practice. Because assessment scores are currently tied to city funded subsidies, motivation for participation in the program becomes a critical factor for those programs that depend on the subsidies. Lessons learned can support other endeavors to improve the quality of childcare and education and include engaging ECE teachers in intentional dialogue that results in internal motivation to improve the quality of their teaching, along with systemic supports that include teachers, families and funding streams to sustain quality care.
There are strong demands for high quality early childhood learning experiences in the United States as many young children experience non-maternal childcare during their early years. Low quality care in the first few years of life can have a long-lasting impact on a child’s growth and behavior. Over the past two decades, there has been a growing body of research linking high quality, community-based early care and education (ECE) to a variety of positive developmental outcomes for young children (Shonkoff & Phillips, 2000). Evaluations of carefully crafted, high-quality demonstration preschool programs for children living in poverty have also shown benefits to their later academic achievement and cost savings in future remedial education services (Karoly, Kilburn & Cannon, 2005). High-quality childcare programs can provide young children with a firm foundation of early cognitive and social competencies and eventually narrow the achievement gap in future years, especially for children from low-income communities. However, it is also widely recognized that many young children in the United States, particularly lower-income children (Matthews, 2006), are in childcare programs that are of low to mediocre quality and do not support children’s positive development (Helburn, 1995; NICHD Early Child Care Research Network, 2000).
As a result of previous research that promises significant returns on investment in quality ECE programs, improving children’s access to high-quality childcare programs has become a national priority. Indeed, many states and local communities are implementing a number of initiatives to improve the quality of childcare centers especially those that care and educate lower-income children. Many of these initiatives include assessment of classroom quality using the Environment Rating Scales (ERS), including the Early Childhood Environment Rating Scale Revised (ECERS-R; Harms, Clifford & Cryer, 2005) for preschool classrooms, the Infant/Toddler Environment Rating Scale Revised (ITERS-R; Harms, Cryer & Clifford, 2006) for infant and toddler classrooms, the School-Age Care Environment Rating Scale (SACERS; Harms, Jacobs & Romano, 1995) for group-care programs for children of school age, and the Family Child Care Environment Rating Scale revised (FCCERS-R; Harms, Cryer & Clifford, 2007) or the Family Day Care Rating Scale (FDCRS; Harms & Clifford, 1989) for home-based settings. Results of these assessments inform teachers and providers about the quality of their classrooms. Some initiatives also offer technical assistance targeted at needs identified in the assessment to assist teachers in improving quality along dimensions measured in the ERS. Many states and local initiatives use ratings in ERS assessment to determine public financial subsidy and additional incentives to classroom and family childcare sites.
The context for this paper documents the past 10 years of community effort to improve ECE quality in San Francisco. The authors aim to contextualize the city-wide childcare quality improvement initiative, a project entitled ‘Gateway to Quality (GTQ)’, and to discuss lessons learned from such efforts to raise quality standards. The ultimate goal of GTQ was to develop a systematic, coordinated and collaborative approach to provide high quality childcare and education to infants, toddlers, and preschoolers in San Francisco by determining the baseline status of quality in early care and education, by implementing tri-annual assessments of classrooms and family childcare homes, and by providing technical assistance, along with fiscal and material resources to support improvement. In this article, the authors present an overview of GTQ, describe how assessments and support for quality improvement were provided, and discuss implications and key characteristics that appeared to influence sustainable quality improvement in early childhood education and care.
Program Development and Context
Historically, regulations and quality controls for early care and education in the U.S. have received little attention at the federal level and been mostly developed at local and state levels. Licensing is the most prevalent system to regulate early childhood programs. However, the requirements and process for licensing vary by state and tend to include minimal standards that are not sufficient to ensure quality care. Accreditation systems developed by national associations such as NAEYC entail higher standards but the costly and lengthy process is often a barrier for most childcare centers, which influences the feasibility of obtaining accreditation.
More recently, the development and implementation of Quality Rating and Improvement Systems (QRIS) are occurring in many states across the countryin order to improve childcare quality by defining quality standards, educating consumers and providers on program quality, and providing incentives and support for quality improvement. A QRIS is “a unique tool for system reform that has the potential to reach programs that serve a wide range of children and are financed by many public and private sources, including parent fees” (QRIS National Learning Network, n.d., para 1).
Key elements of QRIS include adequate funding, sufficient marketing and public awareness, availability of technical support, and appropriate structure and oversight (Zellman & Perlman, 2008; Zellman, Perlman, Le, & Setodji, 2008). In recognition of the need to build and support quality early care and education options for families, currently 26 states and locales in the U.S. have developed statewide QRIS and several others are in development. The processes by which these QRIS came into existence, the eligibility criteria and requirements for participation, the rating processes, quality standards, and evaluation of these systems differ greatly from state to state, as do the agencies that administer and implement them.
While California has not yet developed a statewide QRIS, professionals and scholars in many counties and cities have been working to establish measures and indicators for quality early care and education. The City and County of San Francisco, the focus of this paper, has invested considerable local government resources to build a stable supply of quality care and education programs by improving program facilities, maximizing state and federal subsidy revenues, supporting training, providing public health and mental health consultation services to programs, and by enhancing teacher salaries. To better understand the impact of years of public and private investments that had been supporting childcare and education, and to raise the overall quality of San Francisco’s childcare programs, city leaders in early care and education turned their focus on identifying a baseline measure of quality, done in a systematic and productive way that could engender positive change for childcare providers.
In the spring of 2001, San Francisco’s Child Care Planning Advisory Committee (CPAC) developed a strategic plan that identified the need for a comprehensive approach to improving the quality of childcare. The first step was to identify a valid and reliable assessment that could determine if the substantial investments in wages, training, professional development, and health and mental health consultation were being translated into improved practice and whether weaknesses in programs and staff preparation could be identified and targeted for future resources.
A second step was initiated in 2002 by the Department of Human Services when it imposed new conditions on childcare programs participating in WAGES Plus, a program that provided funds to preschool sites and family childcare homes to bring staff wages to a minimum standard. Staff participating in the WAGES Plus program were required to take a course at City College of San Francisco (CCSF) on the Environmental Rating Scale and to self-assess the quality of their programs using the appropriate scale. At the same time, Bright Beginnings, a program to increase the supply and quality of infant and toddler care, had initiated similar assessments using the Infant/Toddler Environment Rating Scale Revised (ITERS-R) and the Family Day Care Rating Scales (FDCRS) in family childcare (FCC) homes and infant/toddler centers. The significant financial rewards to providers of the Bright Beginnings and WAGES Plus programs served as built-in incentives and made it less difficult to secure provider acceptance of having an outside assessment of their centers and homes.
An Advisory Board was formed in 2002 consisting of a collaboration of city departments, nonprofit agencies, local foundations, childcare programs, and educational institutions, all with strong commitment to improving the quality of childcare. The goal was to develop a system to evaluate the quality of center-based and family childcare sites in San Francisco. Key stakeholders in the Advisory Board contributed to design of the project and monthly meetings took place to make initial policy decisions and coordinate the project. Table 1 describes the agencies involved in the program and their related activities.
Gateway to Quality
After the initial pilot period, the program initially called Partners in Quality Child Care (PQCC), was renamed to Gateway to Quality (GTQ). The Advisory Board reviewed the qualifications and a position description for a program coordinator to oversee the project, organize assessor training, schedule site visits, and work with the provider community to design a mechanism for communication that would be informative and also encourage their participation. In addition, the program recruited, trained and monitored a team of eight assessors.
The Environmental Rating Scales (ERS), selected as the assessment tool, included the Early Childhood Environment Rating Scale Revised (Harms, Clifford & Cryer, 2005) for center-based preschool classrooms, the Infant/Toddler Environmental Rating Scale Revised for infant/toddler classrooms (Harms, Cryer & Clifford, 2006) for center-based infant/toddler classrooms, and the original Family Day Care Rating Scale (Harms & Clifford, 1989) for family childcare homes. The scales are designed for different age groups of children and broadly measure seven different domains of classroom quality: the physical environment, health and safety, the language and literacy environment, daily activities, interactions, classroom daily structure, and support for parents and staff. Each subscale of ERS has a 1–7 Likert scale with 1 indicating inadequate quality, 3 indicating minimal quality, 5 indicating good quality, and 7 indicating excellent quality. Since the inception of the program, the FDCRS has been revised as the Family Child Care Environment Rating Scale Revised (Harms, Cryer & Clifford, 2007), and the assessment team employed the new version beginning in 2008.
Assessors’ scope of work included performing assessments, reporting results to providers, writing a quality improvement plan and providing technical assistance targeted at the needs identified from the assessment. The eight assessors had a broad range of backgrounds, including prior teaching experience in early care and education, certification in ECE tools, advocacy, consulting, small business practices as well as language facility in English, Spanish and Cantonese. The assessment team received training on the ERS that included an introduction and overview followed by four days of field training. As the project continued, a decision was made to train two staff to reliability with one of the authors of the ERS. These individuals then served as “anchors” for the team, able to provide training and ongoing reliability checks of the assessor team every three months to maintain reliability. All assessors achieved 85% inter-rater reliability before performing external assessments. Later in the process, the assessment team was divided into center-based assessors and family childcare based assessors. This division of labor allowed assessors to better hone their expertise by focusing on one type of environment or the other.
Participants and Procedures
The target population for assessment included all licensed childcare sites and family childcare homes in the city and county of San Francisco receiving public funds and those serving families with subsidies. The providers that initially took part in the assessment and quality improvement process belonged to at least one of two subsidy programs. The first was WAGES Plus, a program administered by the Human Services Agency. Bright Beginnings was the second and was administered by Children’s Council of San Francisco. Bright Beginnings provided fiscal support for preschool sites and FCC homes that care for infants/toddlers. Participants in WAGES Plus initially received increased funding for taking part in the evaluation process. Participants in Bright Beginnings received increased funding depending upon the assessment score. All providers received improvement plans and technical assistance.
Selection of assessment sites was determined by zip codes representing the childcare classrooms and family childcare sites in the lowest income areas. Once classroom teachers were ready for the assessment, the project director matched assessors with providers, depending on assessor availability and linguistic needs. Assessment dates were set at the mutual convenience of the provider and assessor. Unless there were extenuating circumstances, each family childcare home and each preschool site had one assessor as primary contact. An assessor would assess all classrooms at each site. Additionally, each assessor would stay with the site or provider throughout the process, including score reporting, improvement planning, and providing technical assistance.
Assessments generally took place in the morning, over a period of approximately four hours in each classroom. Near the end of the assessment, assessors interviewed appropriate staff to obtain additional information. Assessments were scored off-site and no detailed information was provided at the time of assessment. A quality improvement plan was written by the assessor based upon assessment results, followed by collaborative planning with the provider. Using a standard form designed specifically for this project, each item on the scale was discussed and the missed indicators for each item were identified. Specific observations about the classroom or FCC site were discussed, and suggestions for improvements were included on the score reporting form. Technical assistance strategies were collaboratively determined between the assessor and provider, based upon discussion and assessment results. Technical assistance included consultations, workshops, suggested resources, as well as identification and support with grant opportunities. The report meeting was the opportunity for dialogue related to strengths and needs of sites/homes and was the beginning of the collaborative relationship between assessor and staff.
Overview of Assessment Results
The goal of the initial assessment was to identify the baseline of quality in the city and to demonstrate strength based approach by identifying the strengths of the centers and family childcare homes while offering suggestions and resources to support improvement in the areas of weaknesses. Quality Improvement Grants were offered as an incentive to encourage participation.
Pilot assessments were conducted from 35 centers and 48 family childcare homes using ERS during 2003–2004. The initial scores were relatively high showing that 83% of the center based classrooms and 63% of family childcare homes scoring higher than 5. Possible factors that have contributed to high scores may include the selective sample of participants who had a high level of preparation prior to assessment. Providers were required to take a community college class on the assessment tool, including performing a self-assessment. Moreover, providers were visited by a consulting group with expertise in the scales to provide pre-assessment technical assistance and education. Many providers took advantage of the opportunity to improve their environments and adjust practices before the assessment took place. The pilot assessment participants were also proactive in joining this project. Participating providers tended to be active in the community, regularly attending workshops and ongoing professional development activities, and were members of local networks. This active, engaged, connected, and supported segment of the provider population may not be representative of the city overall.
After the project pilot period which also gave assessors sufficient time to hone their assessment knowledge and skills, more childcare centers and family childcare homes were recruited for further assessment. The total number of sites participating in assessment since 2004 includes 277 family childcare homes and 204 childcare centers which contain a total of 563 classrooms. Table 2 presents numbers of the assessed classrooms by fiscal years.
Tables 3, 4, and 5 depict aggregate assessment scores for 2004–2011. The 7-year ECERS mean score is 4.559 (SD.848) and the ITERS mean score is 4.404 (SD.811). The mean score on FCCERS is based on a 3 year-assessment which is 3.747 (SD .906). Assessment for the 2007–2012 also include reassessments. As GTQ assessments were based on voluntary participation, the data were not necessarily representative of all programs in the city and county of San Francisco. Assessment scores were collected on a three-year cycle and mainly used as the basis for a quality improvement plan and technical assistance. Although the mean scores are presented in Tables 3, 4, and 5 to provide an overall picture of the assessment results, further statistical analyses were not conducted due to the nature of the data.
In the pilot phase and for several years in the process of the GTQ assessment, centers and FCC sites were able to apply for grants to receive funding for items directly related to suggestions on the quality improvement report. The funding was linked to the assessments and improvement plans were prioritized based on identified needs from the ERS assessments. San Francisco’s Child Care Planning Advisory Committee (CPAC) used the assessment data to draft a reimbursement model for providers that is based on quality criteria.
Currently scores from these assessments are tied to continuing participation in city funded subsidies and reassessment is required every three years. All classrooms or family childcare homes must receive a score of 3.0 or higher to participate in Infant Toddler Sustaining Grants program (ITSGP), which officers a monthly stipend for sites serving infants and toddlers of low income families.
Preschool for All (PFA) has different eligibility requirements. This program offers participating programs per children imbursement for all 4-year-olds enrolled; access to training and professional development opportunities for the teachers and administrative staff; and bonuses for teachers with BA degrees. Childcare sites in PFA must receive a score of 4.5 or higher in all classrooms serving 4-year-olds and a site-wide average of 4.0 or higher if they are serving other age groups. In 2010, PFA contracted with GTQ to coordinate “PRE-PFA” services to support some of the lower quality programs to be able to meet the baseline criteria for PFA. A new PRE-PFA designation was created so that PFA could provide supports for quality improvement to programs that had not previously been able to meet the baseline criteria. Once PRE-PFA status is awarded, programs can access grant funding to purchase materials, renovate space and receive on-site professional development services as needed.
In 2007, San Francisco First 5, a city initiative, coordinated the development of a city wide technical assistance program which included coaching services. Currently, approximately 40 classrooms per year receive coaching. GTQ expanded its services to meet this initiative and adopted a coaching model guided by a belief in the importance of teachers’ reflective practice in improving the quality of their teaching (Schön, 1987) to make substantive and sustainable changes. Consequently, GTQ coaching is relationship based and focuses on supporting teachers by engaging in critical analysis of practice, learning tools, and developing dispositions needed to continuously improve practice in support of better child learning and development. Coaching sessions in GTQ are designed to engage teachers in thinking about their practice, with coaches modeling and scaffolding a teachers’ development of a reflective practice.
Implications and Recommendations for Public Policy and Future Assessment
Access to appropriate childcare is still limited for many families in San Francisco. For example, 3,600 children were waiting for subsidized care and 43 percent of these children were infants and toddlers as of early 2011 (San Francisco Department of Children, Youth and Their Families, 2011). In addition to the limited supply, quality assurance in such care is an even more critical concern. Understanding the needs across early care and education programs by obtaining detailed assessment scores has allowed the city and county of San Francisco to make more informed decisions about resource allocation to support quality early childhood education and care.
While conducting assessments using the ERS enhanced program accountability, policy development, and funding to increase the quality of childcare, there have been many lessons learned related to the assessment process and quality improvement in early care and education programs. The first of many lessons learned was that programs fall into several areas with defined characteristics. Those with stable, ongoing high quality practices frequently have a clearly articulated vision of quality, sound program infrastructure and strong leadership. Such programs consistently score on the higher end of the ERS assessment scales and often have a reputation for providing quality services with a long waiting list for families.
Second, there are programs with emerging quality practices. These programs often have many practices in place to support some quality practices and typically score in the middle range of the ERS scales. These programs often struggle in two or more of the following key areas: administrative systems, fiscal and budget development, professional development of teachers and administrators, staff evaluation and HR systems, and curriculum development. Most of these programs do not have an articulated vision for the care and education they provide.
Lastly, programs with low quality practices are often characterized by a low level of awareness of best practices, and struggle in many of the key areas above. Such programs tend to score on the lowest end of the ERS tools and often have teachers with the lowest levels of formal education. Assessment processes and approaches for quality improvement may be more effective if they are applied with a better understanding of each program’s specific comprehensive characteristics and needs. In the following, recommendations for future assessment are identified based on lessons learned from our experience.
The definition of quality can be subjective among researchers and practitioners and is often directly related to how it is measured and quantified. Many assessments on childcare quality have used a global approach, whether they measure global classroom quality or simply consider a few factors at the program level such as staff-to-child ratios and staff training. More recently, researchers have begun to focus on two primary categories of global quality — structure and process (Cassidy et al., 2005), and studies show that process quality provides more powerful predictors of child outcomes than does the structural feature of the classrooms (Howes et al., 2008; Mashburn et al., 2008). Classroom teachers also tend to emphasize process quality more than structure quality, which may create discrepancies between ERS assessment and classroom practice (Park, Ferretti & Ames, 2012).
Although ERS assessments offer a useful framework for identifying areas to improve in low quality programs, the scales place less emphasis on interactional processes, such as responsive care giving and instructional support that are directly linked to child well-being. For example, the constant comparative analysis of the ECERS-R reveals that the scales measure structure quality rather than process quality (Cassidy et al., 2005).
The ERS can be helpful for programs that are in the early stages of developing high quality services because identifying specific areas for potential improvements may motivate ECE providers to engage in their efforts to make positive changes in their programs. However, continuing to employ the ERS as the only tool for assessment and reassessment can be redundant when programs have maintained good quality scores for a number of years.
In order to sustain continuous quality improvement, enhance classroom environments and teacher practice, and to capture a comprehensive picture of program quality, using additional tools and programmatic information (e.g. CLASS, program administrative data, teacher data and turnover rates) is recommended. Conducting simultaneous assessments to contain cost and gather additional programmatic information can be also considered. To be more efficient with resources, the number of ERS items can be reduced while maintaining the validity and reliability of the instrument by conducting factor analysis on previous ERS assessment data.
Assessment Staff Capacity
An effective assessment system requires staff capacity to interface with multiple stakeholders, schedule assessments, explain how the tools connect to child development and best practices, implement quality assurances, train assessors and maintain their reliability. Sophisticated resources and capacity are needed for data collection, management and analysis. Once the scores are linked to funding, it is critical to ensure that all assessors are trained to reliability and monitored quarterly. Training and maintaining reliability standards requires considerable time and effort. To maintain that investment, it is important to create satisfying and rewarding positions. Historically, there have been a limited number of highly skilled, multi-linguistic applicants who met minimum requirements with background in early child education.
It has been the authors experience that highly trained professionals experience decreased job satisfaction when only performing assessments on a fulltime basis. San Francisco Assessors experienced greater job satisfaction when given opportunities to also engage in more supportive roles with providers while continuing to provide assessments.
In addition, strong linguistic and cultural capacity to communicate with and support the current diverse workforce is critical to an effective assessment system. Hiring multilingual assessors can increase the flexibility to produce reports in multiple languages. Implementing a comprehensive, citywide data collection system that can be used by both assessment and support services is also recommended to reduce administrative burden, evaluate trends, and make data more accessible to participants and funders.
Several challenges were encountered with the assessment process. Detailed post assessment reports were developed that included both strengths and needs in all areas. Although most providers appreciated feedback, writing such reports can be costly and often underutilized. Serious thought about how assessment results will be used by providers and agencies in the design phase will increase cost efficiency. The use of ERS scores to determine funding eligibility can also lead to ‘staging’ and can compromise the ability to use the score data as a valid tool for sustainable quality improvement. When programs are seeking a specific score it increases motivation to select specific teachers and even children to be present on the assessment day. Assessing classrooms every three years also makes linking assessment data to quality improvement difficult, especially in light of high rates of turnover in ECE.
To address these concerns, we recommend (1) creating quality improvement plans at a center level instead of an individual classroom level, or include the development of the quality improvement plans report as a feature of on-site technical assistance services for cost savings; (2) investing in a data system that includes automated reporting and scheduling components; (3) considering a broad window for assessment visits that allows assessments to take place on any day when a program is open and serving children. This can increase the fidelity of assessment data and make scheduling more flexible; (4) assessing classroom quality more frequently to yield current information that can be more easily linked to professional development and quality improvement efforts.
Assessment-related Systems and Infrastructure
The fact that assessment scores are linked to funding causes childcare providers’ anxiety and reduces their openness or willingness to engage in thoughtful conversations about the results of the assessments. Family childcare providers appear more resistant to the Family Child Care Environment Rating Scale-Revised and the assessment process. Given that most childcare programs, specifically those serving children from low income families, are significantly underfunded, providers are eager to access any new funding source available to them. Programs with emerging quality practices in particular are often motivated to participate in assessments for potential new funding and additional benefits. However, the feedback to various programs that relies on assessment information often lacks unified information regarding the system, goals, and resource connections, causing confusion, resistance and anxiety.
Recommendations for improvement include (1) using multiple sources of assessment data and other programmatic information to determine funding; (2) ensuring capacity for outreach, communication, and knowledge-building on the interpretation and application of assessment tools; (3) implementing a multi-pronged approach that includes monthly community meetings, webinars, and blogs about assessment tools; (4) engaging new FCC participation by offering facilitated and ongoing community-based meetings for providers to prepare for assessment, to strengthen buy-in and reduce resistance. For example, many QRIS offer ‘pre-assessment coaching’ prior to an initial assessment. These types of relationship-based, content-focused groups may be helpful in making important linkages with family childcare, or provide all assessed homes with on-site technical assistance immediately after their assessment.
In this paper, the authors discuss a quality improvement initiative in the city and county of San Francisco, measuring the current quality of childcare programs and provision of funding based on assessment data. In 2012, city funders in San Francisco announced their commitment to designing and implementing a citywide QRIS. Funders include the Department of Children, Youth, and Their Families, First 5-San Francisco, and the San Francisco Human Services Agency. Their recent request for proposals (RFP) included assessment by multiple tools for more comprehensive assessments as well as quality improvement services through coaching as an essential component for the optimal implementation of San Francisco-QRIS.
Sustainable QRIS that has a long lasting impact requires more holistic and qualitative approaches to quality improvement in addition to micro-level interventions with individual teachers and classrooms. As a city or state embarks on the development of QRIS, it is crucial to intentionally build inclusive and responsive systems that represent a consistent, cohesive vision of support for ECE programs, especially for those with low or emerging quality. The current system of assessment and incentives in San Francisco rewards those programs that meet what is deemed a ‘good’ quality score. With the new process there may be more opportunity to take into account the full breadth of needs, values and interests of programs that may be struggling to meet baseline criteria or understand what quality care means for children. Programs that have emerging quality are often capable of meeting the minimum score criteria on the day of assessment but struggle to consistently maintain that quality. Their desperate need to access funding for survival causes stress to many as they try to obtain a qualifying ERS reassessment score every 3 years.
High stakes systems with built in support for motivated low scoring programs can lead to less anxiety among providers. Less provider anxiety can result in increased ability to engage in intentional and reflective dialogue related to quality improvement. In a high-stakes, score-focused system, even with multiple measurements it is important to not lose sight of the critical need for a strong vision, communicated broadly, as well as support for reflective practice, collegial relationships, responsiveness to family needs, and adult-child relationships as essential components of high quality. Framing quality as an intentional process in which all educators can engage and integrate comprehensive information into funding criteria will promote more sustainable quality improvement in early care and education programs.
Cassidy, D. J., Hestenes, L. L., Hansen, J. K., Hegde, A., Shim, J. & Hestenes, S. (2005). Revisiting the two faces of child care quality: Structure and process. Early Education and Development, 16(4), 505–520.
Harms, T. & Clifford, R. M. (1989). The family day care rating scale. New York: Teachers College Press.
Harms, T., Clifford, R. M., & Cryer, D. (2005). The early childhood environment rating scale: Revised edition. New York: Teachers College Press.
Harms, T., Cryer, D. & Clifford, R. M. (2006). The infant/toddler environment rating scale: Revised edition. New York: Teachers College Press.
Harms, T., Cryer, D. & Clifford, R. M. (2007). The family child care environment rating scale: Revised edition. New York: Teachers College Press.
Harms, T., Jacobs, E. V. & Romano, D. (1995). The school-age care environment rating scale. New York: Teachers College Press.
Helburn, S. W. (1995). Cost, quality, and child outcomes in child care centers. Denver: Department of Economics, Center for Research in Economics and Social Policy, University of Colorado at Denver.
Howes, C., Burchinal, M., Pianta, R., Bryant, D., Early, D. & Clifford, R., Barbarin, O. (2008). Ready to learn? Children’s pre-academic achievement in pre-kindergarten programs. Early Childhood Research Quarterly, 23, 27–50.
Karoly, L. A., Kilburn, M. R., & Cannon, J. S. (2005). Early childhood interventions: Proven results, future promise. [Adobe Digital Editions version]. Santa Monica, CA: RAND Corporation. Retrieved from http://www.rand.org/pubs/monographs/MG341
Mashburn, A. J., Pianta, R. C., Hamre, B. K., Downer, J. T., Barbarin, O. A., Bryant, D., Burchinal, M., & Early, D. M., Howes, C. (2008). Measures of classroom quality in prekindergarten and children’s development of academic, language, and social skills. Child Development, 79(3), 732–749.
Matthews, H. (2006). Child care assistance helps families work: A review of the effects of subsidy receipt on employment. Washington, DC: Center for Law and Social Policy. Retrieved from http://www.clasp.org/admin/site/publications/files/0287.pdf
NICHD Early Child Care Research Network (2000). The relation of child care to cognitive and language development. Child Development, 71(4), 960–980.
Park, S., Ferretti, C., & Ames, G. (2012). Quality assessment and improvement: Child care providers’ perspectives. Poster session presented at the Annual Meeting of the American Educational Research Association, Vancouver, Canada.
QRIS National Learning Network (n.d.). Quality Rating and Improvement Systems Framework. Retrieved from http://qrisnetwork.org/ourframework
San Francisco Department of Children, Youth and Their Families (2011). 2011 community needs assessment. Retrieved from http://www.dcyf.org/modules/showdocument.aspx?documentid=10
Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. Jossey-Bass: San Francisco.
Shonkoff, J. P, & Phillips, D. A. (2000). From neurons to neighborhoods: The science of early childhood development. Washington, DC: National Academies Press.
Zellman G. L. & Perlman, M. (2008). Child-care quality rating and improvement systems in five pioneer states: Implementation issues and lessons learned. [Adobe Digital Editions version]. Retrieved from http://www.rand.org/content/dam/rand/pubs/monographs/2008/RAND_MG795.pdf
Zellman, G. L., Perlman, M., Le, V. N., & Setodji, C. M. (2008). Assessing the validity of the Qualistar early learning quality rating and improvement system as a tool for improving child-care quality. [Adobe Digital Editions version]. Retrieved from http://www.rand.org/content/dam/rand/pubs/onographs/2008/RAND_MG650.pdf