A Comparison of International Child Care and US Child Care Using the Child Care Aware — NACCRRA Child Care Benchmarks

Abstract

This is a first of its kind study comparing the USA to other world countries utilizing the Child Care Aware — NACCRRA (National Association of Child Care Resource and Referral Agencies) Child Care Benchmarks related to health and safety rules and regulations. A team of researchers analyzed the child care/early care & education rules and regulations from the USA and a selected group of countries to do a comparative analysis using the Child Care Aware — NACCRRA benchmarking scoring protocol. The results from the analyses were somewhat unexpected in that the scores between the USA and the other countries were not as statistically significant in the overall scores. However, when more specific benchmarks were compared statistically significant differences did appear in the health & safety and professional development areas.

Introduction

The purpose of this paper is to compare several countries (N =20) and the United States on the Child Care Aware — formerly National Association of Child Care Resource and Referral Agencies (NACCRRA) Child Care Benchmarks that have used extensively in the USA to compare state regulatory and monitoring policy and implementation. The use of these benchmarks has been very useful in comparing states in the USA on an agreed upon series of child care benchmarks that have a great deal of support in the research literature (American Academy of Pediatrics [AAP] & American Public Health Association [APHA], 2012, 2013; NACCRRA, 2007, 2009, 2011). Previous research (Organization for Economic Co-operation and Development [OCED], 2006) has focused on early care and education policies in other countries which was a very important first step in making comparisons across countries. This paper will expand upon this comparison in order to begin applying the NACCRRA benchmarks to other countries and establish a baseline between the USA and other countries related to regulatory review and analysis. This study is important because it provides a common rubric for making comparisons between the USA and other countries that is reliable and valid (NACCRRA, 2007, 2009, 2011) related to regulatory analysis. As far as the author can determine from his extensive review of the literature, similar studies of this type have not been attempted utilizing a standardized rubric created by a major national child care organization. There have been other studies completed in which comparisons were made of other countries, the OCED (2006) Starting Strong II study and report is an excellent example of this type of analysis and is recommended reading for anyone interested in reviewing public policy analyses.

The child care benchmarks¹ utilized in this study are based upon the following key indicators: prevention of child abuse, immunizations, staff child ratio, group size, staff qualifications and training, supervision/discipline, fire drills, medication administration, emergency plan/contact, outdoor playground, inaccessibility of toxic substances, and proper hand washing/ diapering (NACCRRA, 2007, 2009, 2011). These benchmarks are more based upon the structural aspects of quality rather than on the process aspects of quality. This is an important distinction between the USA approach and the other countries approaches that becomes important in the explanation of results later in this paper.

Figure 1.

Differential Monitoring Logic Model Algrithm (DMLMA©): A 4th generation ECPQIM — Early Childhood Program Quality Indicator Model

Souce: Fiene, 2012.

Note. Definitions of Key Elements:

CI = Comprehensive Licensing Tool (Health and Safety)(Caring for Our Children)

PQ = ECERS-R, FDCRS-R, CLASS, CDPES (Caregiver/Child Interactions/Classroom Environment)

RA = Risk Assessment, (High Risk Rules)(Stepping Stones)

KI = Key Indicators (Predictor Rules)(13 Key Indicators of Quality Child Care)(NACCRRA Benchmarks)

DM = Differential Monitoring (How often to visit and what to review)

This paper also supports and expands the development of an Early Childhood Program Quality Indicator Model (ECPQIM)(Fiene & Nixon, 1985) which is in a 4^th generation (Fiene, 2012) as a differential monitoring logic model & algorithm helping to guide the program monitoring of child care/early care & education programs (see Figure 1).

Method

Data Collection Process

Data collection was done on a 100 point scale which is delineated in Appendix 1 as developed by the Child Care Aware — NACCRRA Research Team. The same scoring protocol that was utilized in developing the 2007, 2009, and 2011 Reports and comparisons of states by Child Care Aware — NACCRRA was employed in this study in comparing the average scores of the states and the 20 countries. The 100 point scale consisted of 10 child care benchmarks each worth 10 points: ACR = Staff child ratios NAEYC Accreditation Standards met (R1); GS = Group size NAEYC Accreditation Standards met (R2); Director = Directors have bachelor’s degree (R3); Teacher = Lead teacher has CDA or Associate degree (R4); Pre = Initial orientation training (R5); Inservice = 24 hours of ongoing training (R6); Clearance = Background check (R7); Devel = Six developmental domains (R8); Health = Health and safety recommendations (R9); and Parents = Parent Involvement (R10).

Data Scoring

The scoring protocol employed a total raw score approach of 100 points that was used to compare the countries on the 10 child care benchmarks in the aggregate. The scoring protocol also employed a standardized scoring approach (0 to 2 points) on each of the 10 child care benchmarks utilizing the following scale: 0.0 = Does not meet the Child Care Aware — NACCRRA Benchmarks; 0.5 = Marginally meets the Child Care Aware — NACCRRA Benchmarks; 1.0 = Partially meets the Child Care Aware — NACCRRA Benchmarks; 1.5 = Substantially meets the Child Care Aware — NACCRRA Benchmarks; 2.0 = Fully meets the Child Care Aware — NACCRRA Benchmarks.

Data Collectors

A team of undergraduate and graduate research assistants² at the Pennsylvania State University were the data collectors in which each of them reviewed the child care/early childhood rules/regulations/standards from a specific country and scored the rules/regulations/standards on the Child Care Aware — NACCRRA 100 point raw score protocol and the standardized (0 – 2) scoring approach.

Data Sources

The child care regulations selected were for preschool age children only in child care center setting in the 20 countries. Geographically the governmental jurisdiction closest to the national capital was used if applicable national regulations could not be found. More than the final 20 countries selected were reviewed but several countries needed to be dropped because they did not meet the above criteria or the regulations could not be found in English. This was more a convenience sample rather than a stratified scientific sample, a limitation of this study.

Results

The results from this study and analysis were totally unexpected. The results indicated no statistically significant differences between the USA and the other countries selected (Australia, Belgium, Norway, Finland, Sweden, Ireland, United Kingdom, Italy, France, New Zealand, Mexico, Greece, Canada, Austria, Portugal, Philippines, Turkey, Pakistan, Nigeria, Denmark, and Spain — these countries were selected because of their availability of child care/early care & education rules and regulations as described previously above in Data Sources) when comparing the total scores on the 100 point scale; the USA average for all 50 states scored 58 while the 20 countries average score was 56. However, a very different scenario occurs when looking at the ten individual child care benchmarks using the standardized 0–2 scoring protocol. The 20 countries selected in this study scored statistically higher on the following child care benchmarks: Director (t=7.100; p<.0001) and Teacher (t=7.632; p<.0001) qualifications. The USA scored statistically higher on the following child care benchmarks: Health/Safety (t=6.157; p<.0001), Staff Clearances (t= 3.705; p<.01), and Pre-Service (t=4.989; p<.001) /In-Service training (t=2.534; p <.02) (See Table 1 & Figure 2).

Figure 2.

Mean Comparisons between USA and Twenty Countries on Child Care Aware — NACCRRA Benchmarks

Note 1. Legend:

Parents = Parent Involvement (R10)

Health = Health and safety recommendations (R9)

Devel = Six developmental domains (R8)

Clearance = Background check (R7)

Inservice = 24 hours of ongoing training (R6)

Pre = Initial orientation training (R5)

Teacher = Lead teacher has CDA or Associate degree (R4)

Director = Directors have bachelor’s degree (R3)

GS = Group size NAEYC Accreditation Standards met (R2)

ACR = Staff child ratios NAEYC Accreditation Standards met (R1)

Scoring:

0.0 = Does not meet Child Care Aware — NACCRRA Benchmarks.

0.5 = Marginally meets Child Care Aware — NACCRRA Benchmarks.

1.0 = Partially meets Child Care Aware — NACCRRA Benchmarks.

1.5 = Substantially meets Child Care Aware — NACCRRA Benchmarks.

2.0 = Fully meets Child Care Aware — NACCRRA Benchmarks

Table 1. Mean Comparisons between USA and Twenty Countries on Child Care Aware — NACCRRA Benchmarks

The results showed that both the USA and all other countries mean scores were 58 and 56 respectively on the 100 point scale — this is a raw scale score and not the standardized score (0–2: see Table 1 and Figure 2) which was used in the comparisons for each benchmark. This is not a particularly good score if you think in terms of exams, but for states and countries with vastly complex bureaucracies maybe this isn’t as bad as it looks. Could it be that the USA is better than we think or is it that the USA and all other countries are providing just mediocre child care?

The reason for using aggregate data in this study was to be consistent in how data have been collected in the USA utilizing the Child Care Aware — NACCRRA Scoring Protocol. This did delimit the potential analyses for this study and the recommendation would be made in future studies to unbundle the results so that more detailed comparisons could be made. As mentioned in the introduction, the purpose of this study was to provide an initial baseline comparison between the USA and other countries on the Child Care Aware — NACCRRA Scoring Protocol.

Discussion

The purpose of this study was to extend the Child Care Aware — NACCRRA Child Care Benchmarks Scoring Protocol to an international sample comparison. As has been done by the National Science Foundation with math and science testing, these same types of comparisons have been made with the USA not fairing all that well on the math and science comparisons.

It appears that when it comes to child care benchmarks the USA actually appears to be in better shape than many advocates and experts would have thought when compared to other countries or is it that the other countries are providing the same form of mediocre care as it relates to these child care benchmarks. Remember that these benchmarks are heavily weighted towards the structural side of quality rather than the process side of quality.

However, when the individual benchmarks are analyzed then certain patterns occur which seem very consistent with the previous research literature. The 20 countries scored higher on the staffing benchmarks while the USA scored higher on the training and health/safety benchmarks. Clearly this is an indication reflecting public policy in the other countries as versus the USA. Many other countries place more emphasis on the process aspects of quality which involve staff and staff interactions with children. The USA has focused more on the structural aspects of quality which involve health & safety especially in the state licensing of child care. These structural aspects of quality are more easily quantifiable in state rules and regulations which is the locus of control for the licensing of child care. Since the USA does not have national standards that are required (the USA does have national health and safety standards that are recommended practice, such as Caring for Our Children (2012)) as is the case in so many of the countries in this study, this may provide a possible explanation for the results of this study. It will be interesting to see how Quality Rating and Improvement Systems (QRIS) which usually have some process standards impact this overall balance of structural and process aspects of quality. This is an area that needs additional research and more in-depth analysis.

So what does this tell us. I think it is a warning call as has been put forth by Child Care Aware — NACCRRA that we still have a lot of additional work to do in improving child care, not only in the USA, but worldwide. Just as the Child Care Aware-NACCRRA Report Cards (2007, 2009, 2011) have played a role in making positive change in the child care benchmarks over time; we need to expand this reporting and change to a worldwide focus. There is clearly the need to expand from the present analysis of 20 countries and the USA to other countries throughout the world and to track changes over time as Child Care Aware/NACCRRA has done.

Another area of concern within the USA and I am sure in other countries as economies have begun their slow recovery from the economic downturn of 2008 – 2010 is to do more with less. One such approach being explored in the USA is called differential monitoring which helps to re-allocate limited resources in a more cost effective and efficient manner via a risk assessment and key indicator approach. I hope that this comparison utilizing the Child Care Aware-NACCRRA Benchmarking Scoring Protocol and introducing the Early Childhood Program Quality Indicator Model/Differential Monitoring Logic Model and Algorithm (Fiene, 2012) within an international context as first steps in making that happen.