Open Access

Supporting family child care and quality improvement: findings from an exploratory survey of Illinois child care resource and referral agency staff

International Journal of Child Care and Education Policy201610:4

DOI: 10.1186/s40723-016-0020-8

Received: 29 January 2016

Accepted: 7 September 2016

Published: 17 September 2016

Abstract

Improved quality in home-based child care (family child care and family, friend, and neighbor care) is increasingly recognized as a vital component of early care and education service systems in the U.S. and abroad and is a target of recent federal and state policy initiatives in the U.S. This article presents data from a statewide survey of 73 child care resource and referral specialists across Illinois who work with family child care providers on a regular basis through home visiting, training, and technical assistance. Descriptive findings suggest that specialists who work with family child care providers perform a unique role in the early care and education field. The study examines job roles, common challenges and rewards, and needs for training. Specialists’ training needs include understanding the unique context of family child care, home visiting, coaching, and working with families. Understanding the training and professional development needs of support staff as well as the challenges faced in carrying out this work has the potential to inform state professional development systems as well as other initiatives aimed at improving quality in this sector of the early care and education workforce.

Keywords

Family child care Early care and education workforce Professional development Quality improvement systems Policy research

Background

Home-based child care is a widely used form of non-parental early care and education both within the U.S. and abroad. The National Study of Early Care and Education estimates over one million paid home-based child care providers or family child care (FCC) providers and close to three million unpaid home-based caregivers or family, friend, and neighbor (FFN) caregivers care for children ages zero to five in the U.S. (NSECE 2015). Home-based child care is also an essential component of early childhood systems abroad as documented in a recent review of ten countries’ family child care systems (Davis et al. 2012). For example, in Canada a third of children under age four in early care and education arrangements are cared for in FCC homes (Sinha 2014) and nearly a fifth of children in Australia are cared for in home-based child care (Blaxland et al. 2016).

Improving quality in home-based child care has become a target of recent federal and state policy initiatives in the U.S. (Bromer et al. 2013a; Porter et al. 2010) and FCC is increasingly recognized as a vital component of early care and education systems both in the U.S. and other countries. In the U.S., Head Start and Early Head Start now include FCC as an official option for the delivery of services to children and families, and several states include FCC in their universal pre-K initiatives and quality rating and improvement systems (QRIS) designed to help parents choose high-quality child care and to incentivize providers to enhance quality of care for children (Morrissey 2007; Tout et al. 2010; Zellman and Perlman 2008). As a result of these trends, a variety of child care and social service agencies in communities across the country have developed programs to support home-based child care providers—both FCC and FFN caregivers, including child care resource and referral (CCR&R) agencies and networks that offer support services, technical assistance, materials and equipment, and training to providers (Hershfield et al. 2005). Countries such as Australia and Canada also have well-developed early care and education systems that include agencies that support home-based child care providers (Blaxland et al. 2016; Friendly et al. 2015).

Yet states and local initiatives struggle with how to engage FCC providers in these systems and quality improvement initiatives report low rates of FCC participation (Elicker et al. 2011). Support staff and specialists who are equipped with the skills and knowledge to support FCC providers in quality improvements are critical components of systems and programs that seek to engage and sustain provider participation. Understanding the training and professional development needs of support staff as well as the challenges faced in carrying out this work has the potential to inform state efforts to develop training and technical assistance or coaching credentials as part of their professional development systems as well as other initiatives to develop this sector of the early care and education workforce. The current study of agency staff within one state’s early care and education professional development system, offers a preliminary snapshot of agency staff characteristics and job experiences, and is intended to inform future professional development efforts geared towards agency staff as well as future research on the relationship between agency staff, high-quality support, and positive outcomes for providers, children, and families in FCC settings.

Quality in family child care

The focus on quality improvement in FCC is based on research that finds low levels of quality caregiving in home-based child care. In their three-city landmark study of mothers’ child care arrangements and home-based child care providers (FCC and FFN) in the U.S., Kontos et al. (1995) observed that only 10 % of providers offered what would be considered “good” or “high” quality care. The comprehensive and longitudinal National Institute of Child Health and Human Development (NICHD) Study of Early Child Care (2000) examined the relationship between child care arrangements and child development, and found fewer positive child development outcomes in child care homes compared to child care centers. Several studies have also found that low-income children are more likely to receive lower quality care in FCC homes than higher-income children (Helburn et al. 2002; Kontos et al. 1995; Marshall et al. 2003; Morrissey, 2007). In a study of 120 FCC providers across four mid-western states in the U.S., Raikes et al. (2005) found that FCC providers caring for a higher density of families using government-subsidized child care offered lower quality care than programs with a lower subsidy density. In an in-depth five-state observational sub-study of the National Study of Child Care for Low-Income Families in the U.S., Layzer and Goodson (2006) reported that a majority of FCC providers did not regularly read to children and that television watching was a daily activity in most provider homes. These findings point to the need for more research on how to improve quality of FCC for low-income children and families.

Support and quality improvement in family child care

A small body of research identifies professional support, among other variables, as a significant predictor of quality in FCC (Kontos et al. 1995; Porter et al. 2010). In particular, studies in the U.S. and Canada have shown that FCC providers who network with other providers, access community resources such as lending libraries and belong to professional support groups, tend to offer higher quality child care (Doherty et al. 2006; Kontos et al. 1995). More recently, some studies have started to examine the specific approaches to support initiatives including the presence of qualified agency staff that is most likely to improve quality outcomes. In a small randomized control study in one community, McCabe and Cochran (2008) found that the combination of frequent home visits with FCC and FFN providers (twice a month) by a trained home visitor and regular networking meetings for providers had a significant effect on higher quality scores for participating providers compared to a control group. In a multi-state study of an early childhood coaching initiative with both center-based child care and FCC, Bryant et al. (2009) found that a consulting model of support for FCC providers involving one-on-one visits to provider homes by a trained consultant had a positive effect on quality. In a study of staffed FCC networks in Chicago, Bromer et al. (2009) found that providers who were affiliated with a network that offered a combination of home visits, training, and support delivered by a network coordinator with specialized training offered higher quality care than unaffiliated providers.

Characteristics and experiences of support specialists

Despite the potentially important role that agencies play in supporting quality in home-based child care, only a handful of studies set out to study the characteristics and experiences of coaches, consultants, or network coordinators who deliver services and resources and implement interventions with FCC and FFN providers (Porter et al. 2010). In particular, little is known about the career paths, experiences, and training needs of agency specialists who work with FCC providers. Understanding their training and professional preparation, their prior job experiences, experiences working with providers, and needs for training may inform efforts to integrate these job roles into state-level professional development systems. Below is a summary of findings from research studies that describe the roles of early childhood agency staff and consultants who do not work directly with children and families.

Education and prior work experience

Three studies that examine characteristics of agency staff or consultants who do not directly work with children found similar patterns of educational backgrounds and prior work experience. In their descriptive study of “infrastructure” staff who work in child care agencies across California but do not deliver direct services to children, Whitebook et al. (2012) found that staff on average had high levels of education (BA or higher) but fewer held degrees in early childhood education. Half of staff in these roles reported prior experience working directly with children in centers or child care homes while fewer reported prior job experiences working in other social service sectors. Similarly, in a small descriptive study of 18 coaches within one state’s QRIS working with centers and child care homes, Ackerman (2008) reported high levels of education and relevant experience in early childhood settings among coaches. In their literature review of coaching in early childhood, Isner et al. (2011) found higher education levels among QRIS coaches than among the early childhood teachers they served. While the studies described above examined agency staff and coaches working in a variety of early childhood settings, none of these studies examined differences in education and work experiences for agency staff who work with FCC providers compared to those who work with center-based teachers.

Experiences and challenges providing support

A handful of studies focus on agency staff experiences yet few studies report on staff experiences with home-based child care providers. Whitebook et al. (2012) found that a majority of staff reported working directly with child care providers (centers or homes) or other early childhood personnel as well as fulfilling administrative duties. Ackerman (2008) identified three aspects of working with FCC providers that coaches rated as their greatest challenges including: (1) working with providers who have personal issues; (2) observing inappropriate practices with children; and (3) working with providers who are not motivated to change caregiving practices. In a state-specific study of supports to FFN caregivers, agency directors reported that working with FFN caregivers was time intensive and that agencies often lacked adequate staff or funding to carry out the work (Drake et al. 2006).

Training needs of agency specialists

Few studies have examined the types of trainings or professional development that would be most useful for agency staffs who work with FCC providers. Yet, the expansion of quality improvement systems and programs that include FCC providers has resulted in jobs that require skilled and trained professionals who know how to engage FCC providers in quality improvement initiatives. Research on agency staff and coaches who work with both FCC and center teachers suggests that this sector of the early childhood workforce has unique needs for training that addresses both child and adult development. Whitebook et al. (2012) reported that close to half of infrastructure staff would like more training in child development, especially those staffs that do not hold an early childhood education degree. Nearly a third of staff reported wanting additional training in supervision and management. Drake et al. (2006) found that agency staff would like additional knowledge on the specific needs of FFN caregivers as well as models of effective support and professional development for home-based child care providers. Ackerman (2008) found that QRIS coaches who only have a background in early childhood education and experience may lack the necessary skills and knowledge required to perform the work of quality improvement coaching. Together these findings suggest the need for further examination of the types of training and professional development that would most benefit agency specialists in their support and coaching roles with FCC providers.

Agency staff and quality

Most studies of interventions to improve FCC quality do not examine the relationship of agency staff qualifications or training to quality outcomes in FCC. One exception was a study of staffed networks for licensed FCC providers in Chicago that examined the training characteristics of network agency coordinators and found network coordinators who had specialized training in infant studies and family child care played a key role in contributing to effective, quality-enhancing network services (Bromer et al. 2009).

Provider experiences with training and support

The small research based on FCC provider perspectives on support and training suggests that one component of effective support interventions with FCC is the match between support content and provider interests and needs. Researchers find that FCC and FFN providers may be most likely to participate in ongoing professional development offerings that are specifically designed for home-based child care and are relevant to the specific needs of participating providers. Format and type of support or training may also vary depending on provider type, access to transportation, and a variety of other factors. In a descriptive study of FCC provider needs for training, Rusby (2002) found that FCC providers reported wanting training in behavior management, curriculum, stress management, learning environments, and business management. Providers also preferred weekend and evening workshops over in-home consultation visits. In a descriptive study of FFN caregivers in one state, Liu and Anderson (2010), however, found that a majority of FFN providers would like a home visit from an agency expert.

Isolation may also drive the types of training and support FCC providers participate in as well as their training interests. Prior research on FCC providers finds that many providers experience isolation from other colleagues, early childhood resources, and supports (Bromer and Pick 2012; Drake et al. 2006; Lanigan 2011; Porter et al. 2010). As a result, it is not surprising that providers across studies report interest in opportunities for social and peer support and networking with other child care providers (Lanigan 2011; Liu and Anderson 2010; Rusby 2002; Shivers 2008). In a qualitative examination of FCC provider participation in one state’s professional development system over a three-year time period, providers reported that ongoing cohort-based training was beneficial and that the opportunity to develop a trusting supportive relationship with a facilitator and other providers helped to reduce their experiences of isolation and marginalization (Lanigan 2011). In another state-specific study, Drake et al. (2006) found that more isolated providers (i.e., those living in rural areas of the state) needed more support, and providers living in poverty needed more material resources for children in care.

Prior research has also identified unique aspects of FCC settings that are predictive of quality caregiving and may point to areas of support and training that could benefit these caregivers. Forry et al. (2013), for example, found that provider stress and job demands were predictive of lower quality care in FCC homes. Kryzer et al. (2007) found less sensitive caregiving for toddlers in FCC homes with mixed-age groups of children, a common feature of FCC homes, as compared to single age groupings in center-based child care settings. Both of these studies point to the needs for focused training on how to help providers reduce stress, manage the job demands of running a home-based child care business, as well as strategies for working with mixed-age groups of children.

Research questions

Understanding the preparation, experiences, and training needs of coaches, consultants, and others whose primary job role is to support FCC quality improvement may help programs implement support services and encourage and maintain provider participation in quality improvement. The current study seeks to build on the small research base that exists on nonteaching roles in the early care and education workforce. The study’s particular focus on the roles and experiences of agency staff who work with FCC providers within one state-level quality improvement system—namely, the CCR&R agency system—aims to offer new information that could inform future research on the relationship of agency staff characteristics to provider engagement and quality. Three research questions guided the study:
  • What is the educational background and prior work experience of agency specialists who work with FCC providers?

  • What are the challenges and rewards agency specialists experience in their work with FCC providers?

  • What are specialists’ needs for training specific to working with FCC providers and how does this vary across specialists’ education and experience?

Methods

This survey-based study was informed by prior qualitative evaluation research focused on agency specialists’ experiences with training and reflective practice (Bromer and Korfmacher 2012; Bromer et al. 2013b) as well as research examining the association between types of support and quality in FCC homes (Bromer et al. 2009).

Sample

While the exact number of CCR&R staff members working with home-based child care providers in Illinois is not known, it is estimated that there were approximately 90–100 individuals employed in these roles at the start of the study. The 16 CCR&R agencies in Illinois employ approximately 900 staff statewide. A total of 81 individuals responded to the survey, however, six surveys were incomplete, one participant was working in an administrative role unrelated to working with FCC providers, and one participant reported very little interaction with FCC providers in her job. A final sample of 73 participants (90 %) was used for analyses. Throughout the remainder of this report, these 73 respondents will be referred to as “specialists.”

Measures

A web-based survey was designed to understand the broader landscape of work conducted by CCR&R agency specialists with child care providers across the state and to inform future training and technical assistance interventions with FCC providers. The survey was based on prior qualitative protocols and data analyses from an evaluation of a training program for FCC agency specialists (Bromer and Korfmacher 2012; Bromer et al. 2013b). Survey questions focused on staff preparation and career trajectories including training, education, and prior work experiences; types and frequency of services delivered to FCC providers; experiences, rewards, and challenges working with FCC and FFN providers; staff supervision and support; and staff perceptions of training needs (see “Appendix” section Table 9). Fixed-response answers about types of activities performed during visits to provider homes, for example, were based on qualitative data showing that specialists engaged in a variety of approaches to quality improvement such as modeling, observing, and facilitating provider-child interactions. In addition to fixed responses, respondents were also offered the opportunity to provide additional information in an open-ended response format after each section of the survey.

Data collection

CCR&R agencies in Illinois contract with the state to offer training and technical assistance to early care and education programs and providers. According to Child Care Aware (n.d.), CCR&R agencies provide a bridge between parents, providers, community leaders, and policymakers by offering direct services and planning expertise to help families and communities ensure that their children arrive at kindergarten ready to succeed. For the purpose of this study, we focused on CCR&R specialists who work directly with FCC providers. After the distribution of study information and materials to all of the CCR&Rs in Illinois, surveys were distributed to CCR&R directors who were asked to share the survey link with their staff who work directly with FCC providers.

This project was reviewed and approved by the research institution’s Institutional Review Board.

Data analysis

For all survey questions, we examined frequencies across participant responses as well as within subgroups of participants that were of particular interest such as those with and without relevant education or those with and without direct work experience with children. We also conducted preliminary inferential statistics to examine relationships between subgroups of participants and key areas of interest such as training needs. Comment sections where respondents could write in additional information were analyzed for themes that supported the corresponding quantitative responses. Given the limited breadth and quantity of open-ended comments, we did not conduct systematic coding but rather used open-ended responses to elaborate on survey responses. Both authors read through all of the open-ended comments and reached consensus on relevant themes.

Results

Educational background, preparation, and prior job experience

Specialists had high levels of education with over half (58 %) holding a masters degree or having completed some graduate school. More than half of the specialists in this study (54 %) did not hold a degree in early childhood education (ECE), compared with just under half (46 %) who had completed their highest degree in ECE (see Table 1). Specialists were more likely to hold degrees in related fields such as social work, psychology, elementary education, or nursing.
Table 1

Demographic characteristics of specialists (N = 73)

 

% (n)

Highest level of education2

 Master’s degree

41 (29)

 Some graduate school

17 (12)

 Bachelor’s degree

31 (22)

 Associate’s degree

7 (5)

 Some college

3 (2)

 High school diploma/GED

1 (1)

Area of study3

 Early childhood education (ECE)

46 (32)

 Not related to ECE

54 (38)

 Graduate-level coursework in child development1

61 (44)

Race/ethnicity1

 White/caucasian

60 (43)

 Black/African-American

21 (15)

 Latina/hispanic

17 (12)

 Not specified

3 (2)

1,2,3 Indicates number of missing responses

Most specialists (60 %) reported working with FCC providers from 1 to 5 years and a third (33 %) reported working with providers for 6 years or more. Prior work experience was assessed based on specialists’ reports of their two most recent jobs previous to their current work with providers and whether or not they had ever worked as a FCC provider. Specialists who themselves had worked as FCC providers or center-based teachers were considered as having worked with children. Specialists who had worked as a center director, in an administrative, consulting, or home visiting role were considered as having worked with adults. A majority of specialists (71 %) had prior experience working directly with children. Just over half of participants (52 %) had work experience with both children and adults, while 29 % had worked with adults only and 16 % had worked with children only. Just over one fifth of specialists (22 %) reported they had ever been a FCC provider.

Experiences working with home-based child care providers

Nearly half of the specialists in this study (47 %) worked exclusively with FCC providers in the role of “family child care specialist.” However, other job roles also required staff to come in contact with FCC providers including food and nutrition specialists, mental health and nurse consultants, and QRIS specialists or coaches (see Table 2). Regardless of job role, over 95 % reported that working with FCC providers was different from working with center-based teachers and programs.
Table 2

Job roles of specialists (N = 73)

Current job title

% (n)

FCC specialist

47 (34)

Food and nutrition

15 (11)

Mental health or nurse consultants

10 (7)

QRIS specialist

7 (5)

Child care resource specialists/facilitators

7 (5)

Infant/toddler specialists

6 (4)

Training coordinators

4 (3)

Recruitment and retention specialists

3 (2)

Other

3 (2)

The multi-faceted job roles and responsibilities of agency specialists who work with FCC providers included home visits, training, as well as telephone assistance focused on QRIS participation, Child and Adult Care Food Program participation, and other quality improvement initiatives. All but six of the 73 specialists who responded to the survey reported that they conduct visits to both licensed FCC and license-exempt1 FFN provider homes as part of their work as well as communicate with providers via email, telephone, and training or workshops. Given the potential relationship between home visiting and quality improvement in home-based child care (Bromer et al. 2009; McCabe and Cochran 2008), we asked specialists to report the frequency of visits and the types of activities they conduct during visits to homes. Results suggest that specialists did not work within specific programmatic guidelines or models given the wide variation of home visiting approaches and activities reported.

Table 3 shows the many different approaches to conducting visits with FCC homes reported by specialists across agencies and roles. Many specialists (61 %) reported that the delivery of resources and materials for children and providers was a regular part of what they do in a home visit. Just over half of specialists (53 %) reported that they modeled activities for the provider. Fewer specialists reported talking often with the provider about individual children (44 %), or the provider’s own professional development (49 %). Only one third reported talking often with providers about families (34 %) or about working with mixed-age groups of children (29 %).
Table 3

Types of activities conducted during visits to provider homes (N = 67)

Types of activities

Often  % (n)

Sometimes  % (n)

Rarely  % (n)

Never  % (n)

Bring resources and materials for the provider

80 (53)

18 (12)

2 (1)

Bring materials and/or books for children in care

61 (40)

27 (18))

9 (6)

3 (2)

Model child-focused activities for the provider

53 (35)

26 (17)

14 (9)

8 (5)

Talk to the provider about her own professional development

49 (33)

37 (25)

10 (7)

3 (2)

Talk to the provider about a problem she is having with a specific child

44 (29)

38 (25)

15 (10)

3 (2)

Help provider with QRIS

35 (23)

30 (20)

21 (14)

14 (9)

Talk to the provider about a problem she is having with a specific parent or family

34 (23)

46 (31)

15 (10)

5 (3)

Help the provider work with mixed-age groups of children including school-age children

29 (19)

44 (29)

20 (13)

8 (5)

Frequency of visits to provider homes varied across specialists and across roles, and content of visits also varied depending on individual provider needs as the following specialists elaborated in their open-ended responses to the survey question about frequency of visits:

“I work with each individual as they need it which could be a one-time visit to something that is ongoing for months.”

“It depends on the needs of a particular case. Also it depends if the provider calls me back after a period of time. I have providers with whom I have ongoing relationships up to 3 years. It’s based on the provider’s need. Sometimes it’s a few phone calls, sometimes it’s a couple of visits, but often it’s an ongoing relationship that may go for over a year.”

“It depends on their needs at the time. If they want me to do visits in their home I am happy to do so, but often that takes a while, especially for home providers. So it may take meeting them several times at various meetings and trainings. I really leave it more to the discretion and comfort of the provider I’m working with. If we’re working on a particular topic (such as QRIS), then visits may occur several times in a month, but in general it is less frequent than that.”

In addition to conducting home visits, nearly all specialists (95 %) reported offering trainings or workshops for providers. As Table 4 shows, most specialists reported either focusing or mentioning the following topics in training sessions with FCC providers: child care environments (87 %), infant/toddler care (87 %), working with families (92 %), and working with special needs children (86 %). Although close to half of specialists reported mentioning mixed-age groups of children (50 %) or school-age child care (47 %) in trainings with providers, fewer reported focusing trainings on these topics that are unique to FCC providers. Even fewer specialists reported focusing trainings or mentioning topics specific to FCC such as tax preparation (25 %) or business practices (43 %). As one respondent noted in the comment section of the survey question about training topics: “We would provide more trainings on taxes but it’s difficult to find qualified tax preparers who specialize in preparing taxes for family day care home providers.”
Table 4

Training topics covered by agency specialists (N = 69)

Training topics

Held a training that focused on this topic  % (n)

Held a training that covered but didn’t focus on topic  % (n)

Topic not covered in any trainings  % (n)

Child care environments

51 (35)

36 (25)

13 (9)

Caring for infants and toddlers5

48 (31)

39 (25)

13 (8)

Working with parents and families5

47 (30)

45 (29)

8 (5)

Working with special needs children5

38 (24)

48 (31)

14 (9)

Literacy5

38 (24)

41 (26)

22 (14)

Health and safety6

30 (19)

48 (30)

22 (14)

Math and science7

29 (18)

37 (23)

34 (21)

Working with mixed-age groups5

27 (17)

50 (32)

23 (15)

Caring for preschool age children7

27 (17)

52 (32)

21 (13)

Balancing your own family with child care business4

23 (15)

42 (27)

35 (923)

Business practices6

21 (13)

22 (14)

57 (36)

Accessing community resources5

14 (9)

63 (40)

23 (15)

Caring for school-age children9

12 (7)

47 (28)

42 (25)

Taxes6

11 (7)

14 (9)

75 (47)

Licensing6

11 (7)

37 (23)

52 (33)

5,6,7,9 indicates number of missing responses

Rewards and relationships

All specialists reported having positive and trusting relationships with providers based on their responses to a series of fixed-answer questions about the types of relationships they have with providers. A majority of specialists (87 %) reported that they think a few to all of their providers count on them for support like a friend or family member. Just over half also reported having time in their job to develop close relationships with providers (55 %) although just under half (45 %) reported not having adequate time to forge close relationships with all of the providers in their caseloads. Specialists offered several comments about their relationships with providers. One specialist emphasized the importance of balancing a professional approach while also being supportive of a provider’s personal needs:

“I try to maintain a professional relationship with providers. I like to think that it’s a warm and open relationship where we can talk and share information but not one that crosses into familial styles of relating to one another. I also like to avoid being the ‘expert’ of the relationship, but like to have a more balanced approach to working with providers.”

Most specialists cited numerous rewards of working with FCC providers based on responses to fixed-answer questions. Nearly, all specialists reported the rewards of working with providers included having a chance to make a difference in a child’s life (97 %), helping providers develop as professionals (92 %), and improving quality (92 %). As one specialist noted in the open-response part of the survey, “the ability to interact and to be a trusted resource for providers is one of the best things about my job.” A majority of specialists reported that the opportunity to develop meaningful relationships with providers (86 %), and to work with mixed-age groups of children (70 %), was also a reward of the work.

Challenges

Specialists were asked to rate their job challenges in five areas, including challenges with providers’ care of children, challenges with providers’ personal circumstances, organization and time management, and challenges related to lack of supervisor understanding Table 5 shows that specialists reported that issues related to providers’ care of children were the most challenging, with nearly three quarters (72 %) reporting feeling challenged by providers who are resistant to learning new ways of caring for and educating children, and two thirds (62 %) reporting that witnessing inappropriate practices with children in provider homes was somewhat or extremely challenging. Over one-third of specialists reported that various aspects of providers’ personal circumstances were challenging including provider cancellations and scheduling conflicts (47 %), provider motivations (44 %), and provider mental health issues (34 %). However, despite these challenges related to providers’ personal circumstances, only a few specialists (16 %) reported that establishing boundaries with providers was a challenge. Finally, over a third of specialists reported working in dangerous or uncomfortable neighborhoods (47 %) and homes (37 %) as well as time management and organization of home visits (37 %) as challenges of the job.
Table 5

Job challenges of specialists (N = 73)

The following is somewhat or extremely challenging

% (n)

Challenges related to providers’ care of children

 Providers who are defensive about learning new information or ways of helping children1

72 (52)

 Inappropriate practices w/children in provider homes

62 (45)

Challenges related to providers’ personal circumstances

 Provider cancellations and scheduling conflicts

47 (34)

 Providers who are only motivated by gifts/monetary gain

44 (32)

 Mental health or trauma issues of providers

34 (25)

 Establishing personal/professional boundaries w/providers

16 (12)

Dangerous homes and neighborhoods

 Dangerous or uncomfortable neighborhoods

47 (34)

 Dangerous or uncomfortable homes1

37 (27)

Your own organization and time management

37 (27)

Getting supervisors to understand the realities of my work with home-based child care providers

21 (15)

1Indicates number of missing responses

In their open-ended comments, specialists noted additional challenges related to how providers view their caregiving work. One specialist noted that it was challenging to get providers “to transit their thinking from being a babysitter to a professional.” Another specialist reported: “Sometimes they are stuck in their old ways and it is difficult for them to change with each individual child or family situation.” Specialists also mentioned the difficulty of building trust with FCC providers given the private nature of home-based child care settings.

Training needs of agency specialists
Training needs were measured on a two-point scale ranging from 0–1 (see Table 6). (0 = I have received enough training/I see no need for training in this area; 1 = I have not received enough training/I would like training in this area). Nine areas of training were measured, including child development (three items), school-age development (one item), developing a business (one item), organization and case management (one item), home visits/coaching & consultation (two items), listening and communication skills (two items), working with families/family systems (two items), understanding family child care (one item), and adult learning theory (one item). Specialists reported the highest demand for training in the areas of developing a business, working with families, school-age development, understanding family child care, and home visiting and coaching or consultation. Specialists reported the least demand for training in listening/communication skills, adult learning theory, early childhood development, and organization and case management.
Table 6

Training needs of specialists

 

Mean

Std. deviation

Developing a business

.67

.47

Working with families and family systems

.61

.44

School-age development

.59

.50

Understanding family child care

.59

.50

Home visiting and coaching and consultation

.58

.42

Organization and case management

.48

.50

Child development

.42

.44

Adult learning theory

.38

.49

Listening and communication skills

.34

.46

Relationship between education and training needs
We examined whether training demands were related to the level of education attained by the specialists or the area in which they received their degree (see Table 7). Those with a BA degree or less in any field (low education) reported a statistically significant greater need for training in home visits/coaching and consultation (p ≤ .05) and working with families/family systems (p ≤ .001) than those with a BA degree or higher (high education). Specialists with less than a BA degree were also more likely to report wanting training in case management, listening and communication skills, and adult learning theory (p ≤ .10). Specialists without graduate-level coursework reported a statistically significant greater need for training in child development (p ≤ .05) and working with families and family systems (p ≤ .001) than specialists who did not report having graduate-level coursework. While only marginally significant, specialists with graduate-level coursework were also more likely to report wanting training in home visiting and coaching and consultation, listening and communication skills, and adult learning theory (p ≤ .10).
Table 7

Training needs and level of education

 

High education: BA or higher

Low education: less than BA

p

Grad course work (n = 44)

No grad course work (n = 28)

p

ECE (n = 32)

No ECE (n = 38)

p

High education; ECE (n = 22)

High education; Not ECE (n = 19)

Low education; ECE (n = 9)

Low education; Not ECE (n = 19)

p

Areas of training

Child development

0.36

0.50

 

0.34

0.56

*

0.28

0.53

*

0.15

0.60

0.52

0.46

**

School-age development

0.54

0.67

 

0.52

0.71

 

0.47

0.68

+

0.36

0.74

0.67

0.63

+

Developing a business

0.66

0.70

 

0.64

0.75

 

0.63

0.71

 

0.59

0.74

0.67

0.68

 

Organization and case management

0.39

0.60

+

0.41

0.57

 

0.38

0.53

 

0.32

0.47

0.56

0.58

 

Home visiting/consultation

0.49

0.72

*

0.51

0.68

+

0.53

0.61

 

0.46

0.53

0.78

0.68

 

Listening and communication skills

0.26

0.45

+

0.26

0.45

+

0.30

0.34

 

0.30

0.21

0.33

0.47

 

Working with families

0.49

0.78

***

0.49

0.79

***

0.55

0.64

 

0.48

0.50

0.78

0.79

*

Understanding family child care

0.59

0.60

 

0.52

0.68

 

0.56

0.61

 

0.55

0.63

0.67

0.58

 

Adult learning theory

0.29

0.50

+

0.30

0.50

+

0.28

0.45

 

0.23

0.37

0.44

0.53

 

Differences between groups: + significant at p ≤ .10; * significant at p ≤ .05; ** significant at p ≤ .01; *** significant at p ≤ .001

Specialists who did not have any formal higher education in early childhood education (ECE) were more likely to report a need for training in child development than those who had some formal education in ECE, and this difference was statistically significant (p ≤ .05). Specialists without formal ECE education were also somewhat more likely than specialists with an ECE degree to report a need for training in school-age development (p ≤ .10).

When education area and level were combined, specialists with high levels of education and no ECE coursework were the most likely to report they needed training in child development (p ≤ .01). Specialists with low levels of education regardless of ECE content were the most likely to report they needed training in working with families and family systems (p ≤ .05). Those with higher education in ECE were the least likely to report they needed training in school-age development (p ≤ .10). Overall, specialists with more education regardless of whether they held ECE degrees, were more likely to report they needed additional training in areas specific to working with FCC providers such as home visiting and consultation,

Relationship between work experience and training needs
Table 8 shows the relationships between prior work experience with children and/or adults and training needs. Specialists who only worked with children in the past were more likely to report a need for training in adult learning theory compared to those who worked previously with either adult only or with both adults and children (p ≤ .05). Specialists who only worked with adults compared to those who worked with only children or with adults and children were less likely to report a need for training in home visits and coaching and consultation (at p ≤ .05).
Table 8

Training needs and work experience

 

Worked with children only (n = 12)

Worked with adults only or children and adults (n = 56)

p

Worked with adults only (n = 20)

Worked with children only or children and adults (n = 48)

p

Areas of training

Child development

.58

.38

 

.33

.44

 

School-age development

.58

.59

 

.70

.54

 

Developing a business

.67

.70

 

.65

.71

 

Organization and case management

.58

.46

 

.35

.54

 

Home visiting/consultation

.71

.55

 

.40

.66

*

Listening and communication skills

.50

.29

 

.28

.34

 

Working w/families

.71

.57

 

.48

.65

 

Understanding family child care

.67

.55

 

.45

.62

 

Adult learning theory

.58

.29

*

.30

.35

 

Differences between groups: + significant at p ≤ .10; * significant at p ≤ .05; ** significant at p ≤ .01; *** significant at p ≤ .001

Discussion

Findings from this descriptive study confirm earlier studies’ reports of agency staff characteristics and experience. Yet, the current study also contributes new knowledge about agency staff experiences working with FCC providers in particular and suggests that CCR&R specialists who work with FCC providers perform a unique role in the early care and education field. Specialists in this study shared similar characteristics (e.g., high levels of education and extensive years of work experience) with management and coaching staff who do not work directly with children examined in other research (Isner et al. 2011; Whitebook et al. 2012). Also, the current study found that few specialists working with FCC providers were ever providers themselves. This new finding suggests the lack of a clearly articulated career path for FCC providers who may seek to move from direct care into consultation and technical assistance work.

Prior research has not focused specifically on the work of agency specialists with FCC providers. This study contributes new information about the variation in approaches and activities conducted during technical assistance visits and trainings with FCC providers. Although nearly all specialists in this study reported conducting home visits with providers, variation in frequency of visits as well as content of visits were reported including monitoring, modeling caregiving practices, delivering materials and equipment, and offering professional development and support to providers.

Current study findings regarding specialists’ challenges and difficulties working with FCC providers (e.g., working with resistant providers and observing inappropriate caregiving practices) are similar to findings from past research (Ackerman 2008). However, current study findings contribute additional information about the particular challenges of FCC specialists who unanimously agreed that offering assistance and support to FCC providers is different from working with center-based teachers and staff, and cited particular challenges that may be unique to working with providers in their homes such as the logistics and safety concerns involved in home visiting with FCC providers.

The current study examined the training topics offered by specialists to providers as well as the training needs of specialists specifically focused on their work with FCC providers. Findings suggest that some of the training topics most commonly covered by specialists in workshops for providers align with prior research findings on provider training needs and interests. Prior research finds that providers want supports that are responsive and relevant to their unique child care contexts such as how to work with mixed-ages (including infants and school-age children) and how to set up home-based child care environments (Rusby 2002). Research on predictors of FCC quality suggests that support around job demands and stress as well as working with infants and toddlers in mixed-age groups could benefit providers and the children in their care (Forry et al. 2013; Kryzer et al. 2007). The current study finds specialists were most likely to report offering trainings on topics generic to all child care providers regardless of setting including child care environments, infant care, and working with families, but less likely to cover topics unique to FCC such as working with mixed-ages, balancing family life with a FCC business, business practices, or working with school-age children.

The current study also offers preliminary new information about the relationship between education, work experience, and needs for training among specialists working with FCC. Specialists in the current study reported needing more training in FCC-specific topics for themselves (school-age care, business practices, and working with families), pointing to the gap in professional development offerings tailored to this sector of the early childhood workforce. Moreover, the current study suggests that many specialists may come into these job roles without prior formal education in child development and/or without prior experience working with adults. Specialists without formal education in early childhood clearly reported needing additional training in child development. Not surprisingly, specialists whose prior job experiences were limited to working with children as a preschool teacher, for example, reported greater needs for training in adult learning styles. These findings are based on a small sample and suggest preliminary patterns that future research should examine in larger samples with more specific questions about the relationship between career path and current job experiences.

Program and policy recommendations

In illuminating the unique features of the work of FCC support specialists, this study has implications for quality improvement initiatives and professional development systems that seek to include FCC and other home-based child care providers. First, models or guidelines for how to work effectively with child care providers in their homes should be established as part of quality improvement and professional development systems. The current study suggests wide variation in approaches and types of services delivered to providers by agencies and a lack of clear models or program implementation. Nearly, all specialists in the study reported conducting home visits, but frequency of home visiting and content of visits varied across specialist reports. While some research points to the promise of coaching and consultation as effective strategies for improving quality of home-based child care (Bryant et al. 2009), experts caution that only highly manualized and consistent coaching interventions may be effective at changing practice (IOM 2012). As Isner et al. (2011) suggest, more research is needed on implementation of coaching programs that could illuminate which aspects of coaching models are most effective in changing provider practices with children and families.

Second, findings regarding specialists’ training needs suggest that state professional development systems should include training focused on topics that are particularly salient for staff working with FCC providers such as home visiting and consultation strategies and models, working with mixed-age groups of children and school-age child care in particular, business practices, and working with families. Moreover, the lack of FCC experience and knowledge that many specialists report bringing to their work with providers points to the need for professional development systems that articulate career pathways for FCC providers to move into consultation and technical assistance roles.

Findings from this study about specialists’ prior work experiences with children and adults as well as their needs for both child development knowledge across the age span (infancy through school-age) as well as how to work with both very young children and adults further suggests that professional development systems need to include a broad array of offerings on child development as well as adult learning styles and strategies. Bromer et al.’s study of FCC networks (2009), for example, found that providers in networks with staff who received specialized training in infant/toddler care and how to work with FCC providers, offered higher quality care than providers in networks who did not have specially trained staff. This finding suggests that the preparation of agency staff in content areas that are relevant to FCC makes a difference in quality improvement efforts.

Overall, findings from this exploratory study have implications for early childhood professional development systems and initiatives aimed at improving quality care across child care sectors both in the U.S. and abroad. Such systems should consider developing guidelines for approaches to professional development (e.g., coaching, workshops, training series) that are most likely to impact quality improvements in child care homes. Systems and policies that seek to rate and improve quality in FCC homes should also consider the distinctive features of home-based child care in developing quality standards and competency benchmarks for providers seeking credentialing as well as for agency staff who work with FCC providers.

Limitations and future research

This article reports on an exploratory and descriptive examination of agency specialists’ characteristics, job challenges and rewards, and needs for training. The reliance on survey data without additional qualitative data limits the depth of information available about how specialists experience their work with providers. Moreover, the focus on one state in the U.S. limits the generalizability of the findings to other state systems as well as other program and policy contexts within and outside of the U.S.

Future research is needed to build on the findings reported here to further examine the roles of agency specialists who support FCC providers across state and policy contexts. Collecting data on agency staff characteristics and experiences in larger multi-state samples could inform state and local administrators about how to best invest resources in professional development for early care and education staff, as well as to more effectively implement new models and programs for improving quality in home-based settings. With recent initiatives such as the Early Head Start Child Care partnerships which aim to include FCC as well as center-based providers, programs will need to focus efforts and resources on how to best engage, support, and maintain participation of these providers. Findings from the current study only skim the surface of what we still need to learn about the diversity of coaches, mentors, home visitors, and other specialists who support and work with home-based child care providers across a variety of program and policy initiatives.

Footnotes
1

In Illinois, providers who care for more than three nonrelative children are required to be licensed by the state to offer child care in their home.

 

Abbreviations

FCC: 

family child care

FFN: 

family, friend or neighbor

CCR&R: 

child care resource and referral agency

QRIS: 

quality rating and improvement system

ECE: 

early care and education

Declarations

Authors’ contributions

JB conceived of the study, designed the survey, recruited the sample, guided the data analysis, and wrote the report. CW assisted with data collection and statistical analyses. Both authors read and approved the final manuscript.

Acknowledgements

The Illinois Network of Child Care Resource and Referral Agencies (INCCRRA) helped facilitate data collection for this study.

Competing interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Herr Research Center for Children and Social Policy, Erikson Institute
(2)
Catholic Charities Fort Worth

References

  1. Ackerman, J. D. (2008). Coaching as part of a pilot quality rating scale initiative: Challenges to—and supports for—the change-making process. Early Childhood Research & Practice, 10 (2). Retrieved from: http://ecrp.uiuc.edu/v10n2/ackerman.html.
  2. Blaxland, M., Adamson, E., & Cortis, N. (2016). Perspectives on quality in Australian family day care (SPRC Report 01/16). Sydney: Social Policy Research Centre, UNSW Australia. https://www.sprc.unsw.edu.au/media/SPRCFile/Perspectives_on_quality_in_Australian_family_day_care__Final_report.pdf.
  3. Bromer, J., & Korfmacher, J. (2012). Evaluation of a relationship-based training pilot for agency specialists working with home based child care providers. Chicago, IL: Herr Research Center for Children and Social Policy, Erikson Institute. Retrieved from http://www.erikson.edu/wp-content/uploads/Erikson-FCCSTP-Phase-1-Final-Report-7-5-12.pdf.
  4. Bromer, J., & Pick, H. (2012). Home-based child care provider perspectives on agency support: A sub-study of an evaluation of a relationship-based training pilot for agency specialists: final report to Illinois action for children. Chicago: Herr Research Center.Google Scholar
  5. Bromer, J., Van Haitsma, M., Daley, K., & Modigliani, K. (2009). Staffed support networks and quality in family child care: The family child care network impact study. Chicago, IL: Herr Research Center for Children and Social Policy, Erikson Institute. Retrieved from: http://www.erikson.edu/wp-content/uploads/Full_report_web.pdf.
  6. Bromer, J., McCabe, L., & Porter, T. (2013a). Special section on understanding and improving quality in family child care: introduction and commentary. Early Childhood Research Quarterly, 28(4), 875–878.View ArticleGoogle Scholar
  7. Bromer, J., Weaver, C., & Korfmacher, J. (2013b). Evaluation of Erikson institute’s family child care specialist training program phase II final report. Chicago, IL: Herr Research Center for Children and Social Policy, Erikson Institute. Retrieved from http://www.erikson.edu/wp-content/uploads/Erikson-FCCSTP-Phase-II-Final-Report-1-8-14.pdf.
  8. Bryant, D. M., Wesley, P. W., Burchinal, M., Sideris, J., Taylor, K., Fenson, C., & Iruka, I. (2009). The QUINCE—PFI study: an evaluation of a promising model for child care provider training. Final report. Chapel Hill, NC: FPG Child Development Institute, University of North Carolina at Chapel Hill. Retrieved from: http://www.researchconnections.org/childcare/resources/18531/pdf.
  9. Child Care Aware (n.d.). Child care resource & referral. Retrieved from http://www.childcareaware.org/about/child-careresource-referral. Accessed 13 Sept 2016.
  10. Davis, et al. (2012). An international perspective on regulated family day care systems. Australian Journal of Early Childhood, 37(4), 127–137.Google Scholar
  11. Doherty, G., Forer, B., Lero, D. S., Goelman, H., & LaGrange, A. (2006). Predictors of quality in family child care. Early Childhood Research Quarterly, 21, 296–312.View ArticleGoogle Scholar
  12. Drake, P. M., Greenspoon, B., Unti, L., Fawcett, L. K., & Neville-Morgan, S. (2006). Family, friend, and neighbor child caregivers: results of a statewide study to determine needs and desires for support. Early Childhood Education Journal, 33(4), 239–244.View ArticleGoogle Scholar
  13. Elicker, J., Langill, C., Ruprecht, K., Lewsader, J., & Anderson, T. (2011). Evaluation of paths to QUALITY, Indiana’s child care quality rating and improvement system: final report (Technical Report#3) Department of Human Development & Family Studies Center for Families, Purdue University. Retrieved from http://www.in.gov/fssa/files/PTQFinalReportRev11012.pdf.
  14. Forry, N. D., Iruka, I. U., Tout, K., Torquati, J. C., Susman-Stillman, A. R., Bryant, D. M., et al. (2013). Predictors of quality and child outcomes in family child care settings. Early Childhood Research Quarterly, 28(4), 893–904.View ArticleGoogle Scholar
  15. Friendly, M., Grady, B., Macdonald, L., and Forer, B. (2015). Early childhood education and care in Canada 2014. Toronto: Childcare Resource and Research Unit. Retrieved from: http://childcarecanada.org/sites/default/files/ECEC-2014-full-document-revised-10-03-16.pdf.
  16. Helburn, S. W., Morris, J. R., & Modigliani, K. (2002). Family child care finances and their effect on quality and incentives. Early Childhood Research Quarterly, 17(4), 512–538.View ArticleGoogle Scholar
  17. Hershfield, B., Moeller, A., Cohen, A. J., & The Mills Consulting Group. (2005). Family child care networks/systems: a model for expanding community resources. New York: Child Welfare League of America.Google Scholar
  18. Institute of Medicine (IOM) and National Research Council (NRC). (2012). The early childhood care and education workforce: challenges and opportunities: a workshop report. Washington, DC: The National Academies Press.Google Scholar
  19. Isner, K. T., Tout, K., Zaslow, M., Soli, M., Quinn, K., Rothenberg, L., & Burkhauser, M. (2011). Coaching in early care and education programs and quality rating and improvement systems (QRIS): identifying promising features. Washington, D.C. Child Trends. Retrieved from: http://www.childtrends.org/wp-content/uploads/2013/05/2011-35CoachingQualityImprovement.pdf.
  20. Kontos, S., Howes, C., Shinn, M., & Galinsky, E. (1995). Quality in family child care and relative care. New York: Columbia University, Teachers College Press.Google Scholar
  21. Kryzer, E. M., Kovan, N., Phillips, D. A., Domagall, L. A., & Gunnar, M. R. (2007). Toddlers’ and preschoolers’ experience in family day care: age differences and behavioral correlates. Early Childhood Research Quarterly, 22(4), 451–466.View ArticleGoogle Scholar
  22. Lanigan, J. (2011). Family child care providers’ perspectives regarding effective professional development and their role in the child care system: a qualitative study. Early Childhood Education Journal, 38(6), 399–409.View ArticleGoogle Scholar
  23. Layzer, J.I., & Goodson, B. (2006). National study of child care for low-income families: Care in the home: A description of family child care and the experiences of the families and children who use it. Wave 1 report. Cambridge, MA: Abt Associates. Retrieved from: http://www.acf.hhs.gov/opre/resource/national-study-of-child-care-for-low-income-families-care-in-the-home-a-2.
  24. Liu, M., & Anderson, S. G. (2010). Understanding caregiving patterns, motivations, and resource needs of subsidized family, friend, and neighbor child care providers. Child Welfare, 89(3), 99–119.Google Scholar
  25. Marshall, N.L., Creps, C.L., Burstein, N.R, Cahill, K. E., Robeson, W. W., Wang, S.Y., Schimmenti, J., & Glantz, F.B. (2003). Family child care today: A report of the findings of the Massachusetts Cost/Quality Study: Family Child Care Homes. Wellesley, MA: Wellesley College Center for Research on Women and Abt Associates, Inc. Retrieved from: http://www.abtassociates.com/reports/FCCH12_31_03.pdf.
  26. McCabe, L. A., & Cochran, M. (2008). Can home visiting increase the quality of home-based child care? Findings from the Caring for Quality Project (Research Brief No. 3). Cornell, NY: Cornell University Early Childhood Program. Retrieved from: http://www.human.cornell.edu/fldc/cecp/resources/loader.cfm?csModule=security/getfile&PageID=48184.
  27. Morrissey, T.W. (2007). Family child care in the United States. Reviews of research literature review. New York: Child care & early education research connections. Retrieved from: http://www.nccp.org/publications/pdf/text_720.pdf.
  28. National Institute of Child Health and Human Development (NICHD) Early Child Care Research Network. (2000). Characteristics and quality of child care for toddlers and preschoolers. Applied Developmental Science, 4(3), 116–135.View ArticleGoogle Scholar
  29. National Survey of Early Care and Education Project Team. (2015). Fact sheet: Who is providing home-based early care and education? OPRE report no. 2015-43, Washington, DC. Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. http://www.acf.hhs.gov/programs/opre/resource/fact-sheet-who-is-providing-home-based-early-care-and-education.
  30. Porter, T., Paulsell, D., Grosso, P.D., Avellar, S., Hass, R., & Vuong, L. (2010). A review of the literature on home-based child care: implications for future directions. Princeton, NJ: Mathematica Policy Research, Inc. Retrieved from: http://www.acf.hhs.gov/sites/default/files/opre/lit_review.pdf.
  31. Raikes, H. A., Raikes, H., & Wilcox, B. (2005). Regulation, subsidy receipt, and provider characteristics: what predicts quality in child care homes? Early Childhood Research Quarterly, 20(2), 164–184.View ArticleGoogle Scholar
  32. Rusby, J. C. (2002). Training needs and challenges of family child care providers. Child & Youth Care Forum, 31(5), 281–293.View ArticleGoogle Scholar
  33. Shivers, E. (2008). Exploring the training needs of family, friend and neighbor child care providers. Phoenix, AZ: Indigo Cultural Center, Institute for Child Development Research and Social Change. Retrieved from: http://indigoculturalcenter.org/wp-content/uploads/reports/Exploring-the-Training-Needs-of-FFN-copy.pdf.
  34. Sinha, M. (2014). Child care in Canada: Analytical paper (Spotlight on Canadians: Results from the general social survey; Catalogue no. 89-652-X-No. 005). Minister of Industry: Canada. Retrieved from: http://www.statcan.gc.ca/pub/89-652-x/89-652-x2014005-eng.pdf.
  35. Tout, K., Starr, R., Soli, M., Moodie, S., Kirby, G., & Boller, K. (2010). Compendium on quality rating systems and evaluations. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation. Retrieved from: http://www.acf.hhs.gov/opre/resource/compendium-of-quality-rating-systems-and-evaluations.
  36. Whitebook, M., Kipnis F., Sakai, L., & Austin, L.J. E. (2012). Early care and education leadership and management roles: Beyond homes and centers. Early childhood research and practice, 14 (1). Retrieved from: http://ecrp.uiuc.edu/v14n1/whitebook.html.
  37. Zellman, G.L., & Perlman, M. (2008). Child-care quality rating and improvement systems in five pioneer states: implementation and lessons learned. Santa Monica, CA: Rand Corporation. Retrieved from: http://www.rand.org/content/dam/rand/pubs/monographs/2008/RAND_MG795.pdf.

Copyright

© The Author(s) 2016