Program & Service System Evaluation

Project Theme(s): Vulnerable Populations and Communities, Trauma, Systems Change, Community Engaged Research & Evaluation, Training, Consultation & Capacity Building, Substance Use and Mental Health

Area Description: Evaluations conducted in collaboration with faculty and staff combine scientific rigor with the practical realities of implementing feasible evaluations that are ultimately responsive to local needs. As a result, evaluations are consistently useful; they inform practice, program planning and management, and policy development. Center staff and faculty provide trainings and technical assistance to community-based organizations with the goal of enhancing the evaluation capacity of these organizations. In addition to designing evaluations that yield data that is useful in understanding the processes and outcomes of a given project or organization, we strive to develop evaluation infrastructures that are sustainable beyond our tenure so that programs and organizations can continue to utilize data to inform their program and policy planning.

Our team takes a collaborative approach to evaluation where we join with key stakeholders (e.g. funders, policy makers, program staff, consumers) in the development of data collection variables, methods, and outcomes. First, we engage stakeholders to create measureable language to articulate the goals, objectives, indicators, and outcomes that relate to the overall vision of their work. Second, we create structures to collect and report key process and outcome data to inform progress toward their outcomes and goals. Third, data is analyzed and fed back to stakeholders via formal presentations and or quarterly reports and newsletters. Our presentations and reports are made available to multiple stakeholders such as community members, program staff, and policy makers.

The creation of structures for ongoing data collection and reporting provides an opportunity for continuous quality improvement. Our goal in any endeavor is to build the capacity of community-based organizations and their funders to collect and use data to inform program and policy decision making. Our evaluation results have been used to inform funders about structures for program replication, provide process and outcome data to secure additional funding for sustainability, and offered lessons learned used to create policies and procedures for programming.

Dr. Kaufman’s current projects include: Evaluation of the Connecticut Network of Care (CONNECT) Transformation Implementation (SAMHSA); Evaluation of the Domestic Violence Homicide Reduction Demonstration Initiative (National Institute of Justice); Evaluation of Public Health Programs (CT Department of Public Health); Evaluation Consultation to the CMHC Community Services Network; Evaluation of the Village Collaborative Trauma Center (SAMHSA); Evaluation Consultation to Tides Family Services (RI Foundation).

Dr. Crusto’s current projects include: Evaluation of the RI Children’s Mental Health System of Care Expansion Implementation Cooperative Agreement (SAMHSA); Evaluation of the Connecticut Collaboration on Effective Practices for Trauma (CONCEPT) (Administration for Children and Families, ACF); Evaluation Training and Consultation Services for Behavioral Health Programs (Thomas P. Scattergood Foundation, Philadelphia, PA).

Dr. Crusto evaluates the mediating and moderating role of potentially traumatic experiences (psychological trauma such as violence in the home, school, community) and other stress processes (parenting stress, substance use) on preventive and treatment intervention outcomes.

African Americans have the highest incidence of hypertension of any racial or ethnic group in the United States. Studies show that some medications are ineffective in reducing blood pressure in this population, and we are convinced that other underlying mechanisms are at play. Dr. Crusto and co-principal investigator, Jacquelyn Taylor, PhD, PNP-BC, RN, FAAN, associate professor of nursing at Yale University, are examining genetic markers and psychological factors, such as perceived feelings of racism, mental health, and parenting behaviors. This study investigates the possible effect these variables have on increases in blood pressure among African American women and their children over time.

Recently Dr. Crusto and Dr. Taylor received a five-year $3.4 million grant from the National Institute of Nursing Research, a department of the National Institutes of Health, to conduct a large-scale study on the influence of genetic and psychological factors on high blood pressure in African American women and children. Study partners include several early care and education centers in Connecticut to enroll African American children ages three through five years and their biological mothers in the study.

Community-based participatory research (CBPR) has been promoted as a best practice for engaging marginalized and silenced communities in research that meaningfully addresses issues in their lives. Authentic CBPR requires community collaboration at all levels of the research process, from designing research questions, choosing research methods, data collection, analysis, to the dissemination of research findings. CBPR has proven a powerful methodology to examine the ecological nature of health disparities; to understand barriers to accessing health services that may not be visible to academic researchers; and to impact systems level changes to reduce health disparities. At its best CBPR yields rich findings that can inform relevant and culturally sensitive interventions, empower oppressed communities and illustrate complex and multi-layered factors that contribute to health disparities. Dr. Kaufman and her team have developed curricula and trained adult consumers of behavioral health services, parents of children receiving behavioral health services, and youth who have received behavioral health services to conduct all aspects of qualitative research (e.g., focus groups; key informant interviews). The work that these teams have engaged in have resulted in program and policy changes to how behavioral health services are delivered at the local and state levels.

Dr. Kaufman and her team evaluate the implementation of evidenced-based practices within community-based networks of care and the impact of system functioning on service recipient outcomes. In that work we examine the contextual factors, such as exposure to violence, substance use or familial stress that impact individual, family and systems outcomes for populations at risk.