PEER has concluded the initial evaluation of the early STEM education model implemented at Maritime Odyssey Preschool, which was founded in 2016 as a collaboration between The Maritime Aquarium, Odyssey Learning, Inc., and Grace Baptist Church and now serves over 200 children. These partners engaged PEER in 2017 as an evaluation partner who could help them use data to guide their decision-making, understand the program’s effectiveness, and identify areas for growth.
Developing a logic model helped the partners clarify their shared expectation that the early STEM education model could produce a range of positive child outcomes along with increased interest and knowledge in science. The partners also expected that child outcomes would be associated with teachers' use of effective STEM learning activities in classrooms and in the STEM lab, which depend on teacher confidence in STEM.
The group selected measures of teacher attitudes and beliefs about teaching math and science to young children and surveyed teachers three times between the fall of 2018 and the fall of 2019, producing data that informed professional development. The group explored measures of STEM teaching practices but decided it was not feasible to implement STEM-focused classroom observations during the 2019-2020 school year. The group planned to conduct a spring 2020 pilot of Lens on Science/Enfoque on Ciencia,[1] a computer-based adaptive assessment of 3- to 5-year-old children’s science knowledge and science process skills, but that plan was interrupted by the coronavirus pandemic.
The research base[2] and their own experience convinced the partners that Maritime Odyssey Preschool’s early STEM model should increase children’s early language and literacy skills, as well as children’s STEM skills. As a result, the group decided to examine child data from the Preschool Early Literacy Indicator (PELI), an assessment of pre-literacy and oral language skills that was administered in all Norwalk School Readiness programs. Specifically, PEER would examine the association of program participation (quantified in terms of the number of days in attendance) with PELI Language Index and PELI Composite Scores in the spring of the PK-4 year (the final PELI administration before kindergarten).
Analyses focused on two overlapping groups: all children who graduated from Maritime Odyssey Preschool in 2019 (72 children) and the subset of 2019 graduates who attended the program for more than one year (53 children). After combining PELI assessment data with data from enrollment records, PEER examined:
- how children’s PELI scores changed over time,
- which demographic characteristics (age, gender, dual language learner (DLL) status, and free and reduced-price lunch status) predicted growth in PELI scores, and
- the association between program attendance and PELI scores when correcting for demographic characteristics.
PEER found statistical increases over time in PELI scores for both groups of children. Also, older children experienced more growth than younger children, and DLL children experience less growth than non-DLL children. Program participation had a statistical effect on growth in PELI scores for the larger group of children (all 2019 graduates) but not for the smaller group of children (2019 graduates who enrolled during the PK-3 or earlier). Limitations concerning the evaluation’s small sample and the absence of a comparison outside of Maritime Odyssey Preschool makes it impossible to test whether program participation caused increases in PELI scores for the larger group of children.
Future evaluation studies could build on the results of this project in several ways. Introducing a measure of teachers’ STEM content knowledge and STEM teaching skills would allow the partners to examine how teachers’ skills and knowledge change as they participate in ongoing professional development. Piloting the Lens on Science/Enfoque on Ciencias assessment would provide information about children’s growth in knowledge of science content and processes. Alternately, using kindergarten data would allow for examination of children’s success after leaving Maritime Odyssey Preschool. Finally, an evaluation design that allows for comparison of Maritime Odyssey Preschool children to a group of similar children would allow testing of causal claims, especially if sample size were adequate relative to the effect size. PEER hopes the results of this preliminary evaluation will help Odyssey Learning and The Maritime Aquarium continue to refine their model in the years to come.
References
[1] Greenfield, D. B. (2015). Assessment in early childhood science education. In Research in early childhood science education (pp. 353-380). Springer, Dordrecht. https://link.springer.com/chapter/10.1007/978-94-017-9505-0_16.
[2] Brenneman, K., Stevenson-Boyd, J. & Frede, E.C. (2009). Mathematics and Science in Preschool: Policy and Practice. National Institute for Early Education Research (NIEER). http://nieer.org/wp-content/uploads/2016/08/MathSciencePolicyBrief0309.pdf.
Hoisington, C., & Winokur, J. (2019). Let's Talk About It. Science and Children, 57(4), 70-75. https://search.proquest.com/docview/2312780555?pq-origsite=gscholar&fromopenview=true.
Reyes, C. R. (2019). Literacy and Academic Success for English Learners Through Science (LASErS) Evaluation Report. Ziegler Center in Child Development and Social Policy. https://medicine.yale.edu/childstudy/zigler/publications/LASErS_FINAL_REPORT_WEB_VERSION_365841_284_5379_v1.pdf.