RYTE Theoretical Orientation

Evolutionary Evaluation and the Systems Evaluation Protocol were developed by the RYTE Institution in collaboration with the Cornell Office for Research on Evaluation (CORE) with support from a National Science Foundation funded grant (William Trochim, PI, Jennifer Brown Urban, co-PI). A systems evaluation approach considers the complex factors that are inherent in the larger system in which a program is embedded. In addition, our systems evaluation approach focuses on enhancing the internal evaluation capacity of programs and organizations. Our research is focused on assessing whether our approach to systems evaluation is associated with high quality evaluation plans and logic/pathway models and an increase in evaluation capacity for practitioners working in multiple settings.

The Systems Evaluation Protocol (SEP) is a guide to our approach to evaluation planning, implementation, and utilization and is a useful tool for anyone engaged in program evaluation and planning and can be accessed free of charge on the CORE website.

While the Systems Evaluation Protocol can be implemented as a manual process that does not depend on any specific technology platform, it is designed so that it can be enhanced by using the Netway, a Web 2.0 application consistent with second-generation web-based communities and hosted services such as social-networking sites, wikis, and blogs. The Netway is also available and can be accessed at www.evaluationnetway.com.

RYTE Institute is currently applying our approach to Systems Evaluation and evaluation capacity building with character development programs. See the PACE Project for more information.

• Urban, J.B., & Trochin, W.M. (2017). Advancing evaluation of character building programs. Journal of Character Education, 13(2), 13-27.
• Urban, J.B., Burgermaster, M., Archibald, T., & Byrne, A. (2015). Relationships between quantitative measures of evaluation plan and program model quality and a qualitative measure of participant perceptions of an evaluation capacity building approach (PDF). Journal of Mixed Methods Research. 9(20), 154-177. DOI: 10.1177/1558689813516388
• Urban, J.B., Hargraves, M., & Trochim, W.M. (2014). Evolutionary evaluation: Implications for evaluators, researchers, practitioners, funders and the evidence-based program mandate (PDF). Evaluation and Program Planning, 45, 127-139. DOI: 10.1016/j.evalprogplan.2014.03.011
• Trochim, W., Urban, J.B., Hargraves, M., Hebbard, C., Buckley, J., Archibald, T., Johnson, M., & Burgermaster, M. (2012). The guide to the systems evaluation protocol. Ithaca, NY: Cornell Digital Print Services.
• Urban, J.B. & Trochim, W.M. (2017). Advancing evaluation of character building programs. Journal of Character Education, 13(2), 13-27.
• Urban, J.B. & Trochim, W.M. (2009). The role of evaluation in research-practice integration: Working toward the “golden spike” (PDF). American Journal of Evaluation, 30(4), 538-553.

At its core, developmental science aims to explore bi-directional relationships, interactions across contexts, and change over time. Despite major statistical and methodological advances, the field is in need of new methodological tools that integrate existing and new research, at different levels of analysis, account for bi-directional feedback processes and address the complexity inherent in change that occurs throughout the lifespan. Systems science methodologies (including system dynamics, agent based modeling, and network analysis) have been successfully implemented in fields such as physics, engineering, and computer science to address such complexity. More recently, the behavioral and social sciences have begun to harness the potential of these methodological approaches in applications to questions related to public health. These systems science methodologies are particularly well suited for application to developmental science questions, yet they have not yet been harnessed to their full potential.

The application of systems science methods to developmental science questions holds tremendous promise. Systems science approaches integrate multiple levels of analysis—from cells to behavior to society—to understand the ways in which individual, contextual, and organizational factors interact over time. Because of its unique ability to consider simultaneously both the whole system and its individual parts, the application of systems science methodologies in developmental science shows promise for understanding complex, multidimensional issues and for transforming this knowledge into effective interventions that can fundamentally change developmental outcomes.

The challenge at this point is to expose developmental scientists to and educate them about the potential utility of systems science methodologies. This includes finding concrete ways to demonstrate why and how systems science methodologies can help developmental scientists explore systems-based questions about development. The RYTE Institute has engaged in several projects to this end including several publications and presentations. Dr. Urban, together with Dr. Patricia Mabry (Office of Behavioral and Social Sciences Research, National Institutes of Health) edited a special issue of Research in Human Development on Embracing Systems Science: New Methodologies for Developmental Science. Dr. Urban has also worked closely with the Society for the Study of Human Development to promote developmental systems science including organizing a plenary panel and poster session at the 2009 conference and a panel and workshops on systems science methods at the 2011 conference.

The RYTE Institute is currently applying our approach to Systems Evaluation and evaluation capacity building in the following projects:

• The PACE Project
• Cultivating Relational Values in Parents and Children

• Urban, J.B., Linver, M.R., Johnson, S.K., MacDonnell, M., Glina, M., Chauveron, L., & Gama, L. (2018) Developing the next generation of engaged youth: Inspire Aspire – Global Citizens in the Making. Journal of Moral Education. 47(1), 104-125. DOI: 10.1080/03057240.2017.1396967
• Urban, J.B., Linver, M.R., Thompson, J., Davidson, R., & Lorimer, D. (2018). Evaluating youth character education using Evolutionary Evaluation and the Systems Evaluation Protocol. Applied Developmental Science, 22(4), 245-257. DOI: 10.1080/10888691.2017.1285236
• Urban, J.B., Osgood, N., Okamoto, J., Mabry, P., & Lich, K.L. (2014). Developmental systems science: Extending developmental science with systems science methodologies. In P.C.M. Molenaar, R.M. Lerner, & K.M. Newell (Eds.). Handbook of Developmental Systems Theory and Methodology. New York: The Guildford Press.
• Urban, J.B., Osgood, N., & Mabry, P. (2011). Developmental systems science: Exploring the application of non-linear methods to developmental science questions (PDF). Research in Human Development, 8(1), 1-25. doi: 10.1080/15427609.2011.549686
• Urban, J.B. & Mabry, P. (Eds.). (2011). Embracing systems science: New methodologies for developmental science. Research in Human Development, 8(1).

The RYTE Institute is working to build an evidence-base within developmental science that specifically addresses the role of multiple individual (e.g., self-regulation) and contextual factors (e.g., family, school, neighborhood) on adolescent development with a focus on positive youth development.

Studies have examined how individual factors such as self-regulation, contextual factors such as the family, school, neighborhood, and involvement in out-of-school activities affects the developmental trajectories of adolescents including outcomes related to positive youth development.

With funding from the Stephen D. Bechtel, Jr. Foundation, we are working with Boy Scouts of America to conduct research to better understand how scouting helps to build character. We are collaborating with a team at the National BSA office as well as American Institutes for Research (AIR) to develop a model of adult volunteer training and development as well as a model of character development in Boy Scouts youth members. The models will be validated by convening focus groups across the country and will be compared against the research literature. The team is also developing measures of adult and youth character development.

• Linver, M.R., Urban, J.B., MacDonnell, M., Robers, E.D., Quinn, J., Samtani, S., Doubledee, R., Gama, L., & Morgan, D. (2018). Mixed methods in youth purpose: An examination of adolescent self-regulation and purpose. Journal of Research in Human Development, 15(20, 118-138. DOI; 10.1080/15427609.2018.1445925
• Linver, M.R., & Urban, J.B. (2018). “My life purpose is…”: Assessment of youth purpose in context. Journal of Research in Human Development, 14(20, 95-100. DOI: 10.1080/15427609.2018.1445927
• Chauveron, L.M., Linver, M.R., & Urban, J. B. (2015).  Intentional Self-Regulation and Positive Youth Development: Implications for Youth Development Programs.  Journal of Youth Development, 10(3), 89-101.
• MacDonnell, M., Urban, J.B., & Linver, M.R. (2015).  A Mixed-Methods Exploration of Intentional Self Regulation and Youth Beliefs about the Chances of Achieving Future Aspirations.  Journal of Youth Development, 10(3), 102-114.
• Bowers, E. P., Geldhof, G. J., Chase, P. A., Gestsdottir, S., Urban, J. B., & Lerner, R. M. (2015). Self regulation during adolescence: Variations associated with individual ßà context relations. In J. Wright (Ed.), International Encyclopedia of Social and Behavioral Sciences (2nd ed.). London, UK: Elsevier.
• Birnbaum, A., & Linver, M.R. (2012).  Adolescent physical development and health.  In G. L. Creasey & P. A. Jarvis (Eds.), Adolescent development and school achievement in urban communities: Resilience in the neighborhood (pp. 53-64).  NY: Routledge.
• Mueller, M.K., Lewin-Bizan, S., & Urban, J.B. (2011). Youth activity involvement and positive youth development. In R.M. Lerner, J.V. Lerner, & J.B. Benson (Eds.). Advances in Child Development and Behavior: Positive youth development: Research and applications for promoting thriving in adolescence, Vol 41 (pp. 231-249). Burlington: Academic Press.
• Mueller, M.K., Phelps, E., Bowers, E.P., Agans, J., Urban, J.B., & Lerner, R.M. (2011). Youth development program participation and intentional-self regulation skills: Contextual and individual bases of pathways to positive youth development. Journal of Adolescence, 34, 1115-1125.
• Lerner, R.M., Lerner, J.V., Bowers, E., Lewin-Bizan, S., Gestsdottir, S., & Urban, J.B. (2011). Self-regulation processes and thriving in childhood and adolescence: A view of the issues. New Directions for Child and Adolescent Development, 133, 1-10.
• Gestsdottir, S., Urban, J.B., Bowers, E., Lerner, J.V., & Lerner, R.M. (2011). Intentional self regulation, ecological assets, and thriving in adolescence: A developmental systems model. New Directions for Child and Adolescent Development, 133, 61-76.
• Lerner, R.M., Lerner, J.V., Bowers, E., Lewin-Bizan, S., Gestsdottir, S., & Urban, J.B. (Eds.). (2011). Thriving in childhood and adolescence: The role of self regulation processes. New Directions for Child and Adolescent Development, 133.
• Brown, T. L., Linver, M.R., & Evans, M. (2010).  The role of gender in the racial and ethnic socialization of African American adolescents.  Youth and Society, 41, 357-381.
• Urban, J.B., Lewin-Bizan, S. & Lerner, R.M. (2010). The role of intentional self regulation, lower neighborhood ecological assets, and activity involvement in youth developmental outcomes. Journal of Youth and Adolescence, 39(7), 783-800. doi:10.1007/s10964-010-9549-y
• Brown, T. L., Linver, M.R., Evans, M., & DeGennaro, D. (2009). African American parents’ racial and ethnic socialization and adolescent academic grades:  Teasing out the role of gender.  Journal of Youth and Adolescence, 38, 214-227.
• Linver, M.R., Roth, J.L., & Brooks-Gunn, J. (2009).  Patterns of Adolescents’ Participation in Organized Activities:  Are Sports Best When Combined with Other Activities? Developmental Psychology, 45, 354-367.
• Urban, J.B., Lewin-Bizan, S., & Lerner, R.M. (2009). The role of neighborhood ecological assets and activity involvement in youth developmental outcomes: Differential impacts of asset poor and asset rich neighborhoods. Journal of Applied Developmental Psychology, 30(5), 601-614.
• Urban, J.B. (2008). Components and characteristics of youth development programs: The voices of youth-serving policy makers, practitioners, researchers, and adolescents. Applied Developmental Science, 12(3), 128-139.
• Bradshaw, C., Brown, J.S., and Hamilton, S. (2008). Bridging positive youth development and mental health services for youth with serious behavior problems. Child & Youth Care Forum, 37(5-6), 209-263.
• Bradshaw, C., Brown, J.S., and Hamilton, S. (2006). Applying positive youth development and life-course research to the treatment of adolescents involved with the judicial system. Journal of Addictions and Offender Counseling, 27(1), 2-16.
• Brown, J.S. (2006). Rethinking concept mapping for youth participatory evaluation in the context of youth development programs. Journal of Youth Development: Bridging Research and Practice, 1(2). Retrieved from http://www.nae4ha.org/directory/jyd/index.html
• Brooks-Gunn, J., Linver, M.R., & Fauth, R.C. (2005). Children’s competence and SES in the family and neighborhood.  In A.J. Elliot & C.S. Dweck (Eds.), Handbook of competence and motivation (pp. 414-435).  NY:  Gulliford Press.
• Linver, M.R., & Davis-Kean, P.E. (2005).  The slippery slope:  What predicts math grades in middle and high school?  New Directions for Child and Adolescent Development, 110, 49-64.
• Linver, M.R., Brooks-Gunn, J., & Cabrera, N. (2004).  The Home Observation for Measurement of the Environment (HOME) Inventory:  The derivation of conceptually designed subscales.  Parenting:  Science and Practice, 4, 99-114.
• Roth, J.L., Brooks-Gunn, J., Linver, M.R., & Hofferth, S.L. (2003).  What happens during the school day?: Time diaries from a national sample of elementary school teachers.  Teachers College Record, 105, 317-343.
• Archibald, A.B., Linver, M.R., Graber, J.A., & Brooks-Gunn, J. (2002).  Parent-adolescent relationships and adolescent eating problems:  Testing reciprocal effects.  Journal of Research on Adolescence, 12, 451-461.
• Linver, M.R., Brooks-Gunn, J., & Kohen, D.E. (2002). Family processes as pathways from income to young children’s development.  Developmental Psychology, 38, 719-734.
• Yeung, W.J., Linver, M.R., & Brooks-Gunn, J. (2002).  How money matters for young children’s development:  Parental investment and family process.  Child Development, 73, 1861-1879.