The SAGE EPO portfolio is evaluated through a strategic combination of internal and external evaluation practices, applied throughout the lifecycle of a project, with the goal of maximizing the desired programmatic impact. This approach is approach, based on the Collaborative Impact Analysis Method (IAM) (Davis and Scalice, 2015), which combines staff knowledge of programs and products, audiences, and content, with the expertise of an outside evaluator. This combination captures the effects on the behaviors, attitudes, skills, interests, and/ or knowledge of users/program participants, while achieving efficiencies by having SAGE staff conduct much of the development of assessment instruments, data collection efforts, and preliminary data analysis. To ensure success, an external evaluator provides consultation, review and feedback, and/or more robust analysis of data. Risks are monitored on a project-by-project basis as part of the evaluation process, allowing resources to be reallocated as needed to keep projects on schedule.
Each project in the portfolio is periodically reviewed jointly with the external evaluator. Working with the staff lead for the project, the external evaluator scores the robustness of the project’s current evaluation, using a qualitative rubric based on best practices. The outcome of these reviews is a project score and steps to improve the project’s evaluation and impact. In this way, the process delivers the formative and impact data to ensure project efficacy and efficiency. Periodically, each project prepares a report on the impact of the evaluation on the project going forward. These reports are used for high-level, cross-program analysis and strategic planning. In addition, the program activities are monitored regularly by the EPO Standing Committee and at longer intervals by a separate external panel.
Davis, H., (2022). SAGE Evaluation of Project/Program Evaluations.
Taber, J. Bohon, W., Bravo, T., Dorr, P., Hubenthal, M., Johnson, J., Sumy, D., Welti, R. Davis, H. (2017) Increasing the use of evaluation data collection in an EPO program ED43A-01 presented at 2017 Fall Meeting, AGU, San Francisco, CA.
Taber, J. Bohon, W., Bravo, T., Dorr, P., Hubenthal, M., Johnson, J., Sumy, D., Welti, R. Davis, H. (2016) Motivation for Evaluation: A road map for improving program efficacy ED51G-0839 presented at 2016 Fall Meeting, AGU, San Francisco, CA.
Taber, J., Hubenthal, M., Davis, H. (2016). Evaluating Facilities-based Education and Public Outreach Activities. Presented at the Large Facilities Workshop May 2016, Washington, DC.
Taber, J. & Hubenthal, M. (2012). Lessons learned from SAGE EPO program evaluations ED41D-0700 presented at 2012 Fall Meeting, AGU, San Francisco, CA.
Wagner, C., McCullough, J., Tyler, A., Gwatkin, L., Hancock, M., (2009). Evaluation of the SAGE Education and Outreach Program.
Distinguished Lecture Series
Internships
Meetings
Posters
Professional Development
Seismographs in Schools
Social Media
Software
Teachable Moments
Website
Crayne, J., C. Herran, D.F. Sumy, M. Benne, T. Shaggot, and L. Peek (2022), Evaluation of an online, educational animation to communicate about ShakeAlert Earthquake Early Warning, in preparation for submission to the International Journal of Science Education.
Hubenthal, M., & Brudzinski, M. (2022), Factors Influencing Persistence in an Open-access, Online Scientific Computing Training Using Seismic Data , in preparation for submission to the Journal of Geoscience Education.
Sumy, D.F., M.R. Jenkins, J. Crayne, S.E. Olds, M.L. Anderson, J. Johnson, B. Magura, C.L. Pridmore, and R.-M. de Groot (2022), Education Initiatives to Support Earthquake Early Warning: a Retrospective and a Roadmap, In press with Seismological Research Letters.
Zawacki, E. E., Bohon, W., Johnson, S., and Charlevoix, D. J. (2022), Exploring TikTok as an effective platform for geoscience communication, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2022-494.
Hubenthal, M., & Taber, J. (2022). Moving beyond S Minus P Earthquake Locations as “THE” Lab in Seismology Education. Seismological Research Letters, 93(4), 2347–2359. doi.org/10.1785/0220210251
Sumy, D. F., Jenkins, M. R., McBride, S. K., & de Groot, R. M. (2022). Typology development of earthquake displays in free-choice learning environments, to inform earthquake early warning education in the United States. International Journal of Disaster Risk Reduction, 73, 102802. doi.org/10.1016/j.ijdrr.2022.102802
Brudzinsky, M., M. Hubenthal, S. Fasola, E. Schnorr (2021) Learning in a Crisis: Online Skill Building Workshop Addresses Immediate Pandemic Needs and Offers Possibilities for Future Trainings, Seismological Research Letters, Vol 92, No. 5, 3215–3230. doi.org/10.1785/0220200472
Bravo T, Taber J and Davis H (2020) A Case Study of Highly-Engaged Educators' Integration of Real-Time Seismic Data in Secondary Classrooms. Front. Earth Sci. 8:180. doi: 10.3389/feart.2020.00180
Hubenthal, M., Bohon, W., & Taber, J. (2020). A Pandemic Pivot in Earth Science Outreach and Education. Eos, Transactions American Geophysical Union, 101, 1–7, doi.org/10.1029/2020EO152146.
Sumy, D.S., R. Welti, M. Hubenthal (2020) Applications and Evaluation of the SAGE Earthquake Browser: A Web‐Based Tool That Enables Multidimensional Earthquake Visualization, Seismological Research Letters, Vol. 91, No. 5, 2922–2935. doi.org/10.1785/0220190386
Lorena, M. L., Beth, B., Michael, H., & Rebecca, H. (2019). Bilingual science communication: A call for a geoscience community of practice. Journal of Geoscience Education, 0(0), 1–5. doi.org/10.1080/10899995.2019.1578580
Bartel, B.,Agopian, M., and Bohon, W. (2019), The unexpected benefits of science communication training, Eos, 100, doi.org/10.1029/2019EO129023. Published on 24 July 2019.
Hubenthal, M. (2018). Exploring undergraduates’ conceptions of elasticity, within a plate tectonics context, before and after experience with rock’s elastic behavior. Journal of Geoscience Education, 66(4), 261–277. doi.org/10.1080/10899995.2018.1493964
LaDue, N. D., J. Taber, and M. Hubenthal (2015), Promoting new collaborations for education research in geoscience, Eos, 96, doi:10.1029/2015EO032123
Taber, J., Hubenthal, M., Bravo, T., Dorr, P., Johnson, J., McQuillian, P., Sumy, D. F., & Welti, R. (2015). Seismology education and public-outreach resources for a spectrum of audiences, as provided by the SAGE Consortium. The Leading Edge, 34(10), 1178–1184. doi.org/10.1190/tle34101178.1
Hubenthal, M., & Judge, J. (2013). Taking Research Experiences for Undergraduates Online. Eos, Transactions American Geophysical Union, 94(17), 157–158. doi.org/10.1002/2013eo170001
Hubenthal, M., O’Brien, T., & Taber, J. (2011) Posters that foster cognition in the classroom: multimedia theory applied to educational posters. Educational Media International, 48(3), 193–207. doi.org/10.1080/09523987.2011.607322
Smith, M., J. Taber, and M. Hubenthal. (2006) Real-Time Seismic Displays in Museums Appeal to the Public, Eos Trans. AGU, 87 (8), 85, 91. doi.org/10.1029/2006EO080002
Hubenthal, M., Boyd, T., Lahr, J. Taber, J. (2003). Undergraduate engineering students investigate inexpensive seismometer design. Eos, Transactions American Geophysical Union 84 (18), 166-171. doi.org/10.1029/2003EO180003
Hennet, C.B., J.J. Taber, G.E. van der Vink, C.R. Hutt. (2003) Earthquakes in Museums, Seismological Research Letters, Vol 74, No. 5, 628-634.
Bartel, B., Bohon, W., Frank, L., Stovall, W., Poland, M. (2020). Communicating Geohazards: Delivering Information in Crisis and Calm, White paper.
Bartel, B., & Bohon, W. (Eds.). (2019). The Hazards of Hazard Communication: Importance, Rewards, and Challenges of Science in the Public Sphere: A white paper summary of presentations from session PA23B at the 2018 Fall Meeting of the American Geophysical Union. White Paper.
Hubenthal, M. (2018). Bending Rocks in the Classroom. In The Trenches, 8(1), 4–5.
LaDue, N. D., Hubenthal, M., & Dolphin, G. (2018). Using Models to Develop Deep Understandings of Earthquakes. In The Trenches, 8(1), 1–3.
Bravo, T., & Hubenthal, M. (2016). Using Earthquakes as" Teachable Moments". The Science Teacher, 83(3), 14.
Hubenthal, M., Stein, S., & Taber, J. (2012). A Big Squeeze: Examining and Modeling Causes of Intraplate Earthquakes in the. The Earth Scientist, 27(1), 33–39.
Bravo, T., M. Hubenthal, M. Wysession, and R. Butler (2011). Educational aftershocks: Engage students with rapid-response resources, In The Trenches 1, no. 4, 4–7.
Hubenthal, M., Ervin, T. (2011) The Earth Scientist: Seismology Special Issue, (Eds.), 27(1), 1- 39.
Hubenthal, M. (2009, April). Wallpaper or instructional aids: A preliminary case study of science teachers’ perceptions and use of wall-posters in the classroom (S3.10.3). Paper presented at the annual meeting of the National Association of Research in Science Teaching Annual International Conference. Garden Grove, CA. Abstract | White Paper
Hubenthal, M., Braile, L., & Taber, J. (2008). Redefining earthquakes and the earthquake machine. The Science Teacher, 75(1), 32–36.
Wysession, M., Hubenthal, M., & Taber, J. (2008). Using SeisMac to Turn Your Laptop into a Seismograph for Teaching. Seismological Research Letters, 79(5), 723–725. doi.org/10.1785/gssrl.79.5.723
Hubenthal, M. (2006). Revisiting the BOSS model to explore building resonance phenomena with students. The Earth Scientist, 22(2), 12–16.
Hubenthal, M., & Ervin, T. (2005) The Earth Scientist: Seismology Special Issue, (Eds.), 21(2), 1- 32.
Davis, H. & Scalice, D. (2015). Evaluate the Impact of your Education and Outreach Program Using the Quantitative Collaborative Impact Analysis Method (Invited), AGU Fall Meeting, Abstract # ED53D-0871.
