College of Exploration

What We Do

Home → What We Do → Educational Research and Evaluation

Educational Research and Evaluation

The College of Exploration has provided evaluation leadership and support to a variety of STEM education projects at the national level over the past 25 years.  Our experience and expertise include (but are not limited to):

  • Longitudinal Research
  • Needs Assessment
  • Program Evaluation
  • Project Evaluation Guidance
  • Advisory and Technical Support
  • Survey Design
  • Lesson Plan and Educational Module Review
  • Focus Group implementation
  • Developmental Evaluation

 

In Search of Earth's Secrets

The College of Exploration is providing external evaluation services for the In Search of Earth' Secrets five year project funded by the AISL program of NSF. The goal of this project is to make deep earth/ocean science available to informal audiences, bringing the excitement of the Joides Resolution ship and its underwater ocean drilling expeditions to the general public-learners of all ages at libraries, museums, and science centers.
The team, under the direction of Consortium of Ocean Leadership and Lamont-Doherty Earth Observatory consists of informal science venues personnel, scientists and educators in the JRON community, exhibit and graphic designers, Girl Scout groups as well as an evaluation team and an advisory board.
The evaluators will look at the learning outcomes of the project for all participants and stakeholders, with special emphasis placed on minorities, rural populations and girls through the Girl Scout organization. The evaluation will examine the use of STEM content and skills development in public outreach, network and partnership support for learning, support for informal educational venues and enhanced interest and awareness in ocean science by the public.


NOAA Living Marine Resources Cooperative Science Center


The College of Exploration is now engaged in a five-year (2016-2021) evaluation process for the Living Marine Resources Cooperative Science Center (LMRCSC). This Center, funded by NOAA, is led by the University of Maryland Eastern Shore and is a consortium of seven universities nationwide who collaborate on the educational offerings.  The Center “trains and graduates students from underrepresented communities in marine science for careers in research, management, and public policy that support the sustainable harvest and conservation of our nation's living marine resources”. With its partner institutions, the LMRCSC conducts research on marine and estuarine systems consistent with NOAA science. It offers courses, workshops, internships and other educational opportunities for students at the undergraduate, graduate, and post- doctoral levels, as well as teacher and faculty support for teaching and learning about fisheries, ecosystem management, and aquaculture.
The evaluation team, working with the LMRCSC team will utilize multiple sources of information to inform the impacts of project members and activities toward the identified outputs and outcomes, the impact of the project on the members themselves as social network participants, and the broader impacts of the project on the various participants and beneficiaries of the project activities.  A main focus will be on looking at benefits to students at all levels from their LMRCSC activities and tracking their career pathways as they go through the LMSCRC system and out into the working world.

Integration and Application (IAN) Network
ian.umces.edu

The Integration and Application Network of the University of Maryland Center for Environmental Science has engaged the services of the College of Exploration to consult with them in a two-fold organizational assessment and evaluation. IAN creates innovative ways to visually present science, making it more accessible to managers, stakeholders, and the general public. IAN offers science communication services, environmental report card production, and training and capacity building.
The College of Exploration is working hand in hand with the IAN team in developing and implementing a corporate assessment process through a two-pronged approach. First it looks at the effectiveness of IAN in its work with clients as reported by the IAN partners. Secondly, it looks at employee satisfaction within the internal organization and reflects this satisfaction through a “happiness index” and report card process.

The College of Exploration is also providing support for the IAN group for professional development, looking at communication approaches, enrichment activities, and team sharing as a learning system.

Longitudinal Research: The National Ocean Sciences Bowl
www.nosb.org

One long-term project is a longitudinal research study for the Consortium for Ocean Leadership’s National Ocean Sciences Bowl since 1999. This study has looked holistically at the entire NOSB program and its effects on a variety of key audiences, but in particular it follows student alumni who participate in the competition. One key goal has been to track these alumni participants as they move into college and then on to careers. The College of Exploration has managed to keep in touch with more than 500 alumni who regularly provide our team with information about their college choices, coursework, majors and career choices.

In 2107   TCOE continued to track students who have participated in this academic competition through high school into undergraduate and graduate programs and on into careers.  In addition, the evaluation team focused on corporate support for academic programs, with particular emphasis on, and research of the Eastman Company, which has sponsored an NOSB team in Kingsport, TN and also an oceanography course for Dobyns-Bennett High School in Kingsport

For this NOSB research study TCOE has developed dozens of survey instruments and multiple interview protocols to guide project development. Multiple juried publications have emerged from this body of work including:

Bishop, Tina & Walters, Howard D. (2007). The National Ocean Sciences Bowl: Extending the Reach of a High School Academic Competition to College,   Careers, and a Lifelong Commitment to Science.  American Secondary Education, 35(3).

Walters, Howard D., Bishop, Tina, & Wlodarsky, Rachel (2006).  An Impact Assessment of the National Ocean Sciences Bowl—Lessons Learned from a        National Two-Year Study.  Special Report No.2: Current: The Journal of     Marine Education.  Published jointly by the Consortium of Oceanographic Research and Education, NOAA/National Ocean Services, and the National Marine Educators Association.

Evaluation Strategic Planning and Evaluation Advisory Support: The Joint Oceanographic Institution (now part of COL)

In 2005-2006, The College of Exploration (Dr. Walters, Dr. Bishop and Peter Tuddenham) conducted a strategic evaluation for the Joint Oceanographic Institution (JOI) of both existing JOI education tools and materials as well as an assessment of the broader ocean science and science education communities nationally with a view toward the continued development and enhancement of JOI's stature within those communities.  This project led to the familiarization of TCOE evaluators with the JOI and IODP environment, culture, products and activities at that time. This has led to our current evaluation support project with former JOI staff (now part of COL), which is an NSF-funded informal science education project called Ship to Shore Science. http://www.oceanleadership.org/education/deep-earth-academy/educators/school-of-rock/sor-2012-ship-to-shore/

Our role in Ship to Shore Science is to advise and support the informal science education pilot projects and to inform and guide decision-making to the PIs as this project unfolds.

For this strategic evaluation, the evaluators supported the goal of comparing other nationally recognized successful education programs by establishing a project website with virtual meeting spaces for project personnel,  establishing and conducting an online focus group of leading science educators, and preparing a report comparing  JOI education program elements with comparable elements from similar national science education entities.  To address the goal of both establishing baseline data and assessing the effectiveness of existing materials and scientific ocean drilling information against the baseline, Walters and Bishop developed and implemented a survey, which established a baseline of the extent of educators' use of the materials and methods of JOI, the degree to which the JOI materials and educational programs/methods met state, regional, and national science education standards and curricula frameworks, and the match between teacher instructional objectives. An analysis of focus group discussion comments and survey results helped identify strengths, weaknesses and opportunities for JOI Education. The evaluators also conducted a literature review of evaluation and educational methodologies, tool development, emerging trends in professional development for pre-service and in-service teachers, and teachers' use of the web as a source of curriculum materials. A final report provided a summary of all research and evaluative data with recommendations to support decision-making for JOI education and plans for future evaluation of materials and programs.

Online Workshop and Training Evaluation, Educator Needs Assessment

The College of Exploration has a distinguished national and international reputation for designing online spaces and programs for professional development having provided these for a broad cross set of agencies including:  National Geographic Education, NOAA Office of Ocean Exploration and Research, and other NOAA Program Offices, including NOAA Education and NOAA Teacher at Sea, as well as various NSF national projects and partnerships.  Using this online design, the College of Exploration further convened and facilitated the virtual planning processes that created the Ocean Literacy Essential Principles and Fundamental Concepts. Since 2000 the College of Exploration has regularly employed online surveys and online focus groups, as well as tracking online dialogue to assess the online professional development programs. In addition, The College of Exploration has provided needs assessment surveys for clients and for itself to help ascertain needs of educators for professional development and educational resources.

Bishop, Tina & Walters, Howard D. (2003).  The Sustainable Seas Expedition: An Impact Assessment of an Online Teacher Education Program Funded by the National Geographic Society.  Current: The Journal of Marine Education Winter 2002.

Walters, Howard D. & Bishop, Tina (2004).  The Classroom Exploration of the Oceans Online Workshop Series.  Current: The Journal of Marine Education. June 2004.

Online Module Review and Lesson Plan Review

The College of Exploration has provided review and pilot-testing support to clients who have lesson plans and modules to test with educators. We have provided online focus groups for module review and also have created surveys to get feedback from lesson plan reviewers. Clients of this pilot testing include National Park Service, National Geographic Society, and NOAA Fisheries.

Developmental Evaluation

Throughout our time as evaluators, we have consistently and implicitly adhered to the principles of developmental evaluation, even when not explicitly articulating the approach. We have found that this approach, which was put forth by Michael Quinn Patton, offers a more holistic and creative view of evaluation. It has seemed appropriate for a number of our evaluation settings, in which goals are evolving and emerging as the project unfolds. Developmental evaluation realizes the complex and dynamic environment of educational programs and employs methods that engage the learning team in co-creating change. For a current project in New Hampshire--the Math and Science Partnership--The College of Exploration is working with the team to incorporate developmental evaluation principles as the project dynamically changes with new connections occurring in both STEM content and participants in districts across the state.