The way that you organize data collection in Hello Insight will affect how you see the data in final reports. So, it’s important to spend some time defining your goals, reviewing survey administration guidance, and planning your strategy before you begin.
Step 1: Understand Hello Insight’s Three Levels of Data
Data collection and reporting are organized in a hierarchy with three levels, starting with Programs, then Sites, and then Groups.
A Program is a set of coordinated activities, services, or resources that are conducted with fidelity and the same level of intensity or dosage across Sites or Groups. No matter who runs the program or where it is offered, you should expect that young people will have comparable experiences and develop similar outcomes.
A Program can contain one or more Sites, each with one or more Groups. However, all need to use the same Hello Insight Tool. This will ensure consistency across the Program and allow for comparison across Sites and Groups.
When you generate a Program-level report, you will see a roll-up across all Sites and Groups within that Program.
A Site typically refers to a location or region where a Program takes place (such as, Center City, East Avenue Elementary School, or Northwest Recreation Center). It can, however, also be the name of a host or staff member.
A Site must contain one or more Groups.
When you generate a Site-level report, you will see a roll-up across all Groups within that Site.
A Group is the smallest set of participants within a Site that experience the same Program at the same time. It is often identified as a classroom name, a sports team, or the name of an instructor.
When you generate a Group-level report, you will see only that Group of young people.
Step 2: Design a Data Collection and Reporting Structure
To help you design your data collection and reporting structure, review the key considerations and helpful tips outlined below. You can also use the printable Template for Planning Data Collection and Reporting to experiment with mapping various structures for your Programs, Sites, and Groups.
Things to Consider When Defining Each Level
Number of Programs: Determine how many Programs you will evaluate this year. If this is your first year using Hello Insight, start with attainable goals you can build on. For your organization, this could mean beginning with just one or two Programs with the goal of expanding to others later.
Ages Served: Each Hello Insight tool has been validated for a specific age range, so it is important to consider the ages of the young people your Program serves.
Hello Insight: Career Readiness, College Readiness, SEL at Camp, Social and Emotional Learning, Sports, and Youth Leadership in Action tools can be used with young people ages 11 to 23.
Use Hello Insight: Elementary SEL for children ages 8 to 11 or Hello Insight: Early Elementary SEL for ages 5 to 8 (available in Spring 2022). These children will have markedly different experiences in your Program, and you should expect different and more developmentally-appropriate outcomes for them. Note: Only one Hello Insight Tool can be selected when adding a Program. Therefore, if you serve young people from different age ranges within a Program, you will need to create a unique Program for each distinct age group.
Which Hello Insight Tool Will Be Used: If a Tool hasn’t already been chosen for you through a Scholarship, consider which one is most appropriate for each Program you want to evaluate. Check out the complete list of Hello Insight’s tools to learn more about each tool and their research-based theories of change.
Naming Conventions: Include a date range in the names to support year-to-year comparisons (for example, “2021-22 Afterschool Program” or “Summer 2022 Camp”).
Number of Sites: Determine how many Sites you will study this year. Remember to make your goals attainable. If this is your first year with Hello Insight, consider working with Sites that you know are operating with fidelity, with outstanding staff, and where you will likely have success. If these staff members like using Hello Insight, they can become champions of the platform for the rest of your staff next year.
Naming Conventions: Choose site names that are easily understandable to everyone in your organization. If you are creating multiple Sites, try to make their names consistent. Consider how these names might scale over time. For example, do you want all Sites to be named based on school names, physical locations, or funding sources.
Number of Groups: Because each Group receives unique URLs and PDFs, be very careful about the number you create. Try to balance your need for detailed data with a clear understanding of what it will take to manage survey administration strategies on the ground. When you first start, consider the old adage that less is more!
Naming Conventions: Choose Group names that are easily understandable to everyone in your organization. If you are creating multiple Groups, try to make their names consistent and provide relevant descriptors. For example, “Coding Club Grades 4 to 6”, “Cycling Adventures – Beginner Level” or “Camp Session #1.” Consider how these names might be used over time.
Step 3: Create a Data Collection Plan
Create a data collection plan for each Program, Site and Group that your organization intends to survey. The Survey Administration Planning Worksheet is a guide to help you plan your survey administration strategy for each Sites.
Be sure to:
Add the name of the Program and the Hello Insight Tool that you will be using. If you haven’t already determined this, check out www.ins.gt/tools to learn more about each tool and their research-based theories of change.
Add the names of each Group to be surveyed within the Site.
Estimate the number of young people who will be surveyed in each group.
Determine the approximate dates that you will administer pre/post and/or check-in surveys within each Group. Check out Scheduling Survey Administration: Tips and Strategies and Check-In vs. Pre/Post Surveys.
Fill out the Online or On Paper column to document how each group will collect surveys.
Decide how and when staff will be prepped for survey administration.
Think through the best timing and approach for introducing surveys at each site, including how to resolve potential challenges and who will assist with adding Program information to the Dashboard.