Sprockets CPI Resources

So glad you asked!

The Sprockets Activator Continuous Program Improvement (CPI) Cycle is an annual process designed to strengthen the quality of youth programs and enhance the experiences of young people across Saint Paul.

This improvement cycle helps organizations:

• Collect and use data to understand what’s working

• Reflect on both successes and challenges

• Make informed adjustments to improve programming

• Experience incremental change over time to better meet the needs of youth

Whether you're new to the cycle or have been part of it for years, it's all about building a culture of learning and growth within your staff team and program.

Watch the video below to learn more about how the Activator Cycle works and how it can support your program’s impact.

Look for Fall Kick Off information late summer/early fall.

All are invited to attend the Fall Activator Kick Off. This is a time for you and your team to come together around your Activator Continuous Program Improvement Cycle. At least one person who will serve as the quality lead must attend, but other staff are encouraged to attend as well. This is a great way for new and veteran staff from your team to come together, hear from other organizations what has worked (and not worked) for them, and get organized for your year.

At the kick off, Activator Registration information will be available. Each year you must register, which helps Sprockets staff know who to include in the supports throughout the year including scheduling an external observations, registering for your huddle, and other supports to stay on track throughout the year.

If you are new to the M3 process, it can be a lot to take in. These tools can help you figure out the best place to start this year.

Two helpful resources:

Decision Tree - A simple flowchart that guides you step-by-step through each data set. Your answers will point you toward where to focus your data collection efforts this year.

M3 CPI Planning Guide - This fillable PDF from Ignite Afterschool walks you through planning your CPI cycle, including the foundations of the M3, the types of data to collect, and how to plan for a huddle.

Quality Data

Quality practices are what you see happening in a program at the point-of-service. These are the actions staff take that create the environment and opportunities for participants and is about measuring what can be observed during a youth program. It includes things like creating a physically and emotionally safe environments, scaffolding learning, and opportunities for responsibility, leadership, and youth voice.

Measurement Tools

• SEL/YPQA - Sprockets, along with other networks locally and nationally, use the Weikart Center for Youth Program Quality assessment tools. The Social Emotional Learning Program Quality Assessment (SEL PQA) and the Youth Program Quality Assessment (YPQA) are the two we provide supports around (see below). These frameworks provide a shared language that helps us think about quality across programs.
• Other tools - There are a variety of other quality measurement tools programs may consider. Afterschool Program Assessment System (APAS), Network for Youth Success (NYS), and Elements of Effective Practice for Mentoring are a few examples.

Activator Supports

• Registered Activators will receive one observations from a Sprockets External Assessor.
• PQA Self Assessment Training is provided each fall to train up staff from your organization to understand the PQA framework and prepare staff for internal observations.

Find more information about Quality Data and Sprockets supports and resources here

Attendance and Participation Data

Who shows up, how often, and for how long reveals a lot. The more often young people participate in high-quality afterschool, the better the outcomes. So, tracking who shows up (by age, race, gender, ZIP code, or other factors) and how often is a goldmine of information about your ability to have an impact. This is an area of data that many people have collected over the years. Consider what you program goals are around demographics, enrollment, participation, and retention.

Measurement Tools

• Sprockets Shared Data System - Cityspan - Sprockets provide Saint Paul youth serving programs a way to track registration and participation at no cost to you. If you don't currently have a data management and reporting system, lets connect!
• Other tools - There are a variety of attendance tracking systems that programs may use including Apricot, Salesforce, KidTrax, EZ Reports, and more.

Read more about the Sprockets Shared Data System and access training and support materials here.

Experience Data

What young people think and feel about their experience is important. This data is based on the firsthand reports of people participating in the learning. While young people are the priority, this can also include caregivers, staff, and volunteers. You can collect this data through surveys (some formally designed and tested), reflection activities, interviews and focus groups, or get really creative by using photovoice and drawings.

Check out resources for collecting Experience Data

Outcome Data

Youth outcome data articulates how young people are impacted or changed as a result of participating in your program. This can be changes in attitudes, behaviors, knowledge, skills, etc. What difference did you intentionally design your program to make? At the end of the day, this is about making a positive difference.

Measurement Tools - Common tools used to assess outcomes include pre and post-tests, retrospective surveys completed by participants, staff or caregiver, and school information like attendance and grades. Who knows how the program has impacted participants? This information may come firsthand from youth themselves! Those same activities noted above, may be adjusted, and framed around their concepts of their own skills. Family members may report on what they have observed. Staff may be able to identify pattern changes in the skills areas identified. There are a variety of outcome assessment tools available but not one tool works for everyone.

Sprockets Supports - Check the right hand side bar for resources (soon to come) that might help you identify a tool that would work for you, or decide what you can create to capture the impact your programs makes for young people. And let's connect. Sprockets staff will gladly share what they know about the pros and cons of some of the resources.

Quality practices are what you see happening in a program at the point-of-service. These are the actions staff take that create the environment and opportunities for participants and is about measuring what can be observed during a youth program. It includes things like creating a physically and emotionally safe environment, scaffolding learning, and opportunities for responsibility, leadership, and youth voice.

Here are resources you can use when thinking about Quality Data!

PQA Materials

SEL-PQA Materials

• SEL PQA Tool
• SEL PQA Handbook
• SEL PQA Fillable Notes Sheet

Y- PQA materials

• Youth PQA Tool
• Youth PQA Handbook
• YPQA Fillable Notes Sheet

External Observation

After completing the registration to be an Activator at the Kick Off, Sprockets staff will reach out to connect you and an external assessor. The following details will help your external assessor.

• Location information - parking and entrance details, where to meet the contact, etc.
• General flow of the program. Do youth all meet together at the beginning and then break into separate groups? Will the group be transitioning spaces? Will they be outside or in other spaces the observer should prepare for?
• Be sure to notify staff that the observer is coming and understand the purpose of the observation.

Internal Observation

Having data that you/your staff generate is also helpful and can be done through a traditional observation if you have the ability to do that, or can be through an individual or team reflection.

Getting Started with the PQA

• Team PQA Reflection for Scoring
• Team PQA Consensus Observations
• PQA Internal Observations

Here's a helpful video for thinking about what strategy works best for your team to collect self assessments.

Score Reporter

Activator quality leads receive access to a Scores Reporter account for their organization each year. Use this account to submit your internal observation scores, and to run your reports of both internal and external observations in preparation of the huddle. Below are help videos for how to use Scores Reporter.

• How to enter a self assessment
• How to locate a saved assessment
• How to navigate a self assessment form's index
• How to create a self assessment report
• How to run your PQA report "cheat sheet"

The Sprockets Shared Data System collects and stores information from out-of-school time (OST) providers all across St. Paul, supplying comprehensive information about the participation in OST by St. Paul youth, and illuminating the gaps in opportunities faced by St. Paul youth.

Be Part of the Sprockets Shared Data System

By participating in the Sprockets Shared Data System, we can increase our collective impact on creating successful opportunities for more youth. Your unique username and password will give you access to your data for fundraising, board reports, and to help you manage, evaluate, and improve your programs. Yes, your data, all in one place, in standardized reports, at no or low cost to you.

In addition, you’ll be able to compare your program data to collective information about OST programs in Saint Paul. You’ll also be able to access Saint Paul Public Schools aggregate data for youth in your program — including test scores, attendance, and free-and-reduced price lunch status — to help inform your program offerings.

Data from Sprockets Shared Data System participants and Saint Paul Public Schools is only shared with other participating organizations in aggregate form and with no identifying characteristics. Sprockets partners with Wilder Research to manage and evaluate the data, ensuring the highest levels of privacy protection and accurate analysis.

For more information, or to join the Sprockets Shared Data System, contact Erik Skold at erik.skold@ci.stpaul.mn.us.

Data Partner Resources

If your organization already has a user account, here are important resources for you and your team.

Training Materials

• Training Videos One-Sheeter
• User Responsibilities
• Training Manual
• How to prevent duplicates
• Setting a service term view

Training Videos

• Introduction to the Sprockets Database (7 min)
• Data Privacy and Security (10 min)
• Adding Activities (17 min)
• Adding Participants (11 min)
• Entering Participant Information (12 min)
• Enrolling and Dropping Participants (10 min)
• Entering Attendance (6 min)
• Individual and Event Attendance-optional (5 min)
• Running Reports (14 min)
• Sprockets Database User Responsibilities

Data Privacy Consent Forms & Processes

• Consent Flow Chart
• Research Notice (long form)

Data Privacy Short Forms

• English
• Hmong
• Spanish
• Somali

Opt-out Forms

• English
• Hmong
• Spanish
• Somali

What young people think and feel about their experience is important. This data is based on the firsthand reports of people participating in the learning. While young people are the priority, this can also include caregivers, staff, and volunteers. You can collect this data through surveys (some formally designed and tested), reflection activities, interviews and focus groups, or get really creative by using photovoice and drawings.

Experience Data Resources

Getting ready for your upcoming M3 Huddle? We’ve got you covered! Watch the video, or scroll down for all the information and resources you need.

Participation Data

As you prepare your attendance and participation data, here are a few things to consider:

• Program Scope: Will you run the data by individual program sites or aggregate across multiple programs?
• Timeframe: What date range will you use? Do you want to compare this timeframe to the same period last year?
• Attendance Breakdown: Will you analyze participation by number of days attended? What breakdown makes the most sense given your program structure at this point in the year?
• Data Use: What insights would be most helpful?
• Goals: What are your current participation goals—frequency, duration, specific demographics (e.g., zip codes, ages, grades, race/ethnicity)?

Here are resources if you are a part of the Sprockets Shared Data System that will help you run your reports.

Running your Attendance/Participation Reports through Sprockets Shared Data System (14 min)

Quality Data

Getting Started with the PQA

• Team PQA Reflection for Scoring
• Team PQA Consensus Observations
• PQA Internal Observations

Score Reporter

Activator quality leads receive access to a Scores Reporter account for their organization each year. Use this account to submit your internal observation scores, and to run your reports of both internal and external observations in preparation of the huddle. Below are help videos for how to use Scores Reporter.

• How to enter a self assessment(External link)
• How to locate a saved assessment(External link)
• How to navigate a self assessment form's index(External link)
• How to create a self assessment report(External link)
• How to run your PQA report "cheat sheet"

Experience and Outcome Data

More coming

Prepare Your Team

If you or your staff are new to the huddle, THIS M3 HUDDLE INTRO VIDEO lays the groundwork for how we approach data with care and structure the day to support those attending.

You huddled! Now what?

At the huddle your team was able to identify what goals you want to commit to moving forward. Now it's time for your team to commit to action for their Improvement Plan. This video will help you think about how to move your ideas to tangible actions.

Activator Timeline at a Glance

Check the details below for additional information on each phase of the cycle.