Sprockets CPI Resources

So glad you asked!

The Sprockets Activator Continuous Program Improvement Cycle (CPI) is an annual process designed to enhance the quality of your programs and the experiences of youth participants. The Activator Improvement Cycle supports organizations in collecting and using data, reflecting on successes and challenges, and informing adjustments that ensure your organization is continuously evolving to better serve Saint Paul’s youth. Watch the video below to learn more about the cycle.

Activator Timeline at a Glance

Check the details below for additional information on each phase of the cycle.

Look for Fall Kick Off information late summer/early fall.

All are invited to attend the Fall Activator Kick Off. This is a time for you and your team to come together around your Activator Continuous Program Improvement Cycle. At least one person who will serve as the quality lead must attend, but other staff are encouraged to attend as well. This is a great way for new and veteran staff from your team to come together, hear from other organizations what has worked (and not worked) for them, and get organized for your year.

At the kick off, Activator Registration information will be available. Each year you must register, which helps Sprockets staff know who to include in the supports throughout the year including scheduling an external observations, registering for your huddle, and other supports to stay on track throughout the year.

Quality Data

Quality practices are what you see happening in a program at the point-of-service. These are the actions staff take that create the environment and opportunities for participants and is about measuring what can be observed during a youth program. It includes things like creating a physically and emotionally safe environments, scaffolding learning, and opportunities for responsibility, leadership, and youth voice.

Measurement Tools

• SEL/YPQA - Sprockets, along with other networks locally and nationally, use the Weikart Center for Youth Program Quality assessment tools. The Social Emotional Learning Program Quality Assessment (SEL PQA) and the Youth Program Quality Assessment (YPQA) are the two we provide supports around (see below). These frameworks provide a shared language that helps us think about quality across programs.
• Other tools - There are a variety of other quality measurement tools programs may consider. Afterschool Program Assessment System (APAS), Network for Youth Success (NYS), and Elements of Effective Practice for Mentoring are a few examples.

Activator Supports

• Registered Activators will receive one observations from a Sprockets External Assessor.
• PQA Self Assessment Training is provided each fall to train up staff from your organization to understand the PQA framework and prepare staff for internal observations.

Find more information about Quality Data and Sprockets supports and resources here

Attendance and Participation Data

Who shows up, how often, and for how long reveals a lot. The more often young people participate in high-quality afterschool, the better the outcomes. So, tracking who shows up (by age, race, gender, ZIP code, or other factors) and how often is a goldmine of information about your ability to have an impact. This is an area of data that many people have collected over the years. Consider what you program goals are around demographics, enrollment, participation, and retention.

Measurement Tools

• Sprockets Shared Data System - Cityspan - Sprockets provide Saint Paul youth serving programs a way to track registration and participation at no cost to you. If you don't currently have a data management and reporting system, lets connect!
• Other tools - There are a variety of attendance tracking systems that programs may use including Apricot, Salesforce, KidTrax, EZ Reports, and more.

Read more about the Sprockets Shared Data System and access training and support materials here.

Experience Data

What young people think and feel about their experience is important. This data is based on the firsthand reports of people participating in the learning. While young people are the priority, this can also include caregivers, staff, and volunteers. You can collect this data through surveys (some formally designed and tested), reflection activities, interviews and focus groups, or get really creative by using photovoice and drawings.

Outcome Data

Youth outcome data articulates how young people are impacted or changed as a result of participating in your program. This can be changes in attitudes, behaviors, knowledge, skills, etc. What difference did you intentionally design your program to make? At the end of the day, this is about making a positive difference.

Measurement Tools - Common tools used to assess outcomes include pre and post-tests, retrospective surveys completed by participants, staff or caregiver, and school information like attendance and grades. Who knows how the program has impacted participants? This information may come firsthand from youth themselves! Those same activities noted above, may be adjusted, and framed around their concepts of their own skills. Family members may report on what they have observed. Staff may be able to identify pattern changes in the skills areas identified. There are a variety of outcome assessment tools available but not one tool works for everyone.

Sprockets Supports - Check the right hand side bar for resources (soon to come) that might help you identify a tool that would work for you, or decide what you can create to capture the impact your programs makes for young people. And let's connect. Sprockets staff will gladly share what they know about the pros and cons of some of the resources.

The Sprockets Shared Data System collects and stores information from out-of-school time (OST) providers all across St. Paul, supplying comprehensive information about the participation in OST by St. Paul youth, and illuminating the gaps in opportunities faced by St. Paul youth.

Be Part of the Sprockets Shared Data System

By participating in the Sprockets Shared Data System, we can increase our collective impact on creating successful opportunities for more youth. Your unique username and password will give you access to your data for fundraising, board reports, and to help you manage, evaluate, and improve your programs. Yes, your data, all in one place, in standardized reports, at no or low cost to you.

In addition, you’ll be able to compare your program data to collective information about OST programs in Saint Paul. You’ll also be able to access Saint Paul Public Schools aggregate data for youth in your program — including test scores, attendance, and free-and-reduced price lunch status — to help inform your program offerings.

Data from Sprockets Shared Data System participants and Saint Paul Public Schools is only shared with other participating organizations in aggregate form and with no identifying characteristics. Sprockets partners with Wilder Research to manage and evaluate the data, ensuring the highest levels of privacy protection and accurate analysis.

For more information, or to join the Sprockets Shared Data System, contact Erik Skold at erik.skold@ci.stpaul.mn.us.

Data Partner Resources

If your organization already has a user account, here are important resources for you and your team.

Training Materials

• Training Videos One-Sheeter
• User Responsibilities
• Training Manual
• How to prevent duplicates
• Setting a service term view

Training Videos

• Introduction to the Sprockets Database (7 min)
• Data Privacy and Security (10 min)
• Adding Activities (17 min)
• Adding Participants (11 min)
• Entering Participant Information (12 min)
• Enrolling and Dropping Participants (10 min)
• Entering Attendance (6 min)
• Individual and Event Attendance-optional (5 min)
• Running Reports (14 min)
• Sprockets Database User Responsibilities

Data Privacy Consent Forms & Processes

• Consent Flow Chart
• Research Notice (long form)

Data Privacy Short Forms

• English
• Hmong
• Spanish
• Somali

Opt-out Forms

• English
• Hmong
• Spanish
• Somali

Quality practices are what you see happening in a program at the point-of-service. These are the actions staff take that create the environment and opportunities for participants and is about measuring what can be observed during a youth program. It includes things like creating a physically and emotionally safe environment, scaffolding learning, and opportunities for responsibility, leadership, and youth voice.

Here are resources you can use when thinking about Quality Data!

PQA Materials

• SEL PQA Tool
• SEL PQA Reference Guide
• SEL PQA Notes Sheet
• YPQA Tool
• YPQA Reference Guide
• YPQA Notes Sheets

External Observation

After completing the registration to be an Activator at the Kick Off, Sprockets staff will reach out to connect you and an external assessor. The following details will help your external assessor.

• Location information - parking and entrance details, where to meet the contact, etc.
• General flow of the program. Do youth all meet together at the beginning and then break into separate groups? Will the group be transitioning spaces? Will they be outside or in other spaces the observer should prepare for?
• Be sure to notify staff that the observer is coming and understand the purpose of the observation.

Internal Observation

Having data that you/your staff generate is also helpful and can be done through a traditional observation if you have the ability to do that, or can be through an individual or team reflection.

• Self Assessment Tips
• New to the PQA Team Reflection
• Team PQA Reflections
• Team PQA Consensus Meeting
• PQA Internal Observation

Here's a helpful video for thinking about what strategy works best for your team to collect self assessments.

Score Reporter

Activator quality leads receive access to a Scores Reporter account for their organization each year. Use this account to submit your internal observation scores, and to run your reports of both internal and external observations in preparation of the huddle. Below are help videos for how to use Scores Reporter.

• How to enter a self assessment
• How to locate a saved assessment
• How to navigate a self assessment form's index
• How to create a self assessment report
• How to run your PQA report "cheat sheet"

You huddled! Now what?

At the huddle your team was able to identify what goals you want to commit to moving forward. Now it's time for your team to commit to action for their Improvement Plan. This video will help you think about how to move your ideas to tangible actions.