Sunday, June 23, 2013

Survey Results are here!

Hockey parents are tough. We’re dedicated. And we’re not afraid to tell you what we think. It was no surprise then that I got so many responses to the first of what I plan to be annual surveys of hockey parents.

The survey was conducted during the month of May, 2013. This was soon after most programs in the area had held evaluations for next season’s teams. Yes, the annual ritual: trying to figure out who’s playing where, which tryouts overlap which, should we go Tier 1 or Tier 2, who’s the coach, etc., etc., etc.

Travel hockey is different from most sports in the sense that there’s no local/regional restriction on which team a child can join. As long as the parents are willing to drive, kids can try out for whichever team they choose. Every season, kids shuffle from one program to the next for one reason or another.

I didn’t play travel hockey as a child, so when my son first started playing, this annual dance seemed really important and dramatic to me. I got sucked in like everyone else! Now, over five years later, it’s just part of the life of a hockey parent. Every year a few of your child's teammates are going to move to a different program.

The purpose of this survey was to find out why.

We all have our theories, and in reality there are many reasons people could choose to switch hockey programs. But are there any trends? How happy were parents with their hockey experience last season? Do parents more often move their children to take advantage of an opportunity or to escape a bad situation? How do parents decide between programs? What are parents’ priorities and how do they align with those of the coaches? What percentage of kids will actually change programs next year?

Call it a customer satisfaction survey. I asked what you thought about your hockey experience in 2012-13, and you told me. Thanks.

Some people have asked or may wonder why I developed and published this survey. In part, it was to satisfy my curiosity. I’m a loyal guy. We’ve been in the same program since my oldest son was a mite and never had a bad experience – or at least, not bad enough to make me want to leave. I wonder why people leave and wanted to get some concrete data to tell me why they do.

One thing I want to make clear: I did not create this survey at the behest of or for the benefit of any particular hockey program. I created the survey questions myself, with input from a few close, very select advisors (thanks, honey).

But – if I were running a youth hockey program this is the kind of information I’d sure like to know.

I've broken the survey results into two sections. There was just too much information to share in one post. This post gives some general information about the survey respondents and a summary of the results of the “How satisfied were you” questions from survey. Those questions covered the categories Overall hockey experience, Coaching, Practice, and Games.

In an upcoming post, I'll compare the priorities of the parents and the coaches. And I'll cover the parents’ reasons for changing programs – or staying with the same one.

Data Scrubbing

I won’t bore you with the details, but in some cases I had to adjust the raw survey responses. For example, respondents could type in the team name however they wanted so I needed to standardize the names so that “Valley Forge Colonials”, “VFC”, and “Colonials” were all recognized as the same team. That wasn't the only example but it is one of them. This was the most tedious part of the process. I can provide more details on the specific changes if anyone is interested.

Survey Respondents / Demographics

In total, the survey received 115 responses. Of those responses, all but 7 were included in the results presented here. Regarding the responses that were removed:
  • Three were removed because they referred to more than one child or more than one team in the same response
  • Four were removed because they were not related to travel hockey teams
The remaining responses spanned 26 different hockey teams. All but a few responses came from eastern PA, DE, and NJ. There were also a couple of responses from western PA and one from MA. There was one team that accounted for a large percentage of the total responses: the Valley Forge Colonials (31%). This is a function of my contacts being mostly VFC families and does not have a material impact on the results when viewed in aggregate. 

By age group, there were responses from parents of all age groups from Mite up through Midget. The majority of responses came from second year Squirts and second year PeeWees.




By level, the majority of the responses were from parents of Tier 2 players. There was a fairly balanced split, with A and B level players making up over half of the responses. The table below shows a summary of the responses. To read the table, look at one row, which corresponds to one level from the 2012-13 season. The columns show which levels the players will be next season. For example, focusing on the A players from 2012-13: of 34 total responses, 3 are moving to Tier 1 programs next season (AAA Major or Minor), 12 are moving to AA, 12 are staying in A, etc.





Other Characteristics


There were some interesting results that emerged from the background/introductory questions that might be of interest to those running youth hockey programs.

For example, looking at the average distance that families traveled to their home rink during the 2012-13 season we see that 71% of them traveled less than 30 minutes. At the Tier 2 level, that jumps to 76% (it drops to 55% at the Tier 1 level, which should be expected since their are fewer Tier 1 teams so more travel may be required). This shows that although there is no requirement to play for a particular team based on where you live, the proximity of the rink to your home is very important. 

For hockey programs, this should give a good indication of which available players are most likely to consider playing for them. It should also give a good sense of which rinks they are competing with. Outreach efforts in the schools and communities within a 30 minute radius could very well lead to an increase in registrations.





There were also some interesting results in the responses to two questions: How many children do you have playing travel hockey? and How would you describe your hockey experience level?




As you can see from the graph, the majority of hockey parents have only 1 child playing travel hockey (60%). There are a couple of ways to look at this. On the one hand, 60% of the time if you lose one player to another program you only lose one player. Maybe that indicates that retaining individual players is less important and that program managers can focus on overall retention rates instead.

On the other hand, 40% of the time that you lose one player you risk the chance of losing his or her siblings as well. In that case, losing one player can have a double or triple whammy effect. If one player has a bad experience - bad enough to consider leaving - and he or she has siblings in the program, that bad experience can become very costly.

Looking at the level of hockey experience, I was very surprised to see such a large percentage of parents who said they never played at all - not even street hockey - at 45%.



Another 27% indicated that they played street or ice hockey but not travel hockey. That leaves only 28% of travel hockey parents who actually played travel hockey themselves! For those managing hockey programs, perhaps that needs to be taken into account when thinking about the off-ice aspects of the program - communication, setting expectations, orientation for parents of new players, etc. 

For us parents, the best thing to take from this is to remember that when we're doling out sage advice to our youngsters before, after, or during games, we're doing so without ever having lived through this experience ourselves. Keeping that in perspective could very well make those post-game car rides that much less stressful.


How satisfied were you?

The first main section of the survey asked the parents how satisfied you were with a variety of different aspects of the 2012-13 season. The responses were based on a 0-10 scale, where 0 represented “Not applicable” and 10 represented “Extremely satisfied”.

These “How satisfied were you” – or HSWY – questions covered four categories: Overall experience, Coaching, Practice, and Games. There was a lot of data and I won’t cover all of it here, but I will focus on some of the more interesting results in the following subsections. A full collection of graphs of the responses to each HSWY question can be found at the end of this post.

Overall hockey experience

This section was focused on the big picture – the total experience. The questions were aimed at parents’ evaluation of the management of the rink and the hockey program, communication between them, and value for the cost of playing.

Table 1 contains a summary of the questions and responses in this section.


TABLE 1: How satisfied were you with the overall hockey experience?
Question
Average
Standard Deviation
Management of the hockey program
6.8
2.6
Communication between you and the program/rink
6.7
2.7
Commitment to skill development among all players
6.8
2.7
Consistent application of the program's mission statement
6.6
2.7
Frequency of practices
8.1
1.8
Frequency of off-ice training
6.5
2.8
Value for the cost of participating
7.0
2.5
Overall hockey experience
7.4
2.4
Average
7.0
2.5
Maximum
8.1
2.8
Minimum
6.5
1.8

Among the bright spots in this category:
  • Parents are generally happy with the frequency of practices, the overall hockey experience, and the value for the cost of playing. The responses on frequency of practices were especially positive, as seen in the graph below.
  • An average score of 7.0 in this category is not bad, especially considering that that some of those responding to the survey are moving to a new program next season.


In fact, there was a significant difference when looking at the results in this category between those who are staying with the same program next season and those who are moving to a new one. The overall average score in this section was 7.0 for all respondents, but jumps up to 7.6 for those who are staying with the same program next season. It drops to 4.7 for those who are moving to a new program (See Table 2).

TABLE 2: How satisfied were you with the overall hockey experience? Staying vs. Moving
Question
Average: Moving
Average: Staying
Difference
Management of the hockey program
4.0
7.6
-3.6
Communication between you and the program/rink
4.2
7.3
-3.1
Commitment to skill development among all players
4.1
7.5
-3.4
Consistent application of the program's mission statement
3.7
7.3
-3.6
Frequency of practices
7.3
8.3
-1.0
Frequency of off-ice training
4.9
7.1
-2.2
Value for the cost of participating
4.9
7.6
-2.7
Overall hockey experience
4.7
8.1
-3.4
Average
4.7
7.6
-2.9
Maximum
7.3
8.3
-3.6
Minimum
3.7
7.1
-1.0

Looking at the individual questions, there is generally about a 3 point drop in the average score between those who are staying and those who are leaving. The one exception to this is the question on frequency of practices. Parents seem to be happy with practices regardless of how they feel about the rest of the experience.

The sharp drops in the scores for
Management of the hockey program and Overall hockey experience are notable. I am a little wary of the drop in Consistent application of the program’s mission statement – simply because I am doubtful of how many parents actually know their program’s mission statement! I know I don’t know mine.


Here's a graph of the same data:



I’ll add a final note in this section. The Frequency of off-ice training question received a relatively high number of “not applicable” responses. I can say with confidence that this is highly driven by the younger age groups that don’t have organized off-ice practices yet. I was hoping that the parents in such cases would respond with something other than "not applicable" because I wanted them to indicate whether or not they were satisfied with not having off-ice training, but I can understand why so many chose 0. It’s worth pointing out, however, that these responses were not included when calculating the statistics used in this analysis1.


1 Including the zeroes in the calculation of average scores would artificially lower the average scores since 0 indicates “not applicable,” as opposed to dissatisfaction.


Coaching

The coaching section is pretty straightforward. The questions were focused on the parents’ perception of the coaches’ management of practices and games and overall demeanor. The goal was to see if there’s any connection between satisfaction with coaches and players switching programs.

As you can see in Table 3, overall there was a high degree of satisfaction with the coaching last season. The overall average was 7.4, with a maximum score of 8.0 for Fair distribution of ice time during games and a low score of 5.9 for Effectiveness/efficient use of off-ice training time. Coaches were also seen as having positive interaction with the players and being committed to a fun hockey experience.

TABLE 3: How satisfied were you with the coaching?
Question
Average
Standard Deviation
Quality of coaching/instruction at practices
7.2
2.7
Effectiveness/efficient use of practice time
7.2
2.7
Effectiveness/efficient use of off-ice training time
5.9
2.9
Fair distribution of ice time during games
8.0
2.4
Quality of coaching/instruction during games
7.6
2.6
Bench management during games
7.5
2.6
Discipline and player/parent management
7.1
2.7
Demeanor/interaction with the players
7.9
2.3
Commitment to a fun hockey experience
7.9
2.4
Overall coaching quality/effectiveness
7.4
2.6
Average
7.4
2.6
Maximum
8.0
2.9
Minimum
5.9
2.3

The same point I made in the last section about a high number of “not applicable” responses to the off-ice training question applies in this section. This is due to the younger age groups that don’t have off-ice workouts yet. It is notable however, that even excluding the "not applicable" responses, parents are less satisfied with coaches’ use of off-ice training time relative to the other aspects of coaching. This is perhaps one area for coaches to take note and make efforts to improve.

Looking again at the differences between players who are staying with the same program next season and those who are moving to a new program, as expected there are significant differences in the levels of satisfaction with coaching. Table 4 shows the averages from these groups.


TABLE 4: How satisfied were you with the coaching? Staying vs. Moving
Question
Average: Moving
Average: Staying
Difference
Quality of coaching/instruction at practices
4.9
7.9
-3.0
Effectiveness/efficient use of practice time
5.1
7.8
-2.7
Effectiveness/efficient use of off-ice training time
3.6
6.6
-3.0
Fair distribution of ice time during games
5.8
8.6
-2.8
Quality of coaching/instruction during games
5.2
8.2
-3.0
Bench management during games
5.3
8.1
-2.8
Discipline and player/parent management
4.6
7.8
-3.2
Demeanor/interaction with the players
5.9
8.4
-2.5
Commitment to a fun hockey experience
5.5
8.5
-3.0
Overall coaching quality/effectiveness
5.1
8.1
-3.0
Average
5.1
8.0
-2.9
Maximum
5.9
8.6
-3.2
Minimum
3.6
6.6
-2.5

There is generally about a 3 point drop in the average between those who are staying and those who are leaving a given program. It’s a fairly consistent drop across all questions in this section so none of them stands out as extraordinary. The maximum drop for any particular question is 3.2 for the Discipline and player/parent management question and the smallest drop is 2.5 for Demeanor/interaction with the players



This would seem to indicate that parents who are moving to a new hockey program were generally less satisfied with the coaching they received in 2012-13 but no one facet of the coaching was to blame.

This is not to say there is a direct (or indirect) connection between level of satisfaction with coaching and parents’ decision to move to a new program. Rather, it could be that a negative experience unrelated to coaching has soured the perception of coaching effectiveness too.



Practice

The section on Practice was intended to see if the availability of ice was a key driver of parents’ decisions. The questions were focused on amount of ice time and effective use of that time, but included questions about full ice practices and specialty coaches as well. Table 5 shows the results.

TABLE 5: How satisfied were you with practices?
Question
Average
Standard Deviation
Availability/frequency of ice time
8.0
1.9
Availability of full ice practices
6.2
3.2
Practice times
7.6
2.0
Availability of specialized instructors (e.g., skating instructors, goalie coaches, etc.)
5.4
3.2
Amount of idle time for players during practice
7.0
2.6
Discipline/player management during practices
7.0
2.6
Overall practice quality/effectiveness
7.1
2.6
Average
6.9
2.6
Maximum
8.0
3.2
Minimum
5.4
1.9

Scores were generally around 7.0, and the average for all questions in this section was 6.9. Parents were especially satisfied with the availability of ice time, which is a positive sign. Parents were also happy with the practice times.

On the other hand, parents were less satisfied with the availability of full ice practices and specialized instructors.

The question on Availability of full ice practices is one where the high percentage of Valley Forge Colonials responses had a material impact on the survey results. In most (if not all) cases, two VFC teams share an ice surface for practices with each team getting a half sheet. The impact of this was reflected in the survey results. The average score from VFC parents on this question was 3.4, compared with 6.2 in aggregate and 7.3 among parents from other programs.

An interesting result emerged related to the question on Availability of specialized instructors. I expected there to be some dissatisfaction among the parents of goalies. I don’t know how many programs provide dedicated goalie coaches, but mine does not. So I expected to hear about this from goalie parents. And this held true, as the overall average of 5.4 dropped to 3.3 among parents of children who played goalie last season.

But what was surprising was that even among non-goalie parents the average was only 5.7. I take this to mean that parents would be happy with more skating instruction during practices (unless there are other types of specialized instructors that parents would like to see - let me know in the Comments!).

Looking at the differences between players who are staying with the same program next season and those who are moving to a new program (Table 6), there again seemed to be a general lower satisfaction among those who are moving. The biggest differences were observed in Discipline/player management during practices (-3.0) and Overall practice quality/effectiveness (-2.9).

TABLE 6: How satisfied were you with practices? Staying vs. Moving
Question
Average: Moving
Average: Staying
Difference
Availability/frequency of ice time
6.5
8.4
-1.9
Availability of full ice practices
5.3
6.5
-1.2
Practice times
6.3
7.9
-1.6
Availability of specialized instructors (e.g., skating instructors, goalie coaches, etc.)
3.2
5.9
-2.7
Amount of idle time for players during practice
5.0
7.6
-2.6
Discipline/player management during practices
4.7
7.7
-3.0
Overall practice quality/effectiveness
4.9
7.8
-2.9
Average
5.1
7.4
-2.3
Maximum
6.5
8.4
-3.0
Minimum
3.2
5.9
-1.2


Games

The final section of the "How satisfied were you?" questions was focused on Games - volume, travel, and quality of competition. There are a few things these questions might indicate - are players being placed on teams appropriately, are parents overwhelmed by travel demands, is there an appropriate balance between winning and having fun? Table 7 shows the results of the survey.


TABLE 7: How satisfied were you with the games?
Question
Average
Standard Deviation
Amount and frequency of games
8.1
1.8
Amount of travel to/from games
7.7
1.8
Frequency of tournaments
7.5
2.2
Quality of competition was appropriate
7.4
2.1
Balance between winning and fun
7.6
2.2
Overall game experience
7.8
2.0
Average
7.7
2.0
Maximum
8.1
2.2
Minimum
7.4
1.8

As you can see, the survey showed that parents are pretty satisfied with all of those things! This section had the highest overall average score at nearly 8.0 (7.7) and the lowest standard deviation, which indicates the responses were fairly consistent. 

Comparing the responses between those who are staying with the same program next season and those who are moving to a new program (Table 8), there is a modest difference between the two. The Frequency of tournaments, Balance between winning and fun, and Overall game experience responses each showed a difference of around 2 points.  This section has the smallest difference on average between those who are moving and those who are staying - an average drop of only 1.6.


TABLE 8: How satisfied were you with the games? Staying vs. Moving
Question
Average: Moving
Average: Staying
Difference
Amount and frequency of games
7.1
8.4
-1.3
Amount of travel to/from games
6.9
8.0
-1.1
Frequency of tournaments
6.0
8.0
-2.0
Quality of competition was appropriate
6.8
7.6
-0.8
Balance between winning and fun
6.0
8.1
-2.1
Overall game experience
6.0
8.2
-2.2
Average
6.5
8.1
-1.6
Maximum
7.1
8.4
-2.2
Minimum
6.0
7.6
-0.8

Here's the graph:



Based on these results alone, it would appear that the game experience is not a significant driver of movement among programs. There is perhaps an opportunity to play more tournaments - although the number of tournaments a team plays is really only limited by how many the parents are willing to pay for. The differences in Balance between winning and fun and Overall game experience responses are tough to gauge, and they are closely related. If a player isn't having fun, that doesn't make for a very pleasant game experience for the parent. And as I mentioned in the section on Coaching, it could be that something not specifically game related has happened that has contaminated the game experience too. Sometimes a bad experience can snowball into a bad season.


Summary

I saw one of my hockey parent friends at the rink recently and we started talking about the survey. He asked if any surprises had emerged while I was reviewing the results. I told him that one thing that had surprised me is that overall, parents are pretty happy with their hockey experience! With all of the chatter and hand-wringing that goes on near the end of the season (sometimes well before then) about who's moving where, you would think the HSWY scores would be lower.

Now, that's not to say it can't be improved. Scores in the 7.0 - 7.5 range aren't anything to celebrate. There is room for improvement in general and especially in certain specific areas, such as better utilizing off-ice training time and providing more specialized instructors.

The key to focus on is the difference between the parents who stay with a program and those who plan to leave. The parents who stay have scores in the mid-7-to-low-8 range. The parents who leave have scores that are generally 2-3 points lower.

With these HSWY questions it's clear there's a correlation between level of satisfaction and the likelihood that someone will jump ship. But that doesn't tell the full story of what caused the dissatisfaction in the first place. In the next section of the survey I asked that question directly to find out the top three reasons parents move their player to a new program.

I'll tackle that section and the summary of parents' priorities in a future post. In the meantime, here's a complete view of the HSWY survey results in graphical form. Click on the slideshow for a larger photo.


** UPDATED: The slideshow doesn't seem to be working on some mobile devices. Click here for the graphs.

Thanks for checking out the first section of survey results. I'd love to hear your comments - and don't forget to check back soon for the rest.

No comments:

Post a Comment