In order for us all to build the right digital products, we must first make sure we understand current Scout volunteering by conducting research. This post is about how we are using research to build based on your needs, how members were involved in what we have done so far, and how you can be involved yourself in the future. I first presented this at Digital Day, so if you are interested in what happened, or if you want to see and share what you saw that day, you will find that here.
In order to share the entire Digital Day presentation, this is a long post. Here are some shortcuts:
With such a large Scouting membership, how do we listen to everyone? How do we get knowledge across the whole membership?
These are the 3 basic ways we do research
|Observe current behaviour||Analyse our existing information||Ask about your experiences|
|This is what I will be focusing on in this post||We gather this information every time you use the website, and through analysis of the data and knowledge that we have||This includes the survey, which some of you might have filled out, but we are also experimenting with new tools to gather your input|
These are all sources for evidence. Evidence is the actionable information that comes from careful research. We use the resulting insights to create useful products and services. These three types of research support (or ‘validate’) each other, and make our knowledge stronger by helping us come to a more thorough understanding.
As a note, we will always protect your data and your privacy. That’s why you won’t see any videos today, or any photos, or any names (even of groups) from the research, even though this is all based on volunteer input.
What user research helps us accomplish that we can’t do otherwise
- Research is a rigorous, disciplined, and systematic way of understanding issues in order to create solutions for them
- It takes inputs – anecdotes, opinions, suggestions, complaints– and discovers how prevalent and critical they are– looking at behavior patterns in particular
- We can discover the commonalties in the rich variety of experience that is Scouting (not every insight from evidence will reflect every experience) — the differences are also important, as we will go into
- Research enables us to prioritise development, or further investigation
Our research to date: Why we did it
- We, the Digital Team, didn’t have the knowledge we needed to develop the right digital products for you
- We needed research to help us understand the behavior of how Scouting is delivered now
- We also wanted to know more about the variety and richness of Scouting
- Independent research allowed us to reach out across the Movement and listen to individuals that represent the diversity of Scouting – including people who don’t feel connected, newbies, quiet people, noisy people, and old-hats
- Research also helped us establish knowledge to reduce reliance on anecdotal assumptions
We focused first on Section Leaders and Assistant Section Leaders, to address the largest segment of volunteers with the most contact with young people. But this research is just the beginning. Research is important for understanding all roles, and this method applies for future as well. Research with Section Leaders and Assistant Section Leaders will also continue.
Our research to date: How we did it
- We hired an independent research agency, to help establish an objective baseline
- Interviews and video journals to gather in-depth insights with Section Leaders and Assistant Section Leaders in England, Scotland, Northern Ireland, and Wales
- Followed by a survey for all SLs and ASLs to provide input, in order to validate the insights from the interviews
- Analysis of the existing website usage (and improvement in how we gather data)
In addition, the digital team are meeting with sections across the nations to increase understanding.
We don’t know everything, but we know more than before
- We will continue growing in our knowledge
- Our insights will change over time
- We will learn what we need to know at the right time
3 examples of what we have learned
We chose these 3 insights to show you similarities and differences in the ways people deliver scouting. There is more detail, of course, that we will use in design.
We hope these will give you a new perspective and see the differences and similarities in how we deliver scouting.
1. We learned about how flexibility is important for the delivery of Scouting
Each leader has their own way of balancing fun and awards
The research has showed us there are two main drivers in how our leaders organise their programme. This a spectrum, and you might identify yourself somewhere in between them.
Tend to be
Spend a lot of time getting informed about badge criteria and changes
Tend to be
Spend more time getting new ideas for making scouting fun
Both leader types measure their success by young people’s enjoyment
- Both badge-focused and fun-focused want new ideas to make scouting fun
- Both want to keep on top of the badge requirements.
- Some previously badge-focused now see badges as a way to organise fun, rather than the end goal in itself
- Badges are a structure and bonus
- Keeping it fresh, different and unique is important for all leaders regardless of experience
2. We learned what leaders’ common behaviours are
Leaders plan out their year
New ideas come from all directions: other leaders, young people, Facebook, adult helpers, Google.
Fun, individualised programmes result
Leaders are balancing badges and fun here during the programme planning, and throughout the year.
We will be sharing more about planning in future posts.
3. We learned about how section dynamics lead to widely different volunteer experiences
We are going to take you through some behavioural archetypes that the research uncovered, and how these impact how sections run. We will look at 2 types of Section Leader behaviour and 2 types of Assistant Section Leader behaviour. These are models of behaviour, not personality, so they are changeable and not always self-determined.
The Overseer Section Leader has people power and a director mindset
Parents who were scouts, former scouts
- Has overall responsibility but shares decision making
- Delegates the day to day implementation with others
- Good support at section level from ASLs and parents
- Without support, Overseers lose enthusiasm, become stuck in old ways, or just wing it
The Chief Section Leader is doing a lot of the work
Former scouts, Parents who were scouts, Parent-Outdoor hobbyist
- This leader takes on the majority of the work & responsibility, and it’s too much, risking burnout
- Needs more support from adult volunteers at section level
- But Chiefs only delegate out minor tasks
- Feels others cannot be relied on
- Uses lots of tech and tools to make admin easier
- eg OSM, Whatsapp, Facebook
The Deputy Assistant Section Leader creates a role which plays to their strengths
Parents, Former scouts
- Ready to be developed
- Has particular responsibilities and specific, unique skills to offer
- But may feel they do not have all the skills they feel is necessary to lead
- Risk of getting stuck, feeling limited
- Might not be in the best environment to step up
- New challenges need to be offered to them
The Helping Hand Assistant Section Leader pitches in but could be utilised more
Makes up the bulk of adult volunteer numbers
- Like to be part of it, but feel they don’t have time to do more
- Manageably-sized tasks and fun training increase their engagement
- Could be converted into more active volunteers
- Training can feel like a bit of a commitment or chore
Everyone had the same main needs when it comes to running a section
Everyone had ongoing need for
- Information, accessed quickly and easily
- Practical ‘know how’ training
- Inspiration to create their own exciting programme
- More adult help and parent engagement
Additionally, some in each archetype needed
- Connections with other Section Leaders on a local level
- Proactive and responsive support at District level
Sections with good Young Leader input were more satisfied than those without
Young Leaders, often fresh from younger sections themselves, address a knowledge gap
- Sometimes more capable and informed than the Section Leader in technical and practical knowledge
- This informal training removes the need for the SL or ASL to search for some types of information online
- Lots of good ideas and good understanding of what the children will enjoy, based on their own experiences
“Young Leaders know everything they know about pioneering, they know what cubs like, they’re full of energy “
As an integral part of our process, research will be continual and iterative
We will be looking at many sources for evidence as we build products.
What can we learn from what we already have?
Before the research started, we began a large, long-term exercise during which we collected internal Scout knowledge into a model of scouting experiences, and we were able to identify initial gaps in understanding, assumptions, and evidence from that. Continuously developing and sharing our knowledge is critical.
As mentioned at the beginning, we have improved behaviour tracking on the website, so we are able to have a better understanding of what is currently useful.
For example, on the homepage
- Homepage users are navigating to their desired section, which is primarily the Members section.
- Other content in the page gets little engagement
This is just a taste of the data — we will be working with more of this evidence in a later post.
We are also looking for new ways to work with the membership and increase the amount and types of evidence we gather
There are more sources of your input that we use
- Info Centre phone and email queries
- Requests for help on Facebook
- Comments on other forums
(but these aren’t quantified and need to be validated)
We will conduct more research, as needed
We will continue to conduct research as we develop new products, and measure the impact of our work on your experiences.
- Longitudinal research with the same participants over time, measuring change and cycles of behaviour
- Surveys to understand prevalence of need and behaviour, or to get fast feedback
- Usability testing to see if our designs work right
After launch, we continue to learn
- Launch small and grow bigger, focusing on what is working for you
- Make continuous improvements based on how you use what we make
- Fix things that aren’t as usable as they could be
- Continue to release new evidence-based work, supported by continuous releases from development team
- Conduct further targeted research, if necessary, to support new development
We won’t have all the answers straight away, but we will always be making it better
All of this requires your input — Stay involved!
- Analytics (just use the site)
- A/B Testing — which variation of a page performs better for you?
- User Panels — commit to regular collaboration
- Usability Testing — from sketches to pre-production
- And we are always looking for new things that will enable the breadth of the membership to engage
Contact us and let us know if you would like to be involved in developing and testing with us. We have some really exciting work for you coming up and we need your help to make it better. You don’t need to have technical expertise of any kind to be involved (though it is helpful if you can give clear, constructive feedback)
My next post will tell you how we are putting this into practice for our first releases, and about more collaborative practices we are exploring.