Learning Leaps Part 5: Incorporating learning strategy while building products

Welcome to another edition of Learning Leaps, where I’ll be sharing lessons learned from my first 16 months as a product manager at Pluralsight. 

Since transitioning into Product Management, one of the biggest lessons I’ve had to learn was about the difference between learning strategy and product strategy. Thats why for this final edition of Learning Leaps, I’ll be taking a deeper look into the need for learning strategy to be incorporated in technology products.

Whats in a strategy anyway?

Many organizations, especially those in the technology space, focus on the importance of product strategies to help drive their decision making. For those unfamiliar, a product strategy can be defined as a set of decisions and priorities that a company focuses on in order to achieve a shared vision that wins with the users of its products (thanks to my good friend Jess Kadar for this concise definition!).

During my time in the industry, I’ve sat through my fair share of product strategy sessions. These meetings would usually include some sort of discussion about the company mission and vision, OKRs, and even product roadmaps. These sessions were great exercises for the company to illustrate why they were focusing on specific areas of investment. The outcome of the meetings would often empower product teams to drive the specific areas that they oversaw. Despite this, the more I sat in on these discussions, the more I noticed learning strategy not being considered during the creation of technology products (even when creating learning products).

For those unfamiliar, learning strategy can be defined as a set of decisions, techniques, procedures, and processes that learning creators can use to promote desired learning outcomes. As a learning strategist, I’m often thinking about the learning outcomes I’m looking to drive and the best way to deliver them. In other words, I consider things like learner goals and objectives so that I can determine the types of content or activities to help influence a learners knowledge, skills, and behavior.

Learning and product strategy are different yet intertwined

Over the years, the general theme I noticed in the industry is that product strategy and design are often driven by market and customer needs while learning strategy is driven by theory and research (Note: this is an oversimplification and I’m happy to dive deeper with anyone who may be discussing more of the intricacies).

Theres a few problems with this approach. First, as Clark Quinn mentions in Where is the Learning Science in Technology Products?, there is a documented disconnect between what learners think is good for their learning and what actually works. This is the same way that customers will ask for things that they think they want but it still may not get to the root cause of what they actually need.

Second, this approach leads me to think about the ancient qualitative vs. quantitative research debate that happens in product and UX communities. It should not be one or the other but rather a blending of the two or a mixed-methods approach.

What i’m recommending is that learning strategy should be considered an equally as important apart of a business as product strategy. If you look into the market, the best learning and educational products have a solid foundation of learning strategy. To put it succinctly, if you’re building a product to help people learn, you have to know how they learn.

Balancing customer needs with learning science

So whats the best way to balance your customer needs with learning strategy? Heres a few tips to get you started:

Connect with people who have learning expertise

If you don’t have a background in learning, thats okay! You can connect with folks who already have some inside of your organization. These individuals might be sitting on your learning and development, UX, or even product teams. They’ll be familiar with learning theories and models that could help to influence the success of your learning product. Use them!

Incorporate learning research while making product decisions

As a PM with a background in learning, I often lead and conduct my own learning research alongside the product discovery process. This often means leading mixed method research which include surveys, conducting user interviews, or prototype testing. At the same time if I have a specific research question in mind, I’ll look at existing learning research to see what it was says. I’ll incorporate this research into any synthesis and consider it when making any product decisions with my team.

Build out a center of learning research

Depending upon the size of your learning product and organization, it might be worth investing in a center of learning research. Many of the larger learning companies in the industry such as Pearson, edX, and Houghton Mifflin Harcourt have done just this. These centers often focus on staying up to date with the latest learning research and techniques in addition to conducting efficacy studies among customers to prove that learning outcomes are indeed happening.

Do you have any tips on how technology platforms can incorporate more learning science into the experiences they’re building? Post them in the comments below!

Learning Leaps Part 3: Using data to make informed training decisions

Welcome to another edition of Learning Leaps, where I’ll be sharing lessons learned from my first 16 months as a product manager at Pluralsight. Since transitioning into Product Management, one of the biggest lessons I’ve had to learn was about how to use data to make decisions related to learning products. Thats why for this week’s Learning Leaps, I’ll be taking a deeper look into how data can be used to make informed training decisions.

Data in learning (a curious past)

After spending the past 9 years designing and delivering learning experiences, I was no stranger to using data to improve learning. When I first started out in the industry, this data came in the form of “smile sheets” or feedback forms that was presented to learners at the end of a training program. This form would asks learners about their perception and reaction to the learning program that they just attended.

In general, these sheets are a great way to let learners know that we respected their opinions and wanted to know about their overall satisfaction with the learning experience. As my time in the industry went on, I found myself getting more and more frustrated with these forms because satisfaction with learning does not equal learning.

Over time, some of the companies I worked for moved beyond the simple collection of satisfaction scores and measured things like assessment scores or behavioral change. But overall, this was few and far between.

At the same time, there was an increased growth in technology platforms entering the learning industry. These platforms housed digital learning experiences and began to provide practitioners with more insights into how audiences were interacting with their experiences. It was an exciting time to be in learning!

My relationship with learning data underwent another change when I moved into learning experience design, and more recently my PM role at Pluralsight. So I decided to put together a few tips that I’ve learned over the past year that I hope will inspire others while they’re thinking about how data can help inform their learning interventions.

Tips on how to use data to make informed training decisions

Learning interventions are experiments

First and foremost, at their very core, learning interventions are experiments.

Whether it’s instructor led training, elearning, or another solution; a learning intervention is a proposed solution to help solve a performance problem inside of an organization. Ideally, it’s been proposed because there is some type of current behavior taking place that is not aligning with the desired behavior of the organization. After some type of analysis, a practitioner has determined that learning might be a potential solution to help encourage the outcomes the organization is looking for.

As learning practitioners, this means we cannot guarantee that learning will indeed be successful to deliver those outcomes. However, if we take an experimental approach to designing learning experiences we may be able to increase the likelihood that it might succeed. Taking an experimental approach means taking the time to identify the outcomes you’re hoping to implement, clarifying our hypothesis and assumptions, and determining how you’ll measure success. If you don’t determine these things from the outset, you will never be able to identify whether you’ve been successful or not.

Measure enough data to make a decision

Once you have a better idea of the outcomes you’re looking to drive, you’ll then be able to use data to help guide any decisions related to your interventions.

You can use data while you’re in discovery or identifying the problems you’re looking to solve for customers. Typically this type of data might include qualitative research like conducting surveys or observations.

You can also collect evaluative data after your solution has been released into the wild to help you assess how well it’s solving the audiences problem. Based on your overall goals, these could mean measuring things like drop-off rate, learning impact, retention rates, and more.

Whatever your reasons for measurement are, it’s important to remember that the goal of measurement in learning is to gather enough data to make a decision. It is not to collect ALL of the data you possibly can to be absolutely certain of something. If you’ve gathered enough data to make a decision and you keep collecting data, then you’ve gone too far.

Storytelling with data is a skill

Once you’ve collected some type of data is where the fun part comes in. Storytelling! Take the time to think about the story you’re looking to tell to your stakeholders. Ask yourself things like:

  • What were my expectations going into this?
  • Am I surprised by what the data is telling me?
  • What do my stakeholders care about?
  • What decisions am I trying to influence?
  • Whats the best way to convey this to others?

Have this information feed into the way that you craft your story and any decisions you make based off of the data you’ve collected. Remember that anyone can collect metrics and simply report them but it takes skills to turn that data into a story that truly resonates with others.

Do you have any tips for others on how to use data to make informed training decisions? Post them in the comments below!

Be sure to check out next week’s Learning Leaps where I’ll be diving into how to use empathy to uncover the needs of your learners.

Learning Leaps Part 2: What collaboration looks like when creating learning products

Welcome to another edition of Learning Leaps, where I’ll be sharing lessons learned from my first 16 months as a product manager at Pluralsight. While transitioning into a Product Manager role, one of the biggest lessons I’ve had to learn was about facilitating effective collaboration while creating learning experiences. That’s why for this weeks Learning Leaps, I’ll be taking a deeper look to identify what collaboration looks like and provide some tips to help get you started!

Moving from IC to Product Manager

After spending the past 8 years designing and delivering learning experiences, I was no stranger to collaboration on the job. In all of my previous roles, I was always an individual contributor on a cross-functional team. I had enjoyed this type of role and had done well with my approach to collaboration.

Soon after transitioning into my PM role at Pluralsight, I had a huge wakeup call that my existing approach to collaboration needed to change and FAST!

Product Management is collaboration. As a Product Manager, you are driving the product you’re responsible for. This means that you’re frequently coordinating and collaborating with all of the stakeholders who touch your product. So rather than being an individual contributor on a cross-functional team, you’re the one driving the initiatives and overall decision making related to your product line.

This type of collaboration is a skill that is learned and perfected by many Product Managers over time. So I decided to put together a few tricks that I’ve learned over the past year that I hope will help others while they’re collaborating on learning experiences.

Tips for effective collaboration when creating learning experiences and products

Identify and build trust with your stakeholders

A week before joining the team at Pluralsight, I fell down the stairs and broke my ankle. This put a huge damper on the onboarding plan I had created for myself. Since I am the absolute epitome of an introvert, I knew I was going to have to make an extra effort to meet everyone who I would be working with.

Over my first month on the job, I ended up having 30+ virtual 1:1 sessions. I did them in order of priority; starting with my immediate team including developers and UX designer. I put together questions that would help me learn more about their experiences inside of the company and in their roles. By taking the time to learn more about my team and stakeholders, I was able to gain empathy about the challenges they had to overcome on a daily basis. This gave me insight on things I could do to help make their lives easier in our work together.

After the official meet and greets were done, I made sure to put reoccurring meetings with stakeholders on the calendar so I’d never have to think twice about who to talk to and when. To this day, I’m still discovering people that would be great to connect with or touch the products and initiatives i’m working on.

Define the outcomes you’re looking to drive

Once you have a better idea of who you’ll be working with, you’ll want to identify the outcomes you’re looking to drive in your work together. Whenever I kick off a new project or initiative, I’ll usually schedule a meeting for everyone to come together and chat about the outcomes we’re hoping for and why. This alignment meeting makes sure that everyone starts out with the same context and helps us to be more effective in our work together.

Define roles for everyone on the team

When working on a cross functional team, it’s important to remember that everyone brings unique strengths and perspectives to the table. It is through your work together you’re able to deliver better outcomes than one person would be able to individually. Collaboration works best if everyone has an idea of what is being expected of them. Whoever the project leader is (in my case since I was the PM, it was me), will want to make sure everyones roles are clear from the very beginning so there is no confusion as you deliver on your mission.

Communicate early and often

Once you have your stakeholder group figured out, you’ll want to figure out the best way to work together. Since Pluralsight has multiple offices around the world, much of our work is done asynchronously. Depending upon the size of the project I’m working on, i’ll often spin up a slack channel for everyone to communicate and share insights with one another.

I’ll try to limit scheduled meetings for major project milestones like brainstorming, sharing user research synthesis, or discussing priorities for a coming year or quarter. As a Product Manager, I’m usually deep in the weeds of the problems I’m involved with, while others on the team may jump in and out as their schedule allows. Because of this, i’ll also try to over-communicate as much as possible to ensure others can follow along with things as they’re unfolding.

Show your work!

Prior to joining Pluralsight, I had always worked inside companies that were smaller in size. For context, when I joined The Predictive Index, the company had around 30 employees and when I left it had reached 100. Pluralsight on the other hand, is around 1400+ people currently. It goes without being said, collaborating inside of a 100 person company is drastically different than collaborating inside of a 1400 person company.

At a smaller size company, collaboration meant coordinating with maybe 8-10 stakeholders. During a recent project I was working on I had over 30+ stakeholders I had to coordinate with.

While chatting with my teammate Patrick, he noted how important showing your work was inside of larger organizations. He compared it to showing your work during math class. I vividly remember my math teacher trying to drill into my head the need to document my work as I progressed through problems (as a child I hated this activity).

I use a similar approach today for my projects inside of Pluralsight. I use project documents to highlight the outcomes were looking to drive, hypothesis we have, links to designs, experiment ideas, and decisions about future product strategy. This approach allows others to follow along with decisions and how they were made. As much as I despise forcing myself to slow down and document my work in this way, I’ve found that it’s made me more strategic about what i’m communicating and why. It’s also made it easier for me to get buy-in, influence, and manage others.

Seek context with intention

One of the core values that Pluralsight encourages amongst all of it’s employees is to seek context with intention. I’m a huge believer that as children we were driven by our curiosity and as we entered adulthood it was beaten out of us little by little. Thats why this value is my absolute favorite and I utilize it all of the time when working with others.

When working teams, it is not uncommon for everyone to have different viewpoints because of the vantage point they have in their role. I’ll frequently question why someone has a particular opinion, ask why they did something a certain way, what their thought process were and why. By asking questions, i’m able to learn more about the constraints and possibilities of a project.

Do you have any tips for others on how to collaborate more effectively? Post them in the comments below!

The next Learning Leaps will resume in 2020, where we’ll be dialing in on how to use data to make informed training decisions.

Lessons Learned from DevLearn 2019

Last week I attended The eLearning Guild’s DevLearn conference in Las Vegas, NV. It was my second time attending the event (my previous visit was in 2016). For those who haven’t attended a DevLearn conference before, it is a 3 day event where practitioners in the industry gather to discuss industry trends, best practices, and tips and tricks. On top of all of that, the guild also offers 2 days of pre-conference workshops for those looking to expand their skills even more.

Overall, I’m a big fan of the guild events. They’re actually my favorite in the industry to attend. It’s a great opportunity to connect with others, see what their working on, and share stories. I always come back with key nuggets that I cant wait to share with my team. This trip was no exception, below are a few highlights from the trip:

I LOVE my learning network!

First and foremost it must be said. I love my learning network! At this years DevLearn, I was able to meet some amazing people that I’ve been chatting with online for years now (like Tim Slade, Cara North, and Nick Floro).

Spending time with Matthew Pierce & Cara North at Demo Fest

I spent time with some of my former teammates at The Predictive Index. I also met others who are creating learning experiences for industries completely different than mine, such as emergency response and law. It is an amazing experience when you’re connect with others who share the same passion as you. You’re able to learn from their each others experiences, discuss differences, and challenges. It just goes to show how much of a common thread learning and education truly is.

Industry Trends

Overall, I attended about 15+ sessions over a span of 3 days and noticed some trends occurring in the industry:

AI is coming and as learning professionals we need to adapt.

There was a-lot of talk about whether AI is going to take over the future of work or not. This was definitely highlighted by the fact that one of the main keynoters was Sophia, the Robot. The key takeaway from these discussions is that AI will absolutely transform the way we do our work. It has the potential to automate many of the manual processes we do in our work , like capturing screenshots, creating step by step instructions for job-aids, helping write assessment questions, and curating learning content. As practitioners, this will leave us with time to do more of the creative work we love – YAY!

The rise of Learning Data is here!

With the rise of xAPI over the past few years, many in the industry are beginning to think more critically of their learning data. In total, there were over 13 sessions focused solely on data and measurement! I actually attended a pre-conference workshop with Sam Rogers of SnapSynapse about How to Make Better Training Decisions with Your Learning Data.

One of the major takeaways I got from Sam’s session is that in order to truly track the impact of our learning interventions, we need to take time from the outset to identify the outcomes and behaviors were looking to change. If we don’t know this, how will we know if were successful?

Additionally, one major area is the collection of data but what happens next? This is where the beauty of storytelling comes in. As practitioners, we need to think about the what our stakeholders care about, what decisions are we trying to influence with our data, and what is the best way to convey this to them?

There is a difference between learning strategy and product strategy

By far, the biggest takeaway for me came during Frank Nguyen’s guided panel discussion on Transforming from Learning Professional to Learning Leader. Frank and the panel highlighted the importance that as learning leaders we need to force others to think about the instructional strategy rather than immediately jumping to solutioning. This means identifying the true performance problems taking place, advocating for the learners and their needs, and determining an instructional strategy and experiences that support that. Learning is not simply defined by one up learning events but rather an entire ecosystem and all of their parts working together.

Overall, DevLearn was such a great experience. I’m so grateful to meet many of my friends in person. I can also say, i’m really happy to be home in my introvert cave with my cats. I look forward to seeing everyone at Learning Solutions in March 2020!

Lessons Learned from FocusOn17

Two weeks ago, I was fortunate enough to go to The eLearning Guild’s FocusOn 2017 Conference in San Diego, CA. It was my third conference with the Guild and once again I was able to I walk away with some great insights! Each year the FocusOn conference centers around 3 technologies within the learning industry. This year the focuses were mobile, games, and video.

As always, many of my best conversations came from speaking with others in the field. The eLearning Guild conferences provide a great opportunity for practitioners to get together to share their expertise and lessons learned.

I attended a number of sessions about up and coming technologies within the industry. Many of these sessions focused on incorporating the use of virtual reality, personalized learning, and curated content. One thing that jumped out to me with the rise of personalized and curated content is the importance of content management systems. Often times, learning organizations are producing mounds of content and in order to provide better recommendations for our learners we need to make sure that the content we are creating and pushing out is appropriately findable, keyed, and tagged.

I also attended a number of sessions about gamification and scenario based learning. These sessions seemed like a great reinforcement to much of the content that I am learning in my coursework this semester. I was able to see some real world examples of branched scenarios and interactive videos.

Finally, this trip was very eye opening for me personally! I am a little over 1/3 of the way through my masters program with Boise State, while simultaneously working full time with The Predictive Index. This means that I am often heads down with work and classes. This trip allowed me to reach my head above water and see how far I’ve come within the industry. My masters program has allowed me to speak intelligently about theories and concepts, while my full time position allows me to begin applying new lessons learned immediately.

Overall, it was a great conference and I cannot wait to begin applying what I’ve learned within my organization!