Program Report

What is a Program Report?

The Program Report offers an overview of a pre-defined learning program—like compliance training or new hire training. Watch as your learners progress through courses, videos, assessments and keep up with high-level metrics like average assessment scores and competencies demonstrated. Answer common questions, such as who has or hasn’t completed parts of a learning program and share it with managers and leadership.

Who can use this feature?
 User Types
Any user with access to Explore (Global Admins, Regional Admins, and some Users) can edit a Program report. All users can view a Program report.
 Pricing 
Available on paid plans (AnalystCLO, and Enterprise).
 Expertise
This feature is recommended for intermediate and expert users.

Concepts

The program report gives you insights into all aspects of your learning program from many different angles. This section outlines some key concepts that the Program Report is based on to help you navigate.

Program A complete learning program involving multiple activities, learning paths, assessments, competencies and contributing systems.
Learning Path A path is a series of high level sequential milestones that learners are expected to pass through over the course of the program. These should be key milestones that all learners will complete and that are normally completed in a fixed order. There is no technical limit to the number of milestones included in a path but we recommend no more than six.
Workflow A series of tasks designed to be completed in an ideal order, but which can be completed (or partially completed) in a different order. Workflow tasks are generally much more granular than a learning path, for example a workflow might be the steps taken by a call centre agent to help a customer with a particular problem. Workflow reporting looks at how many people followed the ideal workflow and surfaces other common workflows.
Assessment Assessments are quizzes, tests, observations and other interactions that assess learners and can be presented in an Activity Report. You might not want to include every assessment in your Program dashboard, just the most significant ones that you want to monitor.
Competency Competencies are skills, knowledge and abilities evidenced by learners during the course of a program. Competencies are tied to a tracked activity or activities that demonstrate competence. It’s also possible for activities that report a score to indicate a % proficiency in the competency.
Contributing System A contributing system is a tool, product or platform that is used by learners on the program. For example an LMS, a performance observation checklist and a mobile app might be three contributing systems of a program.

Using Simple Configuration to Create a Program Report

  1. Click Explore in the main navigation and choose Program from the report-type carousel:image6.png
  2. You’ll use Explore to customize the report and then click Saves As New Report to create a new report with the options you’ve chosen.image5.png

Using Explore to Customize Program Reports

When using Explore to customize Program Reports, you’ll see 8 options:

Click on any of the option names to look at the settings for that option:

image1.png

Activities

image12.png

After clicking Activities, you’ll be able to choose which activities you want to include in your Program Report. You can choose as many different activities as you’d like, and the Program Report will display information related to the chosen activities. If you don’t select any activities, the Program Report will include all activities in your account that you have permission to see.

To select an activity, use the search bar to find an activity or click the Pick From a List button to choose an activity.

Please note: You’ll only see activities that are already in your account that you have permission to see.

If you want to deselect an activity you’ve already chosen, click the X button next to the activity’s name.

People

image4.png

After clicking People, choose which people to include in your Program Report. You’ll be able to select multiple people and parts of your organization, and those people will be included in the Program. If you don’t choose anyone, this report will include everyone that you have permission to see.

To select people, use the search bar to find specific people or group names. You can also click the pick from a list button to choose specific people or groups.

Please note: You’ll only be able to choose from people that you have permission to access.

If you want to deselect a person or group you’ve already chosen, click the X button next to the person or group’s name.

You can also use the People Filter to select Group Types. If you do, only these types of group will be shown in the groups table. 

program-groups.png

Dates

image9.png

After clicking Dates, you can select a time range for your report to analyze. Click on the dropdown to select Trailing or Custom.

  • Use Trailing to select the previous X days/weeks/months/years from today. While the range stays the same, the dates change to keep up with the current date.
  • Use the Custom option to choose a specific date range.

If you don’t select any dates, the report will analyze all data in the account that you have permission to see.

Card Text

image8.png

From the Card Text menu, customize the text on the report:

  • The Title is at the top of the report on both the Dashboard and the report’s detail page.
  • The Summary appears at the footer of the report on the Dashboard.
  • The Description appears at the top of the report’s detail page under the header.

Program

image11.pngWithin the Program menu, you can add new steps (milestones) to your Program. Each step can be a Simple Step or an Advanced Step. A Simple Step consists of only one activity, while an Advanced Step is composed of multiple activities.

  • Simple Step Selection
    • After clicking Add Simple Step, you can search for a single Activity the same way you would in the above Activities section. You also have the option to apply a name to that activity which will appear on the Program Report's milestone tracker.
  • Advanced Step Selection
    • Adding an Advanced Step allows for two differences from a Simple Step. First, you can select from multiple possible activities instead of just one. Secondly, you can identify whether or not success is required, i.e. does the activity need to be passed, or just completed?
  • The option to select whether steps must be completed in order is available as well. When Steps should be completed in order is selected, the Program milestones chart will always sum to 100%. When Steps can be completed in any order is selected, the Program Milestone chart will show a distinct value between 0 and 100% for each step.

Competencies

image14.pngThe competency menu provides the opportunity to map certain activities in a learning program to a specific competency framework. Note that this section is not required to be completed to configure a Program Report. Selecting Add Competency displays a familiar interface in which you can select an activity to associate with the competency.

image7.png

The Activity Name field allows you to customize the display name of the activity to be associated to the Competency. The Activity field allows you to search for the relevant activity to associate to the Competency. If the activities selected report a score result, that score will be used to calculate the competency on a 0.0 - 1.0 scale.

image10.png

After all desired activities have been associated to the competency, clicking Save will add the Competency to the Program Report.image3.png

Assessments

image2.pngWithin the Assessment menu, you can select one or more assessments by their activity ID after clicking the Add Assessment button.

image13.png

The Assessment Name field can be populated with the display name you prefer to be displayed in the Program Report. The Activity field allows you to search for an assessment within your data. If question data from the assessment is reported which specifies the selected activity as the parent activity id, that question data will be included as well.

Advanced Configuration

The activity, people and dates filters and report text of the Program report can be configured using simple configuration as they can for all reports. In addition, the Program Report can be further configured using Advanced Configuration. Each Program report’s configuration object has a program property which contains a program configuration object. The properties of that object are outlined in the table below.

Property

Explanation

Example

name

The name of the program

Widget 5000 Sales Training

primaryPathId

The id of the primary path for the program.


Currently the program report only displays data about one learning path.

widget-5000-sales

learningPaths

An array of learning path objects representing the paths associated with the program.

Currently the program report only displays data about one learning path as identified by the primary path id.

[
  {..learning path..}
]

workflows

An array of workflow objects representing the workflow associated with the program.

[
  {..workflow..},
  {..workflow..},
  {..workflow..}
]

competencies

An array of competency objects representing the competencies associated with the program.

[
  {..competency..},
  {..competency..},
  {..competency..},
  {..competency..},
  {..competency..},
  {..competency..}
]

dataOrigins

An array of data source objects representing the data sources associated with the program.

[
  {..data source..},
  {..data source..},
  {..data source..}
]

assessments

An array of assessment objects representing the assessments, tests and quizzes associated with the program.

[
  {..assessment..},
  {..assessment..},
  {..assessment..},
  {..assessment..},
  {..assessment..},
]

Learning Paths - Advanced Configuration

A program includes a primary path made up of milestones. The program configuration includes an array of learning path objects as described above. Learning path objects have the following properties:

Property

Explanation

Example

type

The object type. Always ‘learningPath’.

learningPath

name

Name of the learning path to be displayed.

Widget 500 Sales Training

ordered

Determines how the milestone visualization displays.

If false, will display the percentage of people who have completed the milestone, regardless of other milestones.

If true, will display the percentage of people currently on that step.

false

id

Id of the learning path.

widget-5000-sales

steps

An array of step and step collection objects representing the milestones of the learning path.

[
  {..step..},
  {..step..},
  {..step collection..},
  {..step..},
  {..step collection..},
]

Please note: if a set of milestones is not always completed in the same order, setting the 'ordered' property to ‘true’ can distort the results and lead to negative percentages for steps that are commonly skipped or completed out of order.

Workflows - Advanced Configuration

Workflows represent a series of small steps that make up a single workflow within the Program. They are configured in a similar way to paths, except that there can be any number of workflows but only one primary path.

Workflow configuration objects have the following properties:

Property

Explanation

Example

type

The object type. Always ‘workflow’.

workflow

name

Name of the workflow to be displayed.

Create New Product Order

id

Id of the workflow.

new-product-order

steps

An array of step and step collection objects representing the tasks within the workflow.

[
  {..step object..},
  {..step object..},
  {..step collection object..},
  {..step object object..},
  {..step collection object..}
]

sessionType

The type of session. Use “registration” if workflow attempts each have a different registration.

Use “activityId” if workflow attempts each have a different activity id.

Workflows where the learner can have multiple attempts at the workflow with the same registration and activity id are not supported.

registration

stepsRequiredForCompletion

Which steps within the workflow need to be completed in order for the workflow to be considered complete. Either “all” of the steps or just the “finalStep”.

all

finalStep

exitStep

The step, which when completed, indicates that the workflow has been completed by the learner.

Contains a step object.

{..step object..}

timeLimitSeconds

The number of seconds within which the workflow must be completed from when any step is first completed.

3600

scoreActivityId

The activity id used to determine the learner’s score for the workflow.

http://example.com/customerservice/score

Please note: The first step completed should be the first step of the workflow, but learners may deviate from that path. Their time limit starts after they complete any step, even if it is not the first.

Competencies - Advanced Configuration

Competencies represent demonstrations of competence within a program. They are configured in a similar way to paths, but visualized very differently:

  • A program might be expected to have many competencies but only one primary path.
  • Steps that make up a competency should be thought of more as criteria; there is no expectation that they will be completed in a particular order.
  • The score achieved when completing a competency is significant and is used to determine the learner’s proficiency in that competency. Weighting can be applied to steps to account for differences in difficulty of activities demonstrating competence.

Competency configuration objects have the following properties:

Property

Explanation

Example

type

The object type. Always ‘competency’.

competency

name

Name of the competency to be displayed.

Widget 500 Demonstrations

id

Id of the competency.

widget-5000-demo

steps

An array of step and step collection objects representing the criteria for achieving the competency.

[
  {..step..},
  {..step..},
  {..step collection..},
  {..step..},
  {..step collection..},
]

stepWeights

Contains a step weights object defining the weights that should be applied to each step when calculations the competency score.

Properties of the step weights object are the names of steps and the values are the weightings. Weightings should add up to 1 in total.

{
  "Articulating": 0.25,
  "Investigating": 0.75
}

Steps and Step Collections - Advanced Configuration

Paths and competencies both contain arrays of steps and step collections. Step collections are used to require the learner to complete a configurable number of actions from a given list. For example a learner might have to pass 2 out of 3 quizzes in order to complete the milestone. Step collections have the following properties:

Property

Explanation

Example

type

The object type. Always ‘collection’.

collection

name

Name of the step collection. If this is a top level step, this name is displayed as a program milestone.

Complete pre-training e-learning module

completeThreshold

The number of steps within the collection that need to be completed in order for the collection to be considered complete.

2

exemptPersonCustomIds

A list of custom ids of people who are exempt from completing this step. Exempt people/steps are not counted in completion percentages.

["alice@example.com", "bob@example.com"]

exemptGroupCustomIds

A list of custom ids of groups who are exempt from completing this step. Exempt people/steps are not counted in completion percentages.

["Region: West", "Region: South"]

steps

An array of step and step collection objects (collections can be nested).

[
  {..step..},
  {..step..},
  {..step collection..},
  {..step..},
  {..step collection..},
]

Steps have the following properties:

Property

Explanation

Example

type

All steps should have a type of ‘match’.

match

name

Name of the step. If this is a top level step, this name is displayed as a program milestone.

Steps don’t have ids, so make sure the name of the step is unique.

Complete pre-training e-learning module

filter

Filter to be matched. If the filter contains one or more statement for each learner, then that learner is considered to have completed the step.

See advanced filters.

Assessments - Advanced Configuration

The assessments property determines which assessments are listed for the program. It contains an array of assessment filter objects with the following properties:

Property

Explanation

Example

name

The name of the assessment as it should be displayed in this Program report.

E.g. if the full name of an assessment is “Program X: Module 1 Assessment”, then it might be called simply “Module 1” in the assessments section of the Program X program report. 

activityId

The activity id of the assessment to use.

http://example.com/activities/example-assessment

Data Origin - Advanced Configuration

The dataOrigins property determines which data sources are listed for the program. It contains an array of data source filter objects with the following properties:

Property

Explanation

Example

name

The name of the data source as it should be displayed in this Program Report.

Coaching App

filter

Filter used to match data source data.

See advanced filters.

Additional Advanced Configuration Options

Property

Explanation

Example

hideDuration

Hides the "Average Time to Complete this Program" data point in the top part of the Program Report when set to true

true

hideGroups

Hides the "Groups" section when set to true

false

hidePeople

Hides the "People" section when set to true

true

hideInteractions

Hides the "Total Interactions" section when set to true

false

hideContributingSystems

Hides the "Contributing Systems" section when set to true

true

flattenGroups

In the "Groups" section, makes group names not clickable when set to true

false
Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.

If you can't find what you need or you want to ask a real person a question, please contact customer support.