How do I compare the performance of people who have completed training vs. people who have not?

An important question to ask when evaluating the effectiveness of training is how the performance of people who completed the training compares with people who did not complete it. This walkthrough shows you how to set up a line card comparing people who have and have not completed a piece of training on a particular performance KPI.

Before starting this walkthrough, you will need to have data in Watershed about training completion and about job performance. This tutorial is intended for experienced Watershed users.

1. Create a Line Chart

To get started, create a simple line chart showing the KPI you want to look at over time. Filter the card by the group of people you want to look at, update the card text and add the appropriate KPI measure. In this example we are looking at the Average CSAT Rating of the Customer Service group.

In our example scenario, let’s pretend that we want to see whether or not attending the “Module 1” training event at the end of October helps to reduce an expected seasonal drop in CSAT rating in November.

image4.png

2. Filter by people who attended training

This shows us the average CSAT rating for all people in the Customer Service group. Now let’s filter the measure to show only people that attended the Module 1 event. Open up advanced configuration and add the following to the measure filter. In your card, you should change the activity id and verb id to match the training or learning experience you want to evaluate the impact of.

You can also rename the measure to “Attended”.

"peopleFilter":{
  "activityIds": {
    "ids": [
      "http://program.module-1-event.example.com"
    ],
    "regExp": false
  },
  "verbIds": {
    "ids": [
      "http://adlnet.gov/expapi/verbs/attended"
    ]
  }
}

image3.png

3. Filter by people who did not attend training

Now copy the measure to create a 2nd measure called “Did not attend”. Wrap the contents of the people filter in a not filter like so:

"peopleFilter": {
  "not": {
    "activityIds": {
      "ids": [
        "http://program.module-1-event.example.com"
      ]
    },
    "verbIds": {
      "ids": [
        "http://adlnet.gov/expapi/verbs/attended"
      ]
    }
  }
}

image1.png

The complete configuration for our example card looks like this:

{
  "filter": {
    "groupIds": [
      16344
    ]
  },
  "dimensions": [
    {
      "type": "TIME",
      "timePeriod": "MONTH"
    }
  ],
  "measures": [
    {
      "name": "Attended",
      "aggregation": {
        "type": "AVERAGE"
      },
      "valueProducer": {
        "type": "STATEMENT_PROPERTY",
        "statementProperty": "result.score.raw",
        "caseSensitive": true
      },
      "filter": {
        "activityIds": {
          "ids": [
            "http://csat.example.com/csat/rating"
          ]
        },
        "peopleFilter": {
          "activityIds": {
            "ids": [
              "http://program.module-1-event.example.com"
            ]
          },
          "verbIds": {
            "ids": [
              "http://adlnet.gov/expapi/verbs/attended"
            ]
          }
        }
      }
    },
    {
      "name": "Did Not Attend",
      "aggregation": {
        "type": "AVERAGE"
      },
      "valueProducer": {
        "type": "STATEMENT_PROPERTY",
        "statementProperty": "result.score.raw",
        "caseSensitive": true
      },
      "filter": {
        "activityIds": {
          "ids": [
            "http://csat.example.com/csat/rating"
          ]
        },
        "peopleFilter": {
          "not": {
            "activityIds": {
              "ids": [
                "http://program.module-1-event.example.com"
              ]
            },
            "verbIds": {
              "ids": [
                "http://adlnet.gov/expapi/verbs/attended"
              ]
            }
          }
        }
      }
    }
  ],
  "includePeopleWithoutStatements": false,
  "hiddenMeasures": [],
  "line": true,
  "singleGraph": true,
  "allowGaps": false,
  "fillGapsWithZeroes": false,
  "type": "line-chart"
}

Save the card.

4. Interpret the report

Now the card is completed, you can use the chart to determine if the training was successful.

image2.png

In our example, we can see that before the training the people would go on to attend were actually lower performers, than those who would not attend the training. After the October training, the group that attended did still see a small seasonal decline in CSAT but this decline was significantly smaller than the drop for those that did not attend.  In fact, the ‘attended’ group’s CSAT scores are now higher than the ‘did not attend’ group’s scores, whereas before the training it was the other way around.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

If you can't find what you need or you want to ask a real person a question, please contact customer support.