Web Courseworks

As an xAPI conformant LRS, Watershed can receive data from any xAPI conformant Activity Provider. Many Watershed cards are flexible and you can configure Watershed to display useful visualizations and metrics from almost any xAPI data set. To help you get the most out of your data, we’re working with a number of product vendors to ensure that the data they send is optimized to produce the best possible results in Watershed. We want to help you to configure Watershed cards in the best way possible to display that data.

Web Courseworks is an innovative learning technologies company. They believe that online learning should be engaging, accessible, and impactful and that it should change learners and improve the quality of their practice.

Web Courseworks designed and developed CourseStage LMS to help associations and non-profits deliver on-demand online learning, and connect learners in place-based live and virtual live events into a centralized learning platform. Their clients use CourseStage LMS to expand their educational influence in their markets, to increase revenue from their education departments, and to serve their members with outstanding credit and certification-awarding learning experiences.

We worked with Web Courseworks to vizualize xAPI data collected from medical simulation dummies used by the American College of Chest Physicians (CHEST). This guide outlines the project and the vizualizations we created in Watershed in three sections:

Collecting the data

CHEST (American College of Chest Physicians) used xAPI to gather data from high fidelity mannequins and faculty in order to provide detailed feedback for physicians who were trying to learn how to properly intubate difficult airway patients. The data extracted from the mannequins allowed instructors to evaluate pre to post learning outcomes in Web Courseworks’ LMS as well as begin to set benchmarks for physicians nationwide.

We got involved with the project after it had been completed and data had been collected by Web Courseworks in an open source LRS. We pulled the data into Watershed in order to produce visualizations and reports not possible with the original LRS.

Measures

The data collected by Web Courseworks was incredibly detailed, tracking every aspect of interaction with the dummy patient and statistics of it’s health such as breath rate, blood pressure etc. We created over 50 different measures in Watershed. For the sake of brevity, this guide will cover only a selection of those reports that most stood out and the measures underpinning those reports.

Time to IV

Time to IV is the amount of time it takes before the physician recognizes that the patient is in need of intubation and starts the process. Sometimes doctors can be hesitant to start the procedure, so this is an important measure for CHEST. This measure is configured in Advanced Configuration with the following syntax:

{
  "name": "Time to IV",
  "aggregation": {
    "type": "TIME_BETWEEN",
    "ignoreRegistrations": false
  },
  "valueProducer": {
    "type": "TIME_BETWEEN",
    "startFilter": {
      "verbIds": {
        "ids": [
          "http://leardal.xapi.webcourseworks.com/verbs/tapandtouch"
        ]
      }
    },
    "endFilter": {
      "verbIds": {
        "ids": [
          "http://leardal.xapi.webcourseworks.com/verbs/iv"
        ]
      }
    }
  },
  "fingerprint": "Measure|",
  "filter": {}
}

Time to stop after IV

Time to stop after IV is the time from when the doctor starts intubation to when the procedure is complete and the observer marks the scenario as done. In real life, there can be consequences if the procedure takes too long and therefore it’s important for CHEST to see where doctors are taking too long in training.  This measure is configured in Advanced Configuration with the following syntax:

{
  "name": "Time to stop after IV",
  "aggregation": {
    "type": "TIME_BETWEEN",
    "ignoreRegistrations": false
  },
  "valueProducer": {
    "type": "TIME_BETWEEN",
    "startFilter": {
      "verbIds": {
        "ids": [
          "http://leardal.xapi.webcourseworks.com/verbs/iv"
        ]
      }
    },
    "endFilter": {
      "verbIds": {
        "ids": [
          "http://leardal.xapi.webcourseworks.com/verbs/scenariostop"
        ]
      }
    }
  }
}

BP Count

BP count is a count of the number of times the doctor checks the patient's blood pressure. We can also get the value of the diastolic and systolic BP at that time, but since the patient is simulated that’s less interesting for CHEST. The BP count tells the report viewer whether or not the doctor checked the BP multiple times to ensure an accurate reading. Other measures such as HR count and Breath Rate count give similar insight into how many times and how often the doctor checked other vital signs.

image00.png

BP Count is configured with these settings:

Head Tilt and Jaw Thrust Count

Head tilts and jaw thrusts are actions taken by the doctor to try and get the tube into position. These can take a few attempts, but too many attempts is a sign that the doctor is struggling. The measure configured in the screenshot below is a combined count of both actions; it’s also possible to count each action individually.

image01.png

Reports

The data collected by Web Courseworks was incredibly detailed, tracking every aspect of interaction with the dummy patient and statistics of it’s health such as breath rate, blood pressure etc. We created over 50 different measures in Watershed. For the sake of brevity, this guide will cover only a selection of those reports that most stood out and the measures underpinning those reports.

IV Times By Attempt

As described above, the time to IV shows the amount of time before the doctor starts intubation and time from IV shows the time until the procedure is successfully completed. The stacked bar chart vizualization shown below enables the report viewer to see these two times individually as well as in a combined total.

This report uses the Time to IV and Time to Stop After IV measures and is organized by attempt as represented by the context.registration property with the following Advanced Configuration:

"dimensions": [
  {
    "type": "STATEMENT_PROPERTY",
    "statementProperty": "context.registration"
  }
]

image02.png

Counts By Attempt

The heatmap report is a great way of highlighting insights. CHEST are able to use this report to instantly see that very few doctors were checking the heart rate and BP multiple times to get an accurate figure. They are also able see the attempts where the doctor was clearly struggling to get the tube down the patient's throat, indicated by some very high head tilt and jawthrust counts in some cases.

This report uses a range of measures as shown in the screenshot column names and is organized by attempt as represented by the context.registration property with the following Advanced Configuration:

"dimensions": [
  {
    "type": "STATEMENT_PROPERTY",
    "statementProperty": "context.registration"
  }
]

image04.png

Correlations

On seeing the high head tilt and jaw thrust counts, CHEST wanted to know if there was a correlation between those counts and the Time to IV and Time to Stop After IV measures. More data from additional training sessions is needed to make an accurate assessment, and the chart shown below is configured ready for that additional data.

This report uses a range of measures as shown in the screenshot measure names and is organized by attempt as represented by the context.registration property with the following Advanced Configuration:

"dimensions": [
  {
    "type": "STATEMENT_PROPERTY",
    "statementProperty": "context.registration"
  }
]

image03.png

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

If you can't find what you need or you want to ask a real person a question, please contact customer support.