Data Search

Watershed is powered by moments of activity called interaction statements (or xAPI statements). Sources ranging from mobile apps to learning management systems are connected to Watershed, and as users perform actions in these sources, interaction statements are sent into Watershed. Data Search shows every interaction statement that has been sent to your Watershed account and allows you to perform complex searches to find specific ones you're looking for. It is a great tool to troubleshoot data issues.

Data Search and Error Log were discussed as part of our Product Walkthrough webinar series. Watch a recording of the webinar for a comprehensive look at application and setup of the feature.

Finding Data Search

To find Data Search, go to the Data page and click Data Search in the menu on the left:find_data_search.png

Simple Search

The simple Data Search has 4 options: Agent, Verb, Activity, and Statement Dates. Use the Refresh button to search for values you add to one or more of the options.

Hint: you can also press Ctrl + Enter (Option + Enter on Mac) to refresh.

Screen_Shot_2017-08-08_at_10.08.49_AM.png

Agent

Each interaction statement references a certain action performed by an individual. Here's an external resource that takes a deep dive into how people are identified in interaction statements. The Agent search allows you to search for all of the statements related to specific individuals. There are three different options for agent search:

  • mbox: The mbox search allows you to find people by their email address. Note: this will only find statements that specifically have the users email address in the mbox field.Screen_Shot_2017-08-08_at_2.07.39_PM.png
  • account: In many interaction statements, the person performing an action is identified by a unique identifier for a given system, say Twitter.com, and their unique representation on that system, for instance their “Twitter handle”. When choosing the account option, you're given the options to search for Name and Homepage. In the Twitter example, the Name would be the user's account login, and the Homepage would be http://www.twitter.com/. You might need to find an example statement that uses account to identify people before actually using this search. Screen_Shot_2017-08-08_at_1.54.54_PM.png
  • name: This will search for all interaction statements associated with a particular name, regardless of whether the person is associated with an mbox or account in the statement.Screen_Shot_2017-08-08_at_2.06.45_PM.png

Verb

In each interaction statement, verbs are represented by IDs and they are paired with a display name. There are two different options for verb search, and they represent either the ID or the display name:

  • id: Search for all verbs that match the ID you choose.verbid.png
  • name: Search for the actual verb's name to pull up all statements with that verb. verbname.png

Activity

In each interaction statement, activities are represented by IDs and they are paired with a display name. Activities can also have a short description in interaction statements. There are three different options for activity search, and they represent the ID, the display name, or the description:

  • id: Search for all activities that match the ID you choose.activityid.png
  • name: Search for the actual activity's name to pull up all statements with that activity. activityname.png
  • description: Search for a full description or part of a description with the description search.activitydescription.png

Statement Dates

Two different dates are associated with every interaction statement, and you can search for date ranges associated with either date. Use the Since box to choose the beginning of the range and the Until box to search for the end of the range. If you only choose a Since or Until, it will search for all dates with the one condition you chose.

  • stored: The stored date is the date that the statement was added to Watershed.Screen_Shot_2017-08-08_at_2.28.27_PM.png
  • timestamp: The timestamp is when the activity that generated the statement actually occurred. If the data is being sent to Watershed in real time, the stored and timestamp dates should be very close, but if you're importing legacy data or the data does not come into Watershed in real time, the dates could be different. Screen_Shot_2017-08-08_at_2.28.34_PM.png

Advanced Search

Advanced Search gives you some lesser used search options that can help you narrow down to very specific statements you need. 6 additional options come with Advanced Search: Statement IDRegistration ID, Related, Format, Card Filter, and Query. Use the Refresh button to search for values you add to one or more of the options (including options from Simple Search).Screen_Shot_2017-08-08_at_2.35.46_PM.png

Statement ID

Use the Statement ID search to find an individual statement. This is useful for troubleshooting with a team.

Registration ID

Related

These flags are associated with behavior as laid out in the xAPI specification and will find people related to users you search for if the Related Agents box is checked and related activities if the Related Activities box is checked. 

Format

Format determines the format of the xAPI statements displayed in the search results. 

  • default - displays the data as it is used for reporting. This is useful when looking at the underlying data when working on report configuration. When default format is used, an additional checkbox labelled Include hierarchy info in results will appear above the interaction stream. Checking this box will also include person and group data associated with each interaction statement. 
  • exact - displays the data exactly as it was received by Watershed's LRS. This is useful when debugging a data source. 
  • ids - displays ids format as defined by xAPI. This might be useful for debugging a data source that outputs xAPI statements with large definitions. 
  • canonical - displays the canonical format as defined by xAPI. Currently this is the same as the default format, except it is not possible to include hierarchy info. 

Card Filter

Paste in the Advanced Configuration from a card filter to search for all statements that would be used to get the data for the card.

Query

See below for more information about queries.

Custom Queries

In the Queries section of Advanced Search, you can search for statements using a concise query language.query.png

Filter Expressions

You can filter statements using a number of different value tests:

  • Equals: actor.mbox = 'mailto:john@example.com'
  • Not equals: context.registration != 'b7fea80e-1685-4513-be92-59ddd13091f7'
  • In: context.platform in ('Mac', 'Windows')
  • Not In: authority.name not in ('Activity Provider 1', 'System 2')
  • Range: result.score.scaled <= 0.8

Regular Expression Values

In the above filter expressions, you can use either literal quoted values or regular expression (regex) values. To use regular expression values, surround the value with / characters and don't use quotes.

Please note: regular expressions can only be used with string values, not numbers or booleans. 

The following operators work as follows in regular expression values:

. Match any single character
+ Match one or more of the preceding pattern
* Match zero or more of the preceding pattern
? Match zero or one of the preceding pattern
{x, y} Match between x and y of the preceding pattern
(pattern) Group a pattern, to use preceding operators on
| Combine patterns in an OR operator, ex. (abc|123)
[xy] Match character x or y
[x-y] Match any character between x and y
[^x-y] Match any character except those between x and y
[^xy] Match any character except x or y

Examples

Match statements with test activity IDs in the example.com domain:

object.id = /http://example.com/activities/test.*/

Match statements with specific test activities in example.com domain:

object.id = /http://example.com/activities/test[123]/

Match statements involving the word test in the ID:

object.id = /.*test.*/

Match statements involving all actors using 'example.com' or 'example.org' emails:

actor.mbox in (/.+@example.com/, /.+@example.org/)

Or you can achieve the same result with regex alternatives:

actor.mbox = /.+@example.(com|org)/

Composing Expressions

Some examples of how to compose expressions:

AND actor.mbox = 'mailto:john@example.com' AND object.id = 'http://example.com/activities/test1'
OR  result.completion = true OR result.score.raw > 10
NOT 
!(timestamp < '2017-01-01')
Nesting  (actor.mbox = 'mailto:john@example.com' AND result.score.raw > 10) OR timestamp > '2017-01-01'

Sorting

Results can be pages from the query expression:

Single sort
actor.mbox = 'mailto:john@example.com' order by stored descending
Multi sort result.score.scaled > 0.8 order by result.score.scaled desc, timestamp asc

Paging

Results can be paged from the query expression:

Limit
result.score.scaled = 1.0 limit 10
Limit and offset result.score.scaled = 1.0 limit 50,10

Full query language specification:

query := [] [] []
filter :=  AND  |  OR  | () | !(filter) | 
sorting := order by [, ]*
sort-expression := [-] [asc | desc]
paging := limit [,]
filter-expression :=  | 
compare-expression :=  *[=|!=|<|>|<=|>=] *
in-expression :=  [not] in ([,<value])
field := 
value := /regex/ | 
word := * | '*' | "*"

Video Walkthrough

As part of our Product Walkthrough webinar series, David Ells gave us a a comprehensive look at Data Search and Error Log. Check it out below:

 
Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Article is closed for comments.

If you can't find what you need or you want to ask a real person a question, please contact customer support.