Creating a Query

Queries are specified in files separate from the code that runs them. This modularity avoids hard coding the queries into the code itself and risking them being accidentally overwritten or modified between searches. Placing queries in their own files also makes them easier to share them, promoting transparency and replicability.

Event Names

All data is organized around an event name. This name should be a unique signifier for the focal event around which you want to collect data.

The event name is specified through the name of the query file. For example, say we want to search for tweets about Facebook’s Oversight Board. Then we can name our event “facebook_oversight” and create a corresponding search query file facebook_oversight.yaml. The query file should be placed in the appropriate input directory. For example, if we are running a Twitter search, then we will place our query file in the directory input/twitter/search.

Query Structure

The structure and syntax of the query file depends on the platform being queried and the endpoint’s query operators.

Twitter Filter Stream

Twitter filter stream query files are YAML files with one field: rules. The rules field allows you to list multiple filter rules sequentially, each of which will be sent to the Twitter API. Each rule should have two fields itself: value and tag. The value is the filter rule itself. The tag is a colloquial description of the filter rule.

See here for an example of a Twitter filter stream query file. See Twitter’s documentation for details on how to build a filter rule for Twitter’s stream.