BioASQ Scripts

This repository contains some scripts that are useful when dealing with BioASQ data. The following scripts are included:

detect_duplicates.js

Finds pairwise annotated questions by title string comparison.

Usage: detect_duplicates.js [options]

Options:

    -h, --help                           output usage information
    -q, --question-file <file name>      JSON file with questions to check
    -f, --output-format [output format]  output format [list]

Supported output formats:

    list:       list with information about authors of duplicates
    idlist:     list with plain question IDs
    blacklist:  black list for each author with IDs of duplicate by other author
    questions:  JSON dump of duplicate questions
    map:        JSON object mapping question IDs to their corresponding duplicate ID
    merged:     JSON export of duplicate questions with their annotations merged

dump_logs.js

Dumps the log database in log file format.

dump_questions.js

Dumps the questions database as JSON.

Usage: dump_questions.js [options]

Options:

    -h, --help                       output usage information
    -f, --question-file <file name>  JSON file with questions to import
    -d, --database-name <name>       Database to import to
    -c, --collection-name <name>     Collection to import to
    -r, --read-from-stdin            read question IDs from stdin (one per line)
    -n, --finalized-only             dump only questions whose publication level is not `private`

import_questions.js

Imports questions in JSON format into the questions database specified.

Usage: import_questions.js [options]

Options:

    -h, --help                       output usage information
    -f, --question-file <file name>  JSON file with questions to import
    -d, --database-name <name>       Database to import to
    -c, --collection-name <name>     Collection to import to

createTestPhase.py

Creates the files for the task B BioASQ challenge, given dump data from the BioASQ Annotation Tool.

Usage: python createTestPhase.py filename

Filename: the path to the file containing the Annotation Tool data 

Example Call: python createDataFiles.py annotationToolData-toy.json 
Comment: The annotationToolData-toy.json is provided to test the implementation.

createStats.py

Provides a summary with stastistics about a dataset of the task B BioASQ challenge.

Usage: python createStats.py filename

Filename: the path to the file containing the golden data of a task B BioASQ challenge, as created by the script     createTestPhase.py

Example Call: python createStats.py golden.json
Comment: The annotationToolData-toy.json is provided to test the script createTestPhase.py. In a next step you can use its output file "golden.json" to test this script.