Skip to content

add parse_audience to all taxila scrapers#1229

Open
mikesndrs wants to merge 5 commits intoElixirTeSS:masterfrom
DaanVanVugt:update/taxila_scrapers_target_audience
Open

add parse_audience to all taxila scrapers#1229
mikesndrs wants to merge 5 commits intoElixirTeSS:masterfrom
DaanVanVugt:update/taxila_scrapers_target_audience

Conversation

@mikesndrs
Copy link
Contributor

@mikesndrs mikesndrs commented Feb 4, 2026

Summary of changes
Target audience missing for some Taxila scrapers

Checklist

  • I have read and followed the CONTRIBUTING guide.
  • I confirm that I have the authority necessary to make this contribution on behalf of its copyright owner and agree
    to license it to the TeSS codebase under the
    BSD license.

Copy link
Member

@fbacall fbacall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some tests for the auto_parse_vars feature?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a config-driven “auto-parse from description” mechanism (via JSON mapping files) intended to populate fields like target_audience/keywords during ingestion, and removes the per-scraper parse_audience(...) assignments from the Taxila ingestors.

Changes:

  • Add feature.auto_parse_vars and JSON mapping files to auto-populate selected fields from description.
  • Apply auto-parsing during add_event / add_material.
  • Remove explicit target_audience parsing lines from multiple Taxila scrapers and add a unit test for the new behavior.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 26 comments.

Show a summary per file
File Description
test/unit/ingestors/ingestor_test.rb Adds tests for the new auto_parse_vars ingestion behavior.
lib/ingestors/taxila/wur_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/uva_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/uu_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/utwente_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/tdcc_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/surf_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/rug_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/rdnl_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/oscm_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/oscd_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/odissei_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/nwo_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/maastricht_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/leiden_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/lcrdm_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/han_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/dtls_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/taxila/dans_ingestor.rb Removes explicit target_audience assignment (now relying on auto-parse).
lib/ingestors/material_ingestion.rb Adds auto-parsing for materials (currently contains a breaking variable reference).
lib/ingestors/event_ingestion.rb Adds auto-parsing for events and replaces the previous parse_audience method.
lib/ingestors/auto_parser_mappings/target_audience.json Adds keyword-to-audience-category mappings for auto-parsing.
lib/ingestors/auto_parser_mappings/keywords.json Adds keyword-to-keyword-category mappings for auto-parsing.
config/tess.example.yml Documents new feature.auto_parse_vars configuration option.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mikesndrs mikesndrs requested a review from fbacall March 2, 2026 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants