Skip to content

clue prompt templates#808

Open
yongzx wants to merge 7 commits intobigscience-workshop:mainfrom
yongzx:clue
Open

clue prompt templates#808
yongzx wants to merge 7 commits intobigscience-workshop:mainfrom
yongzx:clue

Conversation

@yongzx
Copy link
Contributor

@yongzx yongzx commented Jul 27, 2022

Add prompt templates to the CLUE benchmark tasks.

Currently:

  • 3 original task prompt templates
  • 1 non-original task prompt template

WIP:

  • NLI (ocnli, cmnli)
  • IFLYTEK: it has 119 label names, and I haven’t populated the answer-choices yet.
  • ChID: weird dataset structure where each example contains 4 subexamples. It needs the model to find 4 correct idioms for those 4 examples. The metrics are accuracy, and I am thinking of linearizing the dataset.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants