WG 3: Assessment of Empirical and Cognitive Soundness
This WG approaches DRDs from the perspectives of methodologically-sound cross-linguistic analysis, and of psycholinguistic experimentation. Within the framework of the Action “best practices” will be confronted, including results on inter-annotator agreement, compatibility of the taxonomies with psycholinguistic findings, and performance on automatic sense annotation.
Milestones
Deliverables
•Sharing experimental methodologies for assessing the cognitive processing of DRDs both within and across languages, and for testing the cognitive validity of postulated semantic and pragmatic features used in the annotation scheme;
•Measuring the interrater agreement of the different annotation schemes analysed in WG2, within and between languages;
•Performing automatic sense annotation Challenges.
•Experimental design and methods will constitute specific panels of the Training Schools.
•Administration of open Challenges within one of the established Challenge frameworks such as those run annually by ConNL and SIGSem, where all interested parties can submit their solutions to specific DRD-related tasks, which can then be systematically compared and evaluated.
•Raising additional funding to submit the proposed taxonomies to empirical testing