Jakub Piskorski


2020

pdf bib
TF-IDF Character N-grams versus Word Embedding-based Models for Fine-grained Event Classification: A Preliminary Study
Jakub Piskorski | Guillaume Jacquet
Proceedings of the Workshop on Automated Extraction of Socio-political Events from News 2020

Automating the detection of event mentions in online texts and their classification vis-a-vis domain-specific event type taxonomies has been acknowledged by many organisations worldwide to be of paramount importance in order to facilitate the process of intelligence gathering. This paper reports on some preliminary experiments of comparing various linguistically-lightweight approaches for fine-grained event classification based on short text snippets reporting on events. In particular, we compare the performance of a TF-IDF-weighted character n-gram SVM-based model versus SVMs trained on various of-the-shelf pre-trained word embeddings (GloVe, BERT, FastText) as features. We exploit a relatively large event corpus consisting of circa 610K short text event descriptions classified using a 25-event categories that cover political violence and protest events. The best results, i.e., 83.5% macro and 92.4% micro F1 score, were obtained using the TF-IDF-weighted character n-gram model.

pdf bib
New Benchmark Corpus and Models for Fine-grained Event Classification: To BERT or not to BERT?
Jakub Piskorski | Jacek Haneczok | Guillaume Jacquet
Proceedings of the 28th International Conference on Computational Linguistics

We introduce a new set of benchmark datasets derived from ACLED data for fine-grained event classification and compare the performance of various state-of-the-art models on these datasets, including SVM based on TF-IDF character n-grams and neural context-free embeddings (GLOVE and FASTTEXT) as well as deep learning-based BERT with its contextual embeddings. The best results in terms of micro (94.3-94.9%) and macro F1 (86.0-88.9%) were obtained using BERT transformer, with simpler TF-IDF character n-gram based SVM being an interesting alternative. Further, we discuss the pros and cons of the considered benchmark models in terms of their robustness and the dependence of the classification performance on the size of training data.

2019

pdf bib
Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing
Tomaž Erjavec | Michał Marcińczuk | Preslav Nakov | Jakub Piskorski | Lidia Pivovarova | Jan Šnajder | Josef Steinberger | Roman Yangarber
Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing

pdf bib
The Second Cross-Lingual Challenge on Recognition, Normalization, Classification, and Linking of Named Entities across Slavic Languages
Jakub Piskorski | Laska Laskova | Michał Marcińczuk | Lidia Pivovarova | Pavel Přibáň | Josef Steinberger | Roman Yangarber
Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing

We describe the Second Multilingual Named Entity Challenge in Slavic languages. The task is recognizing mentions of named entities in Web documents, their normalization, and cross-lingual linking. The Challenge was organized as part of the 7th Balto-Slavic Natural Language Processing Workshop, co-located with the ACL-2019 conference. Eight teams participated in the competition, which covered four languages and five entity types. Performance for the named entity recognition task reached 90% F-measure, much higher than reported in the first edition of the Challenge. Seven teams covered all four languages, and five teams participated in the cross-lingual entity linking task. Detailed evaluation information is available on the shared task web page.

pdf bib
JRC TMA-CC: Slavic Named Entity Recognition and Linking. Participation in the BSNLP-2019 shared task
Guillaume Jacquet | Jakub Piskorski | Hristo Tanev | Ralf Steinberger
Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing

We report on the participation of the JRC Text Mining and Analysis Competence Centre (TMA-CC) in the BSNLP-2019 Shared Task, which focuses on named-entity recognition, lemmatisation and cross-lingual linking. We propose a hybrid system combining a rule-based approach and light ML techniques. We use multilingual lexical resources such as JRC-NAMES and BABELNET together with a named entity guesser to recognise names. In a second step, we combine known names with wild cards to increase recognition recall by also capturing inflection variants. In a third step, we increase precision by filtering these name candidates with automatically learnt inflection patterns derived from name occurrences in large news article collections. Our major requirement is to achieve high precision. We achieved an average of 65% F-measure with 93% precision on the four languages.

2018

pdf bib
On Training Classifiers for Linking Event Templates
Jakub Piskorski | Fredi Šarić | Vanni Zavarella | Martin Atkinson
Proceedings of the Workshop Events and Stories in the News 2018

The paper reports on exploring various machine learning techniques and a range of textual and meta-data features to train classifiers for linking related event templates automatically extracted from online news. With the best model using textual features only we achieved 94.7% (92.9%) F1 score on GOLD (SILVER) dataset. These figures were further improved to 98.6% (GOLD) and 97% (SILVER) F1 score by adding meta-data features, mainly thanks to the strong discriminatory power of automatically extracted geographical information related to events.

2017

pdf bib
Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing
Tomaž Erjavec | Jakub Piskorski | Lidia Pivovarova | Jan Šnajder | Josef Steinberger | Roman Yangarber
Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing

pdf bib
The First Cross-Lingual Challenge on Recognition, Normalization, and Matching of Named Entities in Slavic Languages
Jakub Piskorski | Lidia Pivovarova | Jan Šnajder | Josef Steinberger | Roman Yangarber
Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing

This paper describes the outcomes of the first challenge on multilingual named entity recognition that aimed at recognizing mentions of named entities in web documents in Slavic languages, their normalization/lemmatization, and cross-language matching. It was organised in the context of the 6th Balto-Slavic Natural Language Processing Workshop, co-located with the EACL 2017 conference. Although eleven teams signed up for the evaluation, due to the complexity of the task(s) and short time available for elaborating a solution, only two teams submitted results on time. The reported evaluation figures reflect the relatively higher level of complexity of named entity-related tasks in the context of processing texts in Slavic languages. Since the duration of the challenge goes beyond the date of the publication of this paper and updated picture of the participating systems and their corresponding performance can be found on the web page of the challenge.

pdf bib
Multi-word Entity Classification in a Highly Multilingual Environment
Sophie Chesney | Guillaume Jacquet | Ralf Steinberger | Jakub Piskorski
Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017)

This paper describes an approach for the classification of millions of existing multi-word entities (MWEntities), such as organisation or event names, into thirteen category types, based only on the tokens they contain. In order to classify our very large in-house collection of multilingual MWEntities into an application-oriented set of entity categories, we trained and tested distantly-supervised classifiers in 43 languages based on MWEntities extracted from BabelNet. The best-performing classifier was the multi-class SVM using a TF.IDF-weighted data representation. Interestingly, one unique classifier trained on a mix of all languages consistently performed better than classifiers trained for individual languages, reaching an averaged F1-value of 88.8%. In this paper, we present the training and test data, including a human evaluation of its accuracy, describe the methods used to train the classifiers, and discuss the results.

pdf bib
On the Creation of a Security-Related Event Corpus
Martin Atkinson | Jakub Piskorski | Hristo Tanev | Vanni Zavarella
Proceedings of the Events and Stories in the News Workshop

This paper reports on an effort of creating a corpus of structured information on security-related events automatically extracted from on-line news, part of which has been manually curated. The main motivation behind this effort is to provide material to the NLP community working on event extraction that could be used both for training and evaluation purposes.

2015

pdf bib
The 5th Workshop on Balto-Slavic Natural Language Processing
Jakub Piskorski | Lidia Pivovarova | Jan Šnajder | Hristo Tanev | Roman Yangarber
The 5th Workshop on Balto-Slavic Natural Language Processing

pdf bib
Open Relation Extraction for Polish: Preliminary Experiments
Jakub Piskorski
The 5th Workshop on Balto-Slavic Natural Language Processing

2013

pdf bib
Proceedings of the 4th Biennial International Workshop on Balto-Slavic Natural Language Processing
Jakub Piskorski | Lidia Pivovarova | Hristo Tanev | Roman Yangarber
Proceedings of the 4th Biennial International Workshop on Balto-Slavic Natural Language Processing

pdf bib
On Named Entity Recognition in Targeted Twitter Streams in Polish.
Jakub Piskorski | Maud Ehrmann
Proceedings of the 4th Biennial International Workshop on Balto-Slavic Natural Language Processing

2011

pdf bib
Exploring the Usefulness of Cross-lingual Information Fusion for Refining Real-time News Event Extraction: A Preliminary Study
Jakub Piskorski | Jenya Belayeva | Martin Atkinson
Proceedings of the International Conference Recent Advances in Natural Language Processing 2011

2008

pdf bib
Online-Monitoring of Security-Related Events
Martin Atkinson | Jakub Piskorski | Bruno Pouliquen | Ralf Steinberger | Hristo Tanev | Vanni Zavarella
Coling 2008: Companion volume: Demonstrations

2007

pdf bib
Proceedings of the Workshop on Balto-Slavonic Natural Language Processing
Jakub Piskorski | Hristo Tanev
Proceedings of the Workshop on Balto-Slavonic Natural Language Processing

pdf bib
Lemmatization of Polish Person Names
Jakub Piskorski | Marcin Sydow | Anna Kupść
Proceedings of the Workshop on Balto-Slavonic Natural Language Processing

2006

pdf bib
Linguistic Suite for Polish Cadastral System
Witold Abramowicz | Agata Filipowska | Jakub Piskorski | Krzysztof Węcel | Karol Wieloch
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

This paper reports on an endeavour of creating basic linguistic resources for geo-referencing of Polish free-text documents. We have defined a fine-grained named entity hierarchy, produced an exhaustive gazetteer, and developed named-entity grammars for Polish. Additionally, an annotated corpus for the cadastral domain was prepared for evaluation purposes. Our baseline approach to geo-referencing is based on application of aforementioned resources and a lightweight co-referencing technique which utilizes string-similarity metric of Jaro-Winkler. We carried out a detailed evaluation of detecting locations, organizations and persons, which revealed that best results are obtained via application of a combined grammar for all types. The application of lightweight co-referencing for organizations and persons improves recall but deteriorates precision, and no gain is observed for locations. The paper is accompanied by a demo, a geo-referencing application capable of: (a) finding documents and text fragments based on named entities and (b) populating the spatial ontology from texts.

2005

pdf bib
Modelling of a Gazetteer Look-up Component
Jakub Piskorski
Companion Volume to the Proceedings of Conference including Posters/Demos and tutorial abstracts

2004

pdf bib
Integrated Language Technologies for Multilingual Information Services in the MEMPHIS Project
Walter Kasper | Jörg Steffen | Jakub Piskorski | Paul Buitelaar
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

pdf bib
Extraction of Polish Named-Entities
Jakub Piskorski
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

2003

pdf bib
Integrating Information Extraction and Automatic Hyperlinking
Stephan Busemann | Witold Drozdzynski | Hans-Ulrich Krieger | Jakub Piskorski | Ulrich Schaefer | Hans Uszkoreit | Feiyu Xu
The Companion Volume to the Proceedings of 41st Annual Meeting of the Association for Computational Linguistics

2002

pdf bib
An Integrated Archictecture for Shallow and Deep Processing
Berthold Crysmann | Anette Frank | Bernd Kiefer | Stefan Mueller | Guenter Neumann | Jakub Piskorski | Ulrich Schaefer | Melanie Siegel | Hans Uszkoreit | Feiyu Xu | Markus Becker | Hans-Ulrich Krieger
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics

pdf bib
A Flexible XML-based Regular Compiler for Creation and Conversion of Linguistic Resources
Jakub Piskorski | Witold Drożdżyński | Oliver Scherf | Feiyu Xu
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02)

pdf bib
A Domain Adaptive Approach to Automatic Acquisition of Domain Relevant Terms and their Relations with Bootstrapping
Feiyu Xu | Daniela Kurz | Jakub Piskorski | Sven Schmeier
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02)

2000

pdf bib
A Divide-and-Conquer Strategy for Shallow Parsing of German Free Texts
Gunter Neumann | Christian Braun | Jakub Piskorski
Sixth Applied Natural Language Processing Conference