Ambiguous Part-of-Speech Tagging for Improving Accuracy and Domain Portability of Syntactic Parsers Kazuhiro Yoshida ∗ Yoshimasa Tsuruoka † Yusuke Miyao ∗ Jun’ichi Tsujii ∗†‡ ∗ Department of Computer Science, University of Tokyo † School of Informatics, University of Manchester ‡ National Center for Text Mining {kyoshida,tsuruoka,yusuke,tsujii}@is.s.u-tokyo.ac.jp Abstract We aim to improve the performance of a syntac- tic parser that uses a part-of-speech (POS) tagger as a preprocessor. Pipelined parsers consisting of POS taggers and syntactic parsers have several ad- vantages, such as the capability of domain adapta- tion. However the performance of such systems on raw texts tends to be disappointing as they are af- fected by the errors of automatic POS tagging. We attempt to compensate for the decrease in accu- racy caused by automatic taggers by allowing the taggers to output multiple answers when the tags cannot be determined reliably enough. We empir- ically verify the effectiveness of the method us- ing an HPSG parser trained on the Penn Treebank. Our results show that ambiguous POS tagging im- proves parsing if outputs of taggers are weighted by probability values, and the results support pre- vious studies with similar intentions. We also ex- amine the effectiveness of our method for adapting the parser to the GENIA corpus and show that the use of ambiguous POS taggers can help develop- ment of portable parsers while keeping accuracy high. 1 Introduction Some parsers use POS taggers as their preprocessors, and some use integrated models that achieve tagging and parsing simultaneously. Because the latter approach is more general, it is successfully used by some of the state-of-the-art parsers, such as Charniak’s [Charniak and Johnson, 2005], as a natural consequence of the pursuit of accuracy. However, integrated models of tagging and parsing tend to be complex and com- putationally expensive, both in terms of training and run-time costs. On the other hand, pipelined systems of POS taggers and parsers can be built with independently developed taggers and parsers. In such models, we can easily make use of taggers that use state-of-the-art sequence labeling techniques, most of which are difficult to be incorporated into syntactic disam- biguation models. Advantages of pipelined parsers also in- clude their ability to adapt to domains. POS taggers for a new domain are much easier to develop than full parsers, because training corpora for POS taggers are easier to construct com- pared to those for full parsers, which require the annotation of nested phrase structures. However, independence assumption of taggers and parsers may degrade the overall accuracy of the parsers. Wat- son [2006] reported that using an automatic POS tagger caused the F1 score of grammatical relations output by a parser to drop by 2.02 points. She attempted to weaken the independence assumption by letting the taggers output multi- ple tags for each word, weighted by probability values. Her approach improved the F1 score by 0.66 points. In this paper, we verify Watson’s results on ambiguous POS tagging using an HPSG [Pollard and Sag, 1994] parser developed and trained on the Penn Treebank [Marcus et al., 1994]. At the same time, we investigate the effectiveness of ambiguous POS tagging for domain adaptation of parsers us- ing the GENIA corpus [Ohta et al., 2002] as the test set. Ex- perimental results show that the multiple output without prob- ability values cannot improve the parser much and suggest the importance of probability distribution of multiple tags ob- tained by POS taggers. Additionally, we show that the pos- itive effect of ambiguous POS tagging is maintained for do- mains unfamiliar to the parser. 2 Background In this section, we describe the POS tagger and syntactic parser used in our experiments. These taggers and parsers were combined to make a pipelined syntactic parser for raw texts using the strategy described in the next section. As both of our tagging and parsing models are based on log-linear classifiers, we first briefly introduce log-linear models and then describe our tagger and parser. 2.1 Log-linear models Log-linear models are among the most widely used machine learning techniques in NLP literature, and we use the models both for POS taggers and syntactic parsers. A conditional log- linear model calculates the probability of an event E given the IJCAI-07 1783