(Back to NLP Resources)

This is a list of freely available development tools that can be useful in NLP projects. CLT members can edit entries at will; feel free to edit or add new links, but also to remove dead and outdated links.

List of general online resources

Don't forget to look at the CLT toolkit first!

Alternative lists of resources

• Wikipedia list: http://en.wikipedia.org/wiki/List_of_natural_language_processing_toolkits
• LingPipe's list of competitors: http://alias-i.com/lingpipe/web/competition.html
• List from Stanford: http://www-nlp.stanford.edu/links/statnlp.html
• List from LT World: http://www.lt-world.org/kb/resources-and-tools/language-tools
• A list of tools for Polish NLP: http://clip.ipipan.waw.pl/LRT

Generic resources

• CLT toolkit: http://www.clt.gu.se/clt-toolkit
• NLTK, lots of Python resources: http://www.nltk.org
• GATE, General Architecture for Text Engineering: http://gate.ac.uk
• OpenNLP, lots of Java resources: http://incubator.apache.org/opennlp
• FreeLing, open source suite of Language Analyzers: http://nlp.lsi.upc.edu/freeling
• LinguaStream, an Integrated Experimentation Environment for Computational Linguistics: http://www.linguastream.org
• nlpFarm, facility software for natural language processing in Java with focus on conversational systems.: http://sourceforge.net/projects/nlpfarm
• Some tools from KTH: http://www.csc.kth.se/tcs/humanlang/tools.html

List of online resources, grouped by task in alphabetical order

Annotation tools

• NITE XML Toolkit: http://groups.inf.ed.ac.uk/nxt
• Exmaralda: http://www.exmaralda.org/en_index.html
• TranscriberAG, a tool for segmenting, labeling and transcribing speech: http://transag.sourceforge.net

Corpus search and editing

• CWB, Corpus Workbench, tools for managing and querying large text corpora: http://cwb.sourceforge.net
• Stanford search and transformation for trees: http://nlp.stanford.edu/software/tregex.shtml
• TrEd is a fully customizable and programmable graphical editor and viewer for tree-like structures: http://ufal.mff.cuni.cz/tred
• PML-TQ is an powerful open-source search tool for all kinds of linguistaically annotated treebanks: http://ufal.mff.cuni.cz/~pajas/pmltq
• ANNIS2 is a web browser-based search and visualization architecture for complex multilevel linguistic corpora: http://www.sfb632.uni-potsdam.de/d1/annis

Dialogue systems

• TrindiKit is a toolkit for building and experimenting with dialogue move engines and information states: http://www.ling.gu.se/projekt/trindi/trindikit
• Dipper is another dialogue prototyping toolkit: http://www.ltg.ed.ac.uk/dipper
• Here is a list of VoiceXML development tools: http://www.palowireless.com/voicexml/devtools.asp

Entity recognition

• Stanford Named Entity Recognizer: http://nlp.stanford.edu/software/CRF-NER.shtml

Finite-State Machines & Transducers

• AT&T Finite-State Machine Library: http://www2.research.att.com/~fsmtools/fsm
• XFST and some more tools: http://www.fsmbook.com
• HFST, Helsinki Finite-State Transducer Tools: http://www.ling.helsinki.fi/kieliteknologia/tutkimus/hfst
• OpenFst is a library for weighted FSTs: http://www.openfst.org
• Foma: http://code.google.com/p/foma/
• SFST, Stuttgart Finite State Transducer Tools: http://www.ims.uni-stuttgart.de/projekte/gramotron/SOFTWARE/SFST.html

Grammar formalisms

• GF, Grammatical Framework (type-theoretical grammars): http://grammaticalframework.org
• ALE, Attribute Logic Engine (feature-structure grammars): http://www.cs.toronto.edu/~gpenn/ale.html
• The LKB system is a grammar and lexicon development environment for HPSG: http://moin.delph-in.net/LkbTop
• The LinGO Grammar Matrix is an open-source starter-kit for rapid prototyping of precision broad-coverage LKB grammars: http://moin.delph-in.net/MatrixTop
• The PET system for efficient processing of HPSG grammars: http://moin.delph-in.net/PetTop
• XMG, eXtensible MetaGrammar (TAG and related formalisms): http://sourcesup.cru.fr/xmg
• Regulus, a platform for compiling unification grammars into speech-recognition grammars: http://www.issco.unige.ch/en/research/projects/regulus

Grammar inference

• Alignment-Based Learning (ABL) grammatical inference system: http://ilk.uvt.nl/menno/research/software/abl

Lemmatisation

• CST Lemmatiser: http://www.cst.dk/download/uk

Machine learning

• TiMBL, Tilburg Memory Based Learner: http://ilk.uvt.nl/timbl
• Weka: http://www.cs.waikato.ac.nz/~ml/weka
• Mallet: http://mallet.cs.umass.edu
• FlexCRFs is a conditional random field toolkit for segmenting and labeling sequence data: http://flexcrfs.sourceforge.net

Machine translation

• Moses is a statistical machine translation system: http://www.statmt.org/moses
• MBMT, Memory-based machine translation: http://ilk.uvt.nl/mbmt
• DEMOCRAT takes the output of several MT systems and combines the translations in one: http://ilk.uvt.nl/menno/research/software/democrat
• Apertium, open-source machine translation platform: http://www.apertium.org
• GIZA++: http://code.google.com/p/giza-pp
• Chaski is a distributed toolkit for machine translation: http://www.kyloo.net/software/doku.php/chaski:overview

Ontology editing

• Protegé, open source ontology editor: http://protege.stanford.edu

Parsing

• Stanford statistical parser for English: http://nlp.stanford.edu/software/lex-parser.shtml
• MaltParser, data-driven dependency parsing: http://maltparser.org
• BitPar is a parser for highly ambiguous probabilistic context-free grammars: http://www.ims.uni-stuttgart.de/tcl/SOFTWARE/BitPar.html
• TuLiPA, Tübingen Linguistic Parsing Architecture: http://sourcesup.cru.fr/tulipa
• PET, efficient processing of unification-based grammars: http://wiki.delph-in.net/moin/PetTop
• C&C CCG parser: http://svn.ask.it.usyd.edu.au/trac/candc
• DyALog, An environment for building tabular parsers and programs: http://alpage.inria.fr/dyalog.en.html
• Link Grammar, a syntactic parser of English: http://www.abisource.com/projects/link-grammar
• MSTParser is a non-projective dependency parser: http://sourceforge.net/projects/mstparser
• Berkelyparser: http://code.google.com/p/berkeleyparser

Semantics

• utool, The Swiss Army Knife of Underspecification: http://www.coli.uni-saarland.de/projects/chorus/utool
• Boxer, generates semantic representations (DRS – Discourse Representation Structures): http://svn.ask.it.usyd.edu.au/trac/candc/wiki/boxer
• PyPES, Python Platform for Experimentation with Semantics (MRS – Minimal Recursion Semantics): http://www.semantilog.org/pypes.html

Speech recognition

• CMU Sphinx, open source Speech Recognition Toolkit: http://cmusphinx.sourceforge.net
• Julius, open source speech recognition engine: http://julius.sourceforge.jp/en_index.php
• HTK, Hidden Markov Model Toolkit: http://htk.eng.cam.ac.uk

Speech synthesis

• Festival Speech Synthesis: http://www.cstr.ed.ac.uk/projects/festival
• Festvox is a toolkit for building new synthetic voices: http://festvox.org
• eSpeak, open-source software formant synthesis: http://espeak.sourceforge.net
• MARY is an open-source, multilingual Text-to-Speech Synthesis platform: http://mary.dfki.de
• FreeTTS, a Java speech synthesiser: http://freetts.sourceforge.net/docs/index.php
• Flite, a small, fast run-time synthesis engine: http://www.speech.cs.cmu.edu/flite

Tagging

• Stanford log-linear tagger: http://nlp.stanford.edu/software/tagger.shtml
• Hunpos, open-source HMM tagger: http://code.google.com/p/hunpos
• SVMTool, an open source generator of sequential taggers: http://www.lsi.upc.edu/~nlp/SVMTool
• MBT, Memory-based tagger: http://ilk.uvt.nl/mbt

Text generation

• Listing of NLG systems: http://www.nlg-wiki.org/systems

Tokenisation

• UCTO, Unicode Tokenizer: http://ilk.uvt.nl/ucto

Visualisation

• A visualizer for Natural Language Processing problems: http://code.google.com/p/whatswrong

Web services

• CLAM allows you to quickly and transparently transform your NLP application into a webservice: http://ilk.uvt.nl/clam