natural-language-processing.md

Natural Language Processing

Libraries that specialize on processing text.

• dpar - Transition-based statistical dependency parser.
• go-eco - Similarity, dissimilarity and distance matrices; diversity, equitability and inequality measures; species richness estimators; coenocline models.
• go-i18n - A package and an accompanying tool to work with localized text.
• go-mystem - CGo bindings to Yandex.Mystem - russian morphology analyzer.
• go-nlp - Utilities for working with discrete probability distributions and other tools useful for doing NLP work.
• go-stem - Implementation of the porter stemming algorithm.
• go-unidecode - ASCII transliterations of Unicode text.
• go2vec - Reader and utility functions for word2vec embeddings.
• gojieba - This is a Go implementation of jieba which a Chinese word splitting algorithm.
• golibstemmer - Go bindings for the snowball libstemmer library including porter 2
• gounidecode - Unicode transliterator (also known as unidecode) for Go
• icu - Cgo binding for icu4c C library detection and conversion functions. Guaranteed compatibility with version 50.1.
• libtextcat - Cgo binding for libtextcat C library. Guaranteed compatibility with version 2.2.
• MMSEGO - This is a GO implementation of MMSEG which a Chinese word splitting algorithm.
• paicehusk - Golang implementation of the Paice/Husk Stemming Algorithm
• porter - This is a fairly straightforward port of Martin Porter's C implementation of the Porter stemming algorithm.
• porter2 - Really fast Porter 2 stemmer.
• RAKE.go - A Go port of the Rapid Automatic Keyword Extraction Algorithm (RAKE)
• segment - A Go library for performing Unicode Text Segmentation as described in Unicode Standard Annex #29
• sentences - A sentence tokenizer: converts text into a list of sentences.
• snowball - Snowball stemmer port (cgo wrapper) for Go. Provides word stem extraction functionality Snowball native.
• stemmer - Stemmer packages for Go programming language. Includes English and German stemmers.
• textcat - A Go package for n-gram based text categorization, with support for utf-8 and raw text
• when - A natural EN and RU language date/time parser with pluggable rules
• spark-corenlp - DataFrame wrapper for Stanford CoreNLP.
• Awesome NLP with Ruby - Awesome List for Practical Natural Language Processing done in Ruby.
• Parslet - A small Ruby library for constructing parsers in the PEG (Parsing Expression Grammar) fashion.
• pocketsphinx-ruby - Ruby speech recognition with Pocketsphinx.
• Pragmatic Segmenter - Pragmatic Segmenter is a rule-based sentence boundary detection gem that works out-of-the-box across many languages.
• Ruby Natural Language Processing Resources - Collection of links to Ruby Natural Language Processing (NLP) libraries, tools and software.
• Text - A collection of text algorithms including Levenshtein distance, Metaphone, Soundex 2, Porter stemming & White similarity.
• Treat - Treat is a toolkit for natural language processing and computational linguistics in Ruby.
• Treetop - PEG (Parsing Expression Grammar) parser.
• Words Counted - A highly customisable Ruby text analyser and word counter.
• BLLIP Parser - BLLIP Natural Language Parser (also known as the Charniak-Johnson parser)
• colibri-core - C++ library, command line tools, and Python binding for extracting and working with basic linguistic constructions such as n-grams and skipgrams in a quick and memory-efficient way.
• CRF++ - Open source implementation of Conditional Random Fields (CRFs) for segmenting/labeling sequential data & other Natural Language Processing tasks.
• CRFsuite - CRFsuite is an implementation of Conditional Random Fields (CRFs) for labeling sequential data.
• frog - Memory-based NLP suite developed for Dutch: PoS tagger, lemmatiser, dependency parser, NER, shallow parser, morphological analyzer.
• libfolia - C++ library for the FoLiA format
• MeTA - MeTA : ModErn Text Analysis is a C++ Data Sciences Toolkit that facilitates mining big text data.
• MIT Information Extraction Toolkit - C, C++, and Python tools for named entity recognition and relation extraction
• ucto - Unicode-aware regular-expression based tokenizer for various languages. Tool and C++ library. Supports FoLiA format.
• Clojure-openNLP - Natural Language Processing in Clojure (opennlp)
• Infections-clj - Rails-like inflection library for Clojure and ClojureScript
• Stemmer - An English (Porter2) stemming implementation in Elixir.
• go-porterstemmer - A native Go clean room implementation of the Porter Stemming algorithm.
• paicehusk - Golang implementation of the Paice/Husk Stemming Algorithm.
• snowball - Snowball Stemmer for Go.
• go-ngram - In-memory n-gram index with compression.
• Cortical.io - Retina: an API performing complex NLP operations (disambiguation, classification, streaming text filtering, etc...) as quickly and intuitively as the brain.
• CoreNLP - Stanford CoreNLP provides a set of natural language analysis tools which can take raw English language text input and give the base forms of words
• Stanford Parser - A natural language parser is a program that works out the grammatical structure of sentences
• Stanford POS Tagger - A Part-Of-Speech Tagger (POS Tagger
• Stanford Name Entity Recognizer - Stanford NER is a Java implementation of a Named Entity Recognizer.
• Stanford Word Segmenter - Tokenization of raw text is a standard pre-processing step for many NLP tasks.
• Tregex, Tsurgeon and Semgrex - Tregex is a utility for matching patterns in trees, based on tree relationships and regular expression matches on nodes (the name is short for "tree regular expressions").
• Stanford English Tokenizer - Stanford Phrasal is a state-of-the-art statistical phrase-based machine translation system, written in Java.
• Stanford Tokens Regex - A tokenizer divides text into a sequence of tokens, which roughly correspond to "words"
• Stanford Temporal Tagger - SUTime is a library for recognizing and normalizing time expressions.
• Stanford SPIED - Learning entities from unlabeled text starting with seed sets using patterns in an iterative fashion
• Stanford Topic Modeling Toolbox - Topic modeling tools to social scientists and others who wish to perform analysis on datasets
• Twitter Text Java - A Java implementation of Twitter's text processing library
• MALLET - A Java-based package for statistical natural language processing, document classification, clustering, topic modeling, information extraction, and other machine learning applications to text.
• OpenNLP - a machine learning based toolkit for the processing of natural language text.
• LingPipe - A tool kit for processing text using computational linguistics.
• ClearTK - ClearTK provides a framework for developing statistical natural language processing (NLP) components in Java and is built on top of Apache UIMA.
• Apache cTAKES - Apache clinical Text Analysis and Knowledge Extraction System (cTAKES) is an open-source natural language processing system for information extraction from electronic medical record clinical free-text.
• ClearNLP - The ClearNLP project provides software and resources for natural language processing. The project started at the Center for Computational Language and EducAtion Research, and is currently developed by the Center for Language and Information Research at Emory University. This project is under the Apache 2 license.
• CogcompNLP - This project collects a number of core libraries for Natural Language Processing (NLP) developed in the University of Illinois' Cognitive Computation Group, for example illinois-core-utilities which provides a set of NLP-friendly data structures and a number of NLP-related utilities that support writing NLP applications, running experiments, etc, illinois-edison a library for feature extraction from illinois-core-utilities data structures and many other packages.
• Twitter-text - A JavaScript implementation of Twitter's text processing library
• NLP.js - NLP utilities in javascript and coffeescript
• natural - General natural language facilities for node
• Knwl.js - A Natural Language Processor in JS
• Retext - Extensible system for analyzing and manipulating natural language
• TextProcessing - Sentiment analysis, stemming and lemmatization, part-of-speech tagging and chunking, phrase extraction and named entity recognition.
• NLP Compromise - Natural Language processing in the browser
• Topic Models - TopicModels for Julia
• Text Analysis - Julia package for text analysis
• NLP - An NLP library for Matlab
• Stanford.NLP for .NET - A full port of Stanford NLP packages to .NET and also available precompiled as a NuGet package.
• jieba-php - Chinese Words Segmentation Utilities.
• NLTK - A leading platform for building Python programs to work with human language data.
• Pattern - A web mining module for the Python programming language. It has tools for natural language processing, machine learning, among others.
• Quepy - A python framework to transform natural language questions to queries in a database query language
• TextBlob - Providing a consistent API for diving into common natural language processing (NLP) tasks. Stands on the giant shoulders of NLTK and Pattern, and plays nicely with both.
• YAlign - A sentence aligner, a friendly tool for extracting parallel sentences from comparable corpora.
• jieba - Chinese Words Segmentation Utilities.
• SnowNLP - A library for processing Chinese text.
• spammy - A library for email Spam filtering built on top of nltk
• loso - Another Chinese segmentation library.
• genius - A Chinese segment base on Conditional Random Field.
• KoNLPy - A Python package for Korean natural language processing.
• nut - Natural language Understanding Toolkit
• Rosetta - Text processing tools and wrappers (e.g. Vowpal Wabbit)
• BLLIP Parser - Python bindings for the BLLIP Natural Language Parser (also known as the Charniak-Johnson parser)
• PyNLPl - Python Natural Language Processing Library. General purpose NLP library for Python. Also contains some specific modules for parsing common NLP formats, most notably for FoLiA, but also ARPA language models, Moses phrasetables, GIZA++ alignments.
• python-ucto - Python binding to ucto (a unicode-aware rule-based tokenizer for various languages)
• python-frog - Python binding to Frog, an NLP suite for Dutch. (pos tagging, lemmatisation, dependency parsing, NER)
• python-zpar - Python bindings for ZPar, a statistical part-of-speech-tagger, constiuency parser, and dependency parser for English.
• spaCy - Industrial strength NLP with Python and Cython.
• PyStanfordDependencies - Python interface for converting Penn Treebank trees to Stanford Dependencies.
• Distance - Levenshtein and Hamming distance computation
• Fuzzy Wuzzy - Fuzzy String Matching in Python
• jellyfish - a python library for doing approximate and phonetic matching of strings.
• editdistance - fast implementation of edit distance
• textacy - higher-level NLP built on Spacy
• stanford-corenlp-python - Python wrapper for Stanford CoreNLP
• Ruby Linguistics - Linguistics is a framework for building linguistic utilities for Ruby objects in any language. It includes a generic language-independent front end, a module for mapping language codes into language names, and a module which contains various English-language utilities.
• Stemmer - Expose libstemmer_c to Ruby
• Ruby Wordnet - This library is a Ruby interface to WordNet
• Raspel - raspell is an interface binding for ruby
• UEA Stemmer - Ruby port of UEALite Stemmer - a conservative stemmer for search and indexing
• Twitter-text-rb - A library that does auto linking and extraction of usernames, lists and hashtags in tweets
• Contextual Analysis - Add structure to unstructured text using a GUI.
• Sentiment Analysis - Extract sentiment from text using a GUI.
• Text Miner - Text mining using a GUI or code.
• ScalaNLP - ScalaNLP is a suite of machine learning and numerical computing libraries.
• Breeze - Breeze is a numerical processing library for Scala.
• Chalk - Chalk is a natural language processing library.
• FACTORIE - FACTORIE is a toolkit for deployable probabilistic modeling, implemented as a software library in Scala. It provides its users with a succinct language for creating relational factor graphs, estimating parameters and performing inference.
• gibran - Gibran is an Elixir port of WordsCounted, a natural language processor that extracts useful statistics from text.
• Paasaa - Natural language detection for Elixir.
• Woolly - Woolly is an ambitious Text Mining and Natural Language Processing API for Elixir.
• DKPro - A collection of re-usable NLP tools for linguistic pre-processing, machine learning, lexical resources, etc.
• LingPipe - Toolkit for a variety of tasks ranging from POS tagging to sentiment analysis.