Corpus: A corpus is a collection of texts, typically in digital format, used for language research and NLP tasks. A corpus may contain texts in a single language or multiple languages, and can consist of specific types of texts, such as news articles, scientific papers, social media posts, etc. Corpora are used for training and evaluating NLP models, helping models learn how to process and understand language.
For example, a well-known English corpus is the Brown Corpus, which includes texts from various categories such as news, religion, science, etc., consisting of approximately one million words. This enables researchers to test and train their models on diverse textual data.
Document: A document is an individual entity within a corpus, which can be an article, a chapter of a book, an email, a webpage, etc. In NLP tasks, the basic unit for processing is often the 'document'. Each document is independent and contains complete information that can be read and analyzed. The size and length of documents can vary, from short texts like SMS messages to full books.
For example, in sentiment analysis tasks, each product review can be considered a separate document. NLP models analyze the textual content of each document to determine whether the sentiment is positive or negative.
In summary, a corpus is a collection of documents used for training and testing NLP models, while a document is an individual text unit that constitutes the corpus and can be used for specific data processing and analysis. These two concepts complement each other and support various applications and research in NLP.