N-grams data

Corpus of Contemporary American English


 Purchase data 

Overview
Compare to Google
Processing the data

Samples (COCA)
   Level 1 (free)
   Level 2
   Level 3

Historical (COHA)
Free (1 million)

Spanish data
Portuguese data

Related sites
  Full-text data 
  Word frequency
  Collocates
  WordAndPhrase
  Academic vocabulary
  corpus.byu.edu

Contact us


This page briefly describes the format of the n-grams, and how you might search them on your computer.

Most of the n-grams are in a ZIP file, with 26 different files -- for each letter A-Z. After unzipping the files, you'll probably want to concatenate (join) these together into one big file. To do this on a Windows machine, open a command line window (Start / Run / cmd), change the directory ( cd ) to the folder with the files (e.g. cd c:\myNgrams\ ), and type:

copy *.txt nameOfFile.txt

(where "nameOfFile.txt" is the name you want to give it). Each file has a format like the following, with a tab between frequency, each word, and each part of speech:
 

freq

word1

word2

word3

PoS1

Pos2

PoS3

22

filled

with

fluid

vvn

iw

nn1

6

filled

with

flying

vvn

iw

jj

4

filled

with

foam

vvn

iw

nn1

In each case, the leftmost column is the frequency of the n-grams. This is followed by one column for each word (i.e. four columns in the 4-gram file). If the n-gram set that you're using has part of speech, there is one column on the right side for each of the "word" columns on the left side -- in the same order. Finally, if you are using the COHA (historical English) files, the leftmost column will be the frequency in a particular decade, and the rightmost column (a number) will be the decade: 1-20 (1810s-2000s).

Many of you will want to immediately take the files and import them into a robust relational database, like mySQL or SQL Server (that's what I'd do too). But you can also just search the files using a robust text editor. One that I like for Windows is the shareware program TextPad, which can easily handle even files 100 MB or more in size. One of the advantages of TextPad is that it does regular expressions, for advanced pattern matching (but note: regular expressions are a little different (read: weird) in TextPad, and lots of other programs do regular expressions in a more "regular" way).

If you're using TextPad, you'll want to read their help files on regular expressions, but here are a few simple examples.

What you want to find

search

what it means

NOUN + NOUN

nn.*\tnn.*$

Two part of speech tags starting with nn (the code for nouns), each followed by anything else (since it could be nn1 = singular noun, nn2 = plural noun, etc), separated by a tab, and  end of line

*ed word + d* word

^[[:digit:]]+\t[^\t]+ed\td

Beginning of line , followed by one or more numbers (the frequency count), followed by word ending in ed, tab, and word starting with d

VERB the NOUN

^[[:digit:]]+\t[^\t]+\tthe\t[^\t]\t
\tvv0\t[^\t]+\tnn.*$

Beginning of line , followed by one or more numbers (the frequency count), followed by the first word (i.e.  any number of characters that aren't tabs, tab, the, tab, third word, tab, verb (base form), tab, any part of speech, tab, noun, end of line