| Type | Filename  | Size | Date | |
|---|---|---|---|---|
| nlp_slides_part00_introduction.pdf Introduction and course organization. | 1.4 MB | 10/2/24 | ||
| nlp_slides_part01_ngrams.pdf n-gram language models, estimating probabilities from corpora, entropy, cross-entropy, perplexity, edit distance, context-aware spelling correction, beam-search decoding. | 3.28 MB | 10/6/24 | ||
| nlp_slides_part02_text_classification_with_mostly_linear_models.pdf Text classification with (mostly) linear models: Representing texts as bags of words. Boolean and TF-IDF features. Feature selection using information gain. Text classification with k-NN and Naive Bayes. Precision, recall, F1, AUC. Obtaining word embeddings from PMI scores using SVD-based dimensionality reduction. k-means. Linear and logistic regression, (stochastic) gradient descent. Practical advice and diagnostics for text classification with supervised machine learning. Optional slides: semi-supervised classification with Expectation Maximization (EM), lexicon-based features, sentiment lexica, Support Vector Machines (SVMs) and kernels. | 5.75 MB | 10/11/24 | ||
| nlp_slides_part03_text_classification_with_mlps.pdf Perceptrons, training them with SGD, limitations. Multi-Layer Perceptrons (MLPs) and backpropagation. MLPs for text classification, regression, token classification (e.g., for POS tagging, NER). Dropout, batch/layer normalization. Pre-training word embeddings with Word2Vec. Advice for training deep neural networks. | 2.85 MB | 10/25/24 | ||
| nlp_slides_part04_nlp_with_rnns.pdf Recurrent neural networks (RNNs), GRUs/LSTMs. Applications in token classification (e.g., named entity recognition). RNN language models. RNNs with self-attention and applications in text classification. Bidirectional and stacked RNNs. Obtaining word embeddings from character-based RNNs. Hierarchical RNNs. Sequence-to-sequence RNN models with attention, applications in machine translation. Optional slides: Universal sentence encoders, LASER. Pre-training RNN language models, ELMo. | 3.51 MB | 11/1/24 | ||
| nlp_slides_part05_nlp_with_cnns.pdf Quick background on Convolutional Neural Networks (CNNs) in Computer Vision. Image to text generation with CNN encoders and RNN decoders. Text processing with CNNs. | 2.31 MB | 11/15/24 |
