Home Instructors Course Materials TAs Contact Us


CS626: Speech, Natural Language Processing and the Web

Announcement

  • Join MS Teams course group using the code "s3eu6cs"
  • Previous iterations of the course: 2022 | 2021 | 2020

Course Details

CS626: Speech, Natural Language Processing and the Web
Department of Computer Science and Engineering
Indian Institute of Technology Bombay

Time Table and Venue

  • Monday: 05:30 PM to 06:55 PM
  • Thursday: 05:30 PM to 06:55 PM
  • Venue: LH301

Course Description

The general approach in the course will be covering (i) a language phenomenon, (ii) the corresponding language processing task, and (iii) techniques based on deep learning, classical machine learning and knowledge base. On one hand we will understand the language processing task in detail using linguistics, cognitive science, utility etc., on the other hand we will delve deep into techniques for solving the problem. The topics are given now.
  • Sound: Biology of Speech Processing; Place and Manner of Articulation; Peculiarities of Vowels and Consonants; Word Boundary Detection; Argmax based computations; Hidden Markov Model and Speech Recognition; deep neural nets for speech processing.
  • Morphology: Morphology fundamentals; Isolating, Inflectional, Agglutinative morphology; Infix, Prefix and Postfix Morphemes, Morphological Diversity of Indian Languages; Morphology Paradigms; Rule Based Morphological Analysis: Finite State Machine Based Morphology; Automatic Morphology Learning; Deep Learning based morphology analysis.
  • Shallow Parsing: Part of Speech (POS) Tagging; HMM based POS tagging; Maximum Entropy Models and POS; Random Fields and POS; DNN for POS.
  • Parsing: Constituency and Dependency Parsing; Theories of Parsing; Scope Ambiguity and Attachment Ambiguity Resolution; Rule Based Parsing Algorithms; Probabilistic Parsing; Neural Parsing.
  • Meaning: Lexical Knowledge Networks, Wordnet Theory and Indian Language Wordnets; Semantic Roles; Word Sense Disambiguation; Metaphors.
  • Discourse and Pragmatics: Coreference Resolution; Cohesion and Coherence.
  • Deep Learning and Large Language Models: GPT-4, LaMDA, LLAMA2, Pythia, etc. Power of LLMs: Complex Reasoning Abilities, Code Generation, Application to AI Assistants; Limitations: Biases, Alignment Problems, Cost of Training
  • Applications: Machine Translation; Sentiment and Emotion Analysis; Text Entailment; Question Answering; Code Mixing; Analytics and Social Networks, Information Retrieval and Cross Lingual Information Retrieval (IR and CLIR)

References

  • Allen, James, Natural Language Understanding, Second Edition, Benjamin/Cumming, 1995.
  • Charniack, Eugene, Statistical Language Learning, MIT Press, 1993
  • Jurafsky, Dan and Martin, James, Speech and Language Processing, Speech and Language Processing (3rd ed. draft), Draft chapters in progress, October 16, 2019.
  • Manning, Christopher and Heinrich, Schutze, Foundations of Statistical Natural Language Processing, MIT Press, 1999.
  • Jacob Eisenstein, Introduction to Natural Language Processing, MIT Press, 2019.
  • Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press, 2016.
  • Radford, Andrew et. al., Linguistics, an Introduction, Cambridge University Press, 1999.
  • Pushpak Bhattacharyya, Machine Translation, CRC Press, 2017.
  • Journals: Computational Linguistics, Natural Language Engineering, Machine Learning, Machine Translation, Artificial Intelligence
  • Conferences: Annual Meeting of the Association of Computational Linguistics (ACL), Computational Linguistics (COLING), European ACL (EACL), Empirical Methods in NLP (EMNLP), Annual Meeting of the Special Interest Group in Information Retrieval (SIGIR), Human Language Technology (HLT).

Pre-requisites

Data Structures and Algorithms, Python (or similar language) Programming skill

Course Instructors

Teaching Assistants

Lecture Slides

Lecture Topics Readings and useful links
Week 1
(Week of 1st August)
  • Introduction and Motivation
  • Applications of NLP: Radiology Report Automation
  • POS Tagging - Introduction
Week 2
(Week of 2nd August)
  • POS Tagging, HMM
  • Shallow Parsing
  • Math for NLP
Week 3
(Week of 3rd August)
  • Essential Linguistics for NLP
  • HMM and POS tagging
Week 4
(Week of 4th August)
  • HMM and POS tagging
Week 5
(Week of 5th August/1st September)
  • Discriminative PoS Tagging, Beam Search
  • HMM Training by EM
Week 6
(Week 2 of September)
  • Parsing
  • Parsing Algorithms
Week 7
(Week of 11th September)
  • Probabilistic Parsing
  • Dependency Parsing
Week 8
(Week of 25th September)
  • Dependency Parsing Cont.
  • Machine Translation
Week 9
(Week of 2nd October)
  • Machine Translation, APE, and QE
Week 10
(Week of 9th October)
  • Morphology, KGs, LLMs, and Pragmatics
  • Sentiment and Emotion Analysis
Week 11
(Week of 16th October)
  • Sentiment and Emotion Analysis Cont.
Week 12
(Week of 23rd October)
  • NLP Evaluation, Annotation, IAA
  • NLP Evaluation, Annotation, IAA Cont.
Week 13
(Week of 30th October)
  • BLEU, Hypotheis Testing
  • Chi-Square Distribution, Fitting distributions
Week 14
(Week of 6th November)
  • ML and Hypothesis testing, Fuzzy
  • Fuzzy Application, Closure

Lecture videos

Lecture videos are regularly uploaded on MSTeams. Lecture videos will also be available here.

Assignments

Date Assignment# Topic Deadline Link
Aug 15, 2023 Assignment 1 Inhabitant Term Prediction Continuous Evaluation Details
Sep 21, 2023 Assignment 2 POS Tagging Continuous Evaluation Details

Contact Us

CFILT Lab
Room Number: 401, 4th Floor, new CC building
Department of Computer Science and Engineering
Indian Institute of Technology Bombay
Mumbai 400076, India