CS 521: Statistical Natural Language Processing

Spring 2021

Contact Information

Professor: Natalie Parde (parde@uic.edu)
Office Hours: Tuesday 9:30 - 10:30 a.m. / Thursday 3:00 - 4:00 p.m.
 
Piazza: piazza.com/uic/spring2021/2021springcs52143232

What is this class about?

Natural language processing (NLP) is increasingly driven by statistical and neural methods. These techniques are leveraged in many everyday applications, including intelligent virtual assistants (e.g., Siri or Alexa), machine translation systems (e.g., Google Translate), and language models (such as those used for predictive text). This class will introduce advanced topics in statistical and neural NLP, and provide an overview of active research in those topic areas, through a combination of readings, paper presentations and critiques, and a semester-long project. Topics covered will include data collection, common neural architectures for NLP, machine translation, question answering, automated speech recognition, natural language generation, bias in NLP, and NLP applications, among others.

Textbooks

Readings for the first part of the semester will be drawn from the following source:
- Daniel Jurafsky and James H Martin. Speech and Language Processing (3rd Edition). Draft, 2020.

Note that this resource is a draft of the upcoming third edition of Speech and Language Processing. Chapter order and content is subject to change throughout the semester (I'll try to update the course website whenever I notice this occurring, but feel free to ping me if you think something seems out of date). Readings for the second part of the semester will be drawn primarily from journals and conference proceedings. Some suggested papers for each topic in the second part of the semester are provided in the course syllabus. You are welcome to present a paper that is not in the list of suggestions, as long as I approve it.

Assignments

This course acts as a middle ground between UIC's introductory, lecture-based NLP course (CS 421) and the advanced, seminar-style NLP courses (CS 532) offered by the department. The coursework will be closer to what you would expect to find in a seminar-style course, but the general format of the class will contain elements of both. For the first portion of the semester, lectures will be given about different statistical and neural techniques that are common in natural language processing, both at a fundamental level (e.g., deep learning architectures) and for specific applications (e.g., question answering). For the second portion of the semester, students will present and critique research from recent NLP papers. Some further details about the work you will be expected to complete for this course are provided below: Grading rubrics will be posted on Blackboard. Final course grades will be determined according to the following breakdown:

Schedule

The most recent version of the course schedule is available below. This schedule is subject to change ...check back regularly for updates! All deadlines are at 12 p.m. (noon) unless otherwise stated.

Week Topic Readings Deliverables Video Links
1/11 - 1/15 Introduction and Data Collection
1/18 - 1/22 Feedforward and Convolutional Neural Networks Chapter 7
1/25 - 1/29 Deep Learning Architectures for Sequence Processing Chapter 9 1/25: Paper Critique (Topic from 1/11 - 1/29)
2/1 - 2/5 Machine Translation Chapter 11 2/1: Paper Selection Deadline
2/8 - 2/12 Question Answering Chapter 23 2/8: Paper Critique (Topic from 1/11 - 2/12)
2/15 - 2/19 Automatic Speech Recognition and Text-to-Speech Chapter 26
2/22 - 2/26 Project Proposals 2/22: Project Proposal
3/1 - 3/5 Contextual Word Embeddings Research Papers 3/1: Paper Critique
3/8 - 3/12 Sustainable NLP Research Papers 3/8: Paper Critique
3/15 - 3/19 Low-Resource Languages Research Papers 3/15: Paper Critique
3/22 - 3/26 Spring Break
3/29 - 4/2 Natural Language Generation Research Papers 3/29: Paper Critique
4/5 - 4/9 Multimodal NLP Research Papers 4/5: Paper Critique
4/12 - 4/16 Bias in NLP Research Papers 4/12: Paper Critique
4/19 - 4/23 NLP Applications Research Papers 4/19: Paper Critique
4/26 - 4/30 Project Videos 4/26: Project Video and Source
5/3 - 5/7 Finals Week (No Class) 5/3: Project Report

Final Notes

This website is provided partially for student convenience, partially for my own record-keeping purposes, and partially for the benefit of others who are not able to enroll in the course but who may find the content interesting for one reason or another. It is not a substitute for the course page on Blackboard or the course discussion board on Piazza! Please refer to those sources for copies of the full syllabus, assignments, grading rubrics, submission links, and other useful information. If you are not enrolled in the course but would like to request access to those materials, please send me an email introducing yourself and explaining why you would like to have access to them.

Happy studying!