-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathmtcourse.html
More file actions
120 lines (119 loc) · 4.53 KB
/
Copy pathmtcourse.html
File metadata and controls
120 lines (119 loc) · 4.53 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" href="styles.css">
</head>
<title>
HomePage of Machine Translation and Natural Language Generation
</title>
<h1 id="machine-translation-and-natural-language-generation">Machine
Translation and Natural Language Generation</h1>
<p>Vision: Making Communication Easier among All Languages & Making
Techniques Available for All Languages</p>
<hr />
<h2 id="course-description">Course Description</h2>
<p>The course is designed for beginners in Natural Language Generation,
Natural Language Processing and Machine Translation. The main aim is to
exploring the theories and methods in automatically understanding and
generating natural language text, with special focus on
multilingualism.</p>
<p>The participation requires basic knowledge of machine learning.
Please consider taking online courses from Coursera, or watch online
videos.</p>
<p>The course is taught every Spring semaster in School of Computer
Science, Nanjing University, since 2020. It is firstly designed for
graduate students, and then opened for both graduate and undergraduate
students.</p>
<hr />
<h2 id="objectives">Objectives</h2>
<ul>
<li>Understand the fundamental concepts, challenges and applications in
Natural Language Processing/Generation.</li>
<li>Learn the evolution of research in Mutilingualism, including machine
tralsation, multiligual models, etc.</li>
<li>Practice the design, training and application of natural language
generation models.</li>
</ul>
<hr />
<h2 id="outline">Outline</h2>
<h3 id="introduction">1. Introduction</h3>
<ol type="1">
<li>Problems in Natural Language Processing</li>
<li>NLP as Classifications</li>
<li>NLP as Structured Predictions</li>
<li>Natural Language Generation</li>
</ol>
<h3 id="language-models">2. Language Models</h3>
<ol type="1">
<li>Probabilistic Modeling of Natural Language</li>
<li>Statistical Language Models</li>
<li>Neural Language Models and Pretraining</li>
<li>Language Language Models (LLMs)</li>
<li>LLMs and Reinforcement Learning</li>
</ol>
<h3 id="machine-translation">3. Machine Translation</h3>
<ol type="1">
<li>Traditional Machine Translation (Rule-based Machine Translation,
Statistical Machine Translation)</li>
<li>Machine Translation and Deep Learning</li>
<li>Machine Translation and LLMs</li>
<li>*Machine Translation with Less Parallel Data (Low-resource,
Unsupervised Machine Translation)</li>
<li>*Non-Autoregressive Machine Translation (Parallel Generation)</li>
<li>*Interactive Machine Translation (Human-involved Generation)</li>
<li>*Translation Quality Evaluation (Cross-task Knowledge
Transfering)</li>
</ol>
<h3 id="multilingual-llms">4. Multilingual LLMs</h3>
<ol type="1">
<li>Evaluation of Multilinguality</li>
<li>Extending LLMs to New Languages</li>
<li>Aligning Language Abilities among Different Languages</li>
<li>Understanding Multilinguality in LLMs</li>
</ol>
<h3 id="other-topics-in-llms">5. Other Topics in LLMs</h3>
<ol type="1">
<li>Reasoning Abilities in LLMs</li>
<li>Alignment and Safety in LLMs</li>
<li>*LLMs for Biology (Generating Protains)</li>
<li>*Multi-modal Large Langauge Models</li>
</ol>
<h3 id="other-generation-tasks">6. Other Generation Tasks</h3>
<ol type="1">
<li>Summarization: Content Selection</li>
<li>Paraphrase: Semantical Equivalence (Variational Auto-Encoders)</li>
<li>Style Transfer: Controlled Generation (Generative Adversarial
Networks)</li>
<li>Image Captioning: Multi-modal Interaction</li>
</ol>
<p>Note: *marks topics that may change in different semasters.</p>
<hr />
<h2 id="assignments-and-assessment">Assignments and Assessment</h2>
<ul>
<li>Assignment 1: interacting with Language Models</li>
<li>Assignment 2: research on Large Language Models</li>
<li>Assignment 3: research on Machine Translation and
Multilingualism</li>
<li>Final Project: implementation, experiments and discussions on
self-selected topics</li>
</ul>
<p>Assessment involves both written report and in-class presentation and
discussion.</p>
<!-- ## References
- [Link 1](#)
- [Link 2](#)
- [Link 3](#) -->
<hr />
<h2 id="instructor-contact-information">Instructor Contact
Information</h2>
<ul>
<li>Shujian Huang (<a href="index.html">homepage</a>)</li>
<li>Email: huangsj at nju dot edu dot cn</li>
</ul>
<hr />
<h2 id="history">History</h2>
<p><a href="mtcourse.2024.html">MTNLG2024</a></p>
<hr />
<h2 id="acknowledgement">Acknowledgement</h2>
<p>The course is constantly improved with the help from wonderful
teaching assistants: Zaixiang Zheng(2020), Yu Bao(2021), Jiahuan
Li(2022), Wenhao Zhu(2023), Changjiang Gao(2024), Peng Ding(2025)</p>