2nd Semester 2022/23: Meaning-to-text generation with GPT

Instructors

Jonas Groschwitz

ECTS

6

Description

Large language models like ChatGPT and GPT-4 have demonstrated a surprising amount of skill in performing several language based tasks, and have been celebrated as a revolution in the field of AI. However, they also have several questionable properties, such as making up facts without giving the user much cause for suspicion. One way to control language outputs more precisely could be to first determine the desired output in a more structured form such as Abstract Meaning Representation (AMR, Banarescu et al. 2013), and only in a second step generate text that captures the desired meaning. In this course, we will experimentally explore this second step, and test the abilities of the latest GPT models to faithfully translate AMRs into corresponding text. A key part of this experimental exploration will be prompt design, i.e. what exact form the question (prompt) for GPT should take, and what additional context (e.g. examples, or definitions/explanations of relevant words and AMR structure) proves to be helpful. There will also be some time to discuss philosophical dimensions, such as how much intelligence (or lack thereof) our experiments reveal, and what intelligence can even mean in the context of a language model.

Organisation

Students can work alone, or together in small groups. In the first week, we will have regular meetings to decide what approach each student/group will take to the task, while the students make themselves familiar with the formalisms, data and tools we will be using. For the rest of the month, the students/groups will work mostly independently on their project, with regular meetings to discuss progress, insights, and open questions/problems and how to approach them. The last week will be largely reserved for writing a report.

Prerequisites

None. Linguistic, mathematical and programming skills can be applied during the project, but the approach that each student/group takes can be tailored to fit their skills and experience.

Assessment

At the end of the month, students will submit a report on their experiments in the style of a research paper. Assessment is based on both content and clarity of presentation.

References

Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M. and Schneider, N., 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th linguistic annotation workshop and interoperability with discourse (pp. 178-186).