Podcasts by NLP Highlights
Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. The hosts are the members of the AllenNLP team at Allen Institute for AI. All views expressed belong to the hosts and guests and do not represent their employers.
Further podcasts by Allen Institute for Artificial Intelligence
Podcast on the topic Wissenschaft
All episodes
134 - PhD Application Series: PhDs in Europe versus the US, with Barbara Plank and Gonçalo Correia from 2021-10-19T17:16:57
This episode is the second in our current series on PhD applications.
How do PhD programs in Europe differ from PhD programs in the US, and how should people decide between them? In this e...
134 - PhD Application Series: PhDs in Europe versus the US from 2021-10-19T17:16:57
This episode is the second in our current series on PhD applications.
How do PhD programs in Europe differ from PhD programs in the US, and how should people decide between them? In this e...
133 - PhD Application Series: Preparing Application Materials, with Nathan Schneider and Roma Patel from 2021-10-06T00:32:41
This episode is the first in our current series on PhD applications.
How should people prepare their applications to PhD programs in NLP? In this episode, we invite Nathan Schneider (Profe...
132 - Alexa Prize Socialbot Grand Challenge and Alquist 4.0, with Petr Marek from 2021-09-27T22:22:12
In this episode, we discussed the Alexa Prize Socialbot Grand Challenge and this year's winning submission, Alquist 4.0, with Petr Marek, a member of the winning team. Petr gave us an overview of t...
Listen
131 - Opportunities and Barriers between HCI and NLP, with Nanna Inie and Leon Derczynski from 2021-08-20T21:40:54
What can NLP researchers learn from Human Computer Interaction (HCI) research? We chatted with Nanna Inie and Leon Derczynski to find out. We discussed HCI's research processes including methods of...
Listen
130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn from 2021-08-09T16:23:37
In this episode, we talk with Lisa Beinborn, an assistant professor at Vrije Universiteit Amsterdam, about how to use human cognitive signals to improve and analyze NLP models. We start by discussi...
Listen
129 - Transformers and Hierarchical Structure, with Shunyu Yao from 2021-07-02T20:19:06
In this episode, we talk to Shunyu Yao about recent insights into how transformers can represent hierarchical structure in language. Bounded-depth hierarchical structure is thought to be a key feat...
Listen
128 - Dynamic Benchmarking, with Douwe Kiela from 2021-06-19T00:40:01
We discussed adversarial dataset construction and dynamic benchmarking in this episode with Douwe Kiela, a research scientist at Facebook AI Research who has been working on a dynamic benchmarking ...
Listen
127 - Masakhane and Participatory Research for African Languages, with Tosin Adewumi and Perez Ogayo from 2021-06-08T21:48:02
We invited members of Masakhane, Tosin Adewumi and Perez Ogayo, to talk about their EMNLP Findings paper that discusses why typical research is limited for low-resourced NLP and how participatory r...
Listen
126 - Optimizing Continuous Prompts for Generation, with Lisa Li from 2021-05-24T17:42:26
We invited Lisa Li to talk about her recent work, Prefix-Tuning: Optimizing Continuous Prompts for Generation. Prefix tuning is a lightweight alternative to finetuning, and the idea is to tune only...
Listen
125 - VQA for Real Users, with Danna Gurari from 2021-05-04T00:51:34
How can we build Visual Question Answering systems for real users? For this episode, we chatted with Danna Gurari, about her work in building datasets and models towards VQA for people who are blin...
Listen
124 - Semantic Machines and Task-Oriented Dialog, with Jayant Krishnamurthy and Hao Fang from 2021-04-14T00:12:56
We invited Jayant Krishnamurthy and Hao Fang, researchers at Microsoft Semantic Machines to discuss their platform for building task-oriented dialog systems, and their recent TACL paper on the topi...
Listen
123 - Robust NLP, with Robin Jia from 2021-04-05T22:56:12
In this episode, Robin Jia talks about how to build robust NLP systems. We discuss the different senses in which a system can be robust, reasons to care about system robustness, and the challenges ...
Listen
122 - Statutory Reasoning in Tax Law, with Nils Holzenberger from 2020-11-12T01:10:38
We invited Nils Holzenberger, a PhD student at JHU to talk about a dataset involving statutory reasoning in tax law Holzenberger et al. released recently. This dataset includes difficult textual en...
Listen
121 - Language and the Brain, with Alona Fyshe from 2020-10-30T17:41:58
We invited Alona Fyshe to talk about the link between NLP and the human brain. We began by talking about what we currently know about the connection between representations used in NLP and represen...
Listen
120 - Evaluation of Text Generation, with Asli Celikyilmaz from 2020-10-03T00:36:35
We invited Asli Celikyilmaz for this episode to talk about evaluation of text generation systems. We discussed the challenges in evaluating generated text, and covered human and automated metrics, ...
Listen
119 - Social NLP, with Diyi Yang from 2020-09-03T22:26:32
In this episode, Diyi Yang gives us an overview of using NLP models for social applications, including understanding social relationships, processes, roles, and power. As NLP systems are getting u...
Listen
118 - Coreference Resolution, with Marta Recasens from 2020-08-26T21:53:36
In this episode, we talked about Coreference Resolution with Marta Recasens, a Research Scientist at Google. We discussed the complexity involved in resolving references in language, the simplifica...
Listen
117 - Interpreting NLP Model Predictions, with Sameer Singh from 2020-08-13T00:41:36
We interviewed Sameer Singh for this episode, and discussed an overview of recent work in interpreting NLP model predictions, particularly instance-level interpretations. We started out by talking ...
Listen
116 - Grounded Language Understanding, with Yonatan Bisk from 2020-07-03T01:13:22
We invited Yonatan Bisk to talk about grounded language understanding. We started off by discussing an overview of the topic, its research goals, and the the challenges involved. In the latter half...
Listen
115 - AllenNLP, interviewing Matt Gardner from 2020-06-17T20:16:31
In this special episode, Carissa Schoenick, a program manager and communications director at AI2 interviewed Matt Gardner about AllenNLP. We chatted about the origins of AllenNLP, the early challen...
Listen
114 - Behavioral Testing of NLP Models, with Marco Tulio Ribeiro from 2020-05-26T22:15:28
We invited Marco Tulio Ribeiro, a Senior Researcher at Microsoft, to talk about evaluating NLP models using behavioral testing, a framework borrowed from Software Engineering. Marco describes three...
Listen
113 - Managing Industry Research Teams, with Fernando Pereira from 2020-05-22T21:41:08
We invited Fernando Pereira, a VP and Distinguished Engineer at Google, where he leads NLU and ML research, to talk about managing NLP research teams in industry. Topics we discussed include priori...
Listen
112 - Alignment of Multilingual Contextual Representations, with Steven Cao from 2020-05-13T20:25:53
We invited Steven Cao to talk about his paper on multilingual alignment of contextual word embeddings. We started by discussing how multilingual transformers work in general, and then focus on Stev...
Listen
111 - Typologically diverse, multi-lingual, information-seeking questions, with Jon Clark from 2020-04-27T15:21:46
We invited Jon Clark from Google to talk about TyDi QA, a new question answering dataset, for this episode. The dataset contains information seeking questions in 11 languages that are typologically...
Listen
110 - Natural Questions, with Tom Kwiatkowski and Michael Collins from 2020-04-06T18:52:50
In this episode, Tom Kwiatkowski and Michael Collins talk about Natural Questions, a benchmark for question answering research. We discuss how the dataset was collected to reflect naturally-occurri...
Listen
109 - What Does Your Model Know About Language, with Ellie Pavlick from 2020-03-30T17:09:27
How do we know, in a concrete quantitative sense, what a deep learning model knows about language? In this episode, Ellie Pavlick talks about two broad directions to address this question: structur...
Listen
108 - Data-To-Text Generation, with Verena Rieser and Ondr?ej Dus?ek from 2020-03-23T17:08:37
In this episode we invite Verena Rieser and Ondr?ej Dus?ek on to talk to us about the complexities of generating natural language when you have some kind of structured meaning representation as inp...
Listen
107 - Multi-Modal Transformers, with Hao Tan and Mohit Bansal from 2020-02-24T17:29:56
In this episode, we invite Hao Tan and Mohit Bansal to talk about multi-modal training of transformers, focusing in particular on their EMNLP 2019 paper that introduced LXMERT, a vision+language tr...
Listen
106 - Ethical Considerations In NLP Research, with Emily Bender from 2020-02-17T21:55:43
In this episode, we talked to Emily Bender about the ethical considerations in developing NLP models and putting them in production. Emily cited specific examples of ethical issues, and talked abou...
Listen
105 - Question Generation, with Sudha Rao from 2020-02-10T20:15:47
In this episode we invite Sudha Rao to talk about question generation. We talk about different settings where you might want to generate questions: for human testing scenarios (rare), for data aug...
Listen
104 - Model Distillation, with Victor Sanh and Thomas Wolf from 2020-02-03T20:47:03
In this episode we talked with Victor Sanh and Thomas Wolf from HuggingFace about model distillation, and DistilBERT as one example of distillation. The idea behind model distillation is compressin...
Listen
103 - Processing Language in Social Media, with Brendan O'Connor from 2020-01-27T17:01:07
We talked to Brendan O’Connor for this episode about processing language in social media. Brendan started off by telling us about his projects that studied the linguistic and geographical patterns ...
Listen
102 - Biomedical NLP research at the National Institute of Health with Dina Demner-Fushman from 2020-01-20T17:25:22
What exciting NLP research problems are involved in processing biomedical and clinical data? In this episode, we spoke with Dina Demner-Fushman, who leads NLP and IR research at the Lister Hill Nat...
Listen
101 - The lottery ticket hypothesis, with Jonathan Frankle from 2020-01-14T17:52:32
In this episode, Jonathan Frankle describes the lottery ticket hypothesis, a popular explanation of how over-parameterization helps in training neural networks. We discuss pruning methods used to u...
Listen
100 - NLP Startups, with Oren Etzioni from 2020-01-08T21:09
For our 100th episode, we invite AI2 CEO Oren Etzioni to talk to us about NLP startups. Oren has founded several successful startups, is himself an investor in startups, and helps with AI2's start...
Listen
99 - Evaluating Protein Transfer Learning, With Roshan Rao And Neil Thomas from 2019-12-16T17:36:42
For this episode, we chatted with Neil Thomas and Roshan Rao about modeling protein sequences and evaluating transfer learning methods for a set of five protein modeling tasks. Learning representat...
Listen
98 - Analyzing Information Flow In Transformers, With Elena Voita from 2019-12-09T17:51:08
What function do the different attention heads serve in multi-headed attention models? In this episode, Lena describes how to use attribution methods to assess the importance and contribution of d...
Listen
97 - Automated Analysis Of Historical Printed Documents, With Taylor Berg-Kirkpatrick from 2019-11-27T22:08:19
In this episode, we talk to Taylor Berg-Kirkpatrick about optical character recognition (OCR) on historical documents. Taylor starts off by describing some practical issues related to old scanning ...
Listen
96 - Question Answering as an Annotation Format, with Luke Zettlemoyer from 2019-11-12T22:36:42
In this episode, we chat with Luke Zettlemoyer about Question Answering as a format for crowdsourcing annotations of various semantic phenomena in text. We start by talking about QA-SRL and QAMR, t...
Listen
95 - Common sense reasoning, with Yejin Choi from 2019-10-07T16:04:24
In this episode, we invite Yejin Choi to talk about common sense knowledge and reasoning, a growing area in NLP. We start by discussing a working definition of “common sense” and the practical uti...
Listen
94 - Decompositional Semantics, with Aaron White from 2019-09-30T22:33:41
In this episode, Aaron White tells us about the decompositional semantics initiative (Decomp), an attempt to re-think the prototypical approach to semantic representation and annotation. The basic ...
Listen
93 - NLP/ML for clinical data, with Alistair Johnson from 2019-07-22T22:53:23
In this episode, we invite Alistair Johnson to discuss the main challenge in applying NLP/ML to clinical domains: the lack of data. We discuss privacy concerns, de-identification, synthesizing reco...
Listen
92 - Computational Humanities, with David Bamman from 2019-07-05T19:54:46
In this episode, we invite David Bamman to give an overview of computational humanities. We discuss examples of questions studied in computational humanities (e.g., characterizing fictionality, ass...
Listen
91 - (Executable) Semantic Parsing, with Jonathan Berant from 2019-06-26T16:32:03
In this episode, we invite Jonathan Berant to talk about executable semantic parsing. We discuss what executable semantic parsing is and how it differs from related tasks such as semantic dependenc...
Listen
90 - Research in Academia versus Industry, with Philip Resnik and Jason Baldridge from 2019-05-31T14:54:30
How is it like to do research in academia vs. industry? In this episode, we invite Jason Baldridge (UT Austin => Google) and Philip Resnik (Sun Microsystems => UMD) to discuss some of the aspects o...
Listen
89 - Dialog Systems, with Zhou Yu from 2019-05-31T14:04
In this episode, we invite Zhou Yu to give an overview of dialogue systems. We discuss different types of dialogue systems (task-oriented vs. non-task-oriented), the main building blocks and how th...
Listen
88 - A Structural Probe for Finding Syntax in Word Representations, with John Hewitt from 2019-05-07T22:05:46
In this episode, we invite John Hewitt to discuss his take on how to probe word embeddings for syntactic information. The basic idea is to project word embeddings to a vector space where the L2 dis...
Listen
87 - Pathologies of Neural Models Make Interpretation Difficult, with Shi Feng from 2019-04-25T16:35:50
In this episode, Shi Feng joins us to discuss his recent work on identifying pathological behaviors of neural models for NLP tasks. Shi uses input word gradients to identify the least important wor...
Listen
86 - NLP for Evidence-based Medicine, with Byron Wallace from 2019-04-15T19:03:53
In this episode, Byron Wallace tells us about interdisciplinary work between evidence based medicine and natural language processing. We discuss extracting PICO frames from articles describing clin...
Listen
85 - Stress in Research, with Charles Sutton from 2019-03-29T16:34:12
In this episode, Charles Sutton walks us through common sources of stress for researchers and suggests coping strategies to maintain your sanity. We talk about how pursuing a research career is sim...
Listen
84 - Large Teams Develop, Small Groups Disrupt, with Lingfei Wu from 2019-03-26T17:42:16
In a recent Nature paper, Lingfei Wu (Ling) suggests that smaller teams of scientists tend to do more disruptive work. In this episode, we invite Ling to discuss their results, how they define disr...
Listen
83 - Knowledge Base Construction, with Sebastian Riedel from 2019-03-13T19:00:10
In this episode, we invite Sebastian Riedel to talk about knowledge base construction (KBC). Why is it an important research area? What are the tradeoffs between using an open vs. closed schema? Wh...
Listen
82 - Visual Reasoning, with Yoav Artzi from 2019-03-06T16:25:24
In this episode, Yoav Artzi joins us to talk about visual reasoning. We start by defining what visual reasoning is, then discuss the pros and cons of different tasks and datasets. We discuss some o...
Listen
81 - BlackboxNLP, with Afra Alishahi and Tal Linzen from 2019-02-06T16:42:59
Neural models recently resulted in large performance improvements in various NLP problems, but our understanding of what and how the models learn remains fairly limited. In this episode, Tal Linzen...
Listen
80 - Leaderboards and Science, with Siva Reddy from 2019-01-29T17:58:16
Originally used to entice fierce competitions in arcade games, leaderboards recently made their way into NLP research circles. Leaderboards could help mitigate some of the problems in how researche...
Listen
79 - The glass ceiling in NLP, with Natalie Schluter from 2019-01-21T22:49:31
In this episode, Natalie Schluter talks to us about a data-driven analysis of career progression of male vs. female researchers in NLP through the lens of mentor-mentee networks based on ~20K paper...
Listen
78. Where do corpora come from?, with Matt Honnibal and Ines Montani from 2019-01-15T02:21:33
Most NLP projects rely crucially on the quality of annotations used for training and evaluating models. In this episode, Matt and Ines of Explosion AI tell us how Prodigy can improve data annotatio...
Listen
77. On Writing Quality Peer Reviews, with Noah A. Smith from 2019-01-07T21:41:33
It's not uncommon for authors to be frustrated with the quality of peer reviews they receive in (NLP) conferences. In this episode, Noah A. Smith shares his advice on how to write good peer reviews...
Listen
76 - Increasing In-Class Similarity by Retrofitting Embeddings with Demographics, with Dirk Hovy from 2018-11-27T16:08:34
EMNLP 2018 paper by Dirk Hovy and Tommaso Fornaciari. https://www.semanticscholar.org/paper/Improving-Author-Attribute-Prediction-by-Linguistic-Hovy-Fornaciari/71aad8919c864f73108aafd8e926d44e9df51...
Listen
75 - Reinforcement / Imitation Learning in NLP, with Hal Daumé III from 2018-11-21T17:53:58
In this episode, we invite Hal Daumé to continue the discussion on reinforcement learning, focusing on how it has been used in NLP. We discuss how to reduce NLP problems into the reinforcement lear...
Listen
74 - Deep Reinforcement Learning Doesn't Work Yet, with Alex Irpan from 2018-11-16T23:56:35
Blog post by Alex Irpan titled "Deep Reinforcement Learning Doesn't Work Yet"
https://www.alexirpan.com/2018/02/14/rl-hard.html
In this episode, Alex Irpan talks about limitations of c...
73 - Supersense Disambiguation of English Prepositions and Possessives, with Nathan Schneider from 2018-11-13T19:43:23
ACL 2018 paper by Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Jakob Prange, Austin Blodgett, Sarah R. Moeller, Aviram Stern, Adi Bitan, Omri Abend.
In this episode, Nathan discusses h...
72 - The Anatomy Question Answering Task, with Jordan Boyd-Graber from 2018-10-16T21:07:10
Our first episode in a new format: broader surveys of areas, instead of specific discussions on individual papers. In this episode, we talk with Jordan Boyd-Graber about question answering.
71 - DuoRC: Complex Language Understanding with Paraphrased Reading Comprehension, with Amrita Saha from 2018-10-12T17:39:40
ACL 2018 paper by Amrita Saha, Rahul Aralikatte, Mitesh M. Khapra, Karthik Sankaranarayanan
Amrita and colleagues at IBM Research introduced a harder dataset for "reading comprehension", w...
70 - Measuring the Evolution of a Scientific Field through Citation Frames, with David Jurgens from 2018-09-18T19:43:18
TACL 2018 paper (presented at ACL 2018) by David Jurgens, Srijan Kumar, Raine Hoover, Daniel A. McFarland, and Daniel Jurafsky
David comes on the podcast to talk to us about citation frame...
69 - Second language acquisition modeling, with Burr Settles from 2018-09-10T16:53:56
A shared task held in conjunction with a NAACL 2018 workshop, organized by Burr Settles and collaborators at Duolingo.
Burr tells us about the shared task. The goal of the task was to pre...
68 - Neural models of factuality, with Rachel Rudinger from 2018-09-04T16:14:50
NAACL 2018 paper, by Rachel Rudinger, Aaron Steven White, and Benjamin Van Durme
Rachel comes on to the podcast, telling us about what factuality is (did an event happen?), what datasets e...
67 - GLUE: A Multi-Task Benchmark and Analysis Platform, with Sam Bowman from 2018-08-27T18:06:26
Paper by Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.
Sam comes on to tell us about GLUE. We talk about the motivation behind setting up a benc...
66 - Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods, with Jieyu Zhao from 2018-08-20T16:31:48
NACL 2018 paper, by Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang.
Jieyu comes on the podcast to talk about bias in coreference resolution models. This bias ma...
65 - Event Representations with Tensor-based Compositions, with Niranjan Balasubramanian from 2018-08-13T21:00:13
AAAI 2018 paper by Noah Weber, Niranjan Balasubramanian, and Nathanael Chambers
Niranjan joins us on the podcast to tell us about his latest contribution in a line of work going back to Sh...
64 - Neural Network Models for Sentence Pair Tasks, with Wuwei Lan and Wei Xu from 2018-08-08T16:20:57
Best reproduction paper at COLING 2018, by Wuwei Lan and Wei Xu.
This paper takes a bunch of models for sentence pair classification (including paraphrase identification, semantic textual ...
63 - Neural Lattice Language Models, with Jacob Buckman from 2018-08-02T19:20:14
TACL 2018 paper by Jacob Buckman and Graham Neubig.
Jacob tells us about marginalizing over latent structure in a sentence by doing a clever parameterization of a lattice with a model kin...
62 - Sounding Board: A User-Centric and Content-Driven Social Chatbot, with Hao Fang from 2018-07-30T21:51:02
NAACL 2018 demo paper, by Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A. Smith, and Mari Ostendorf
Sounding Board was the system that won the 2017 Am...
61 - Neural Text Generation in Stories, with Elizabeth Clark and Yangfeng Ji from 2018-07-23T21:43:19
NAACL 2018 Outstanding Paper by Elizabeth Clark, Yangfeng Ji, and Noah A. Smith
Both Elizabeth and Yangfeng come on the podcast to tell us about their work. This paper is an extension of ...
60 - FEVER: a large-scale dataset for Fact Extraction and VERification, with James Thorne from 2018-06-28T18:12
NAACL 2018 paper by James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal
James tells us about his paper, where they created a dataset for fact checking. We talk ab...
59 - Weakly Supervised Semantic Parsing With Abstract Examples, with Omer Goldman from 2018-06-12T23:02:16
ACL 2018 paper by Omer Goldman, Veronica Latcinnik, Udi Naveh, Amir Globerson, and Jonathan Berant
Omer comes on to tell us about a class project (done mostly by undergraduates!) that made...
58 - Learning What’s Easy: Fully Differentiable Neural Easy-First Taggers, with André Martins from 2018-06-08T19:21:56
EMNLP 2017 paper by André F. T. Martins and Julia Kreutzer
André comes on the podcast to talk to us the paper. We spend the bulk of the time talking about the two main contributions of th...
57 - A Survey Of Cross-lingual Word Embedding Models, with Sebastian Ruder from 2018-06-05T16:22:34
Upcoming JAIR paper by Sebastian Ruder, Ivan Vuli?, and Anders Søgaard.
Sebastian comes on to tell us about his survey. He creates a typology of cross-lingual word embedding methods, and ...
56 - Deep contextualized word representations, with Matthew Peters from 2018-04-04T21:22:36
NAACL 2018 paper, by Matt Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Chris Clark, Kenton Lee, and Luke Zettlemoyer.
In this episode, AI2's own Matt Peters comes on the show to talk a...
55 - Matchbox: Dispatch-driven autobatching for imperative deep learning, with James Bradbury from 2018-03-28T20:12:51
In this episode, we take a more systems-oriented approach to NLP, looking at issues with writing deep learning code for NLP models. As a lot of people have discovered over the last few years, effi...
Listen
54 - Simulating Action Dynamics with Neural Process Networks, with Antoine Bosselut from 2018-03-26T19:00:48
ICLR 2018 paper, by Antoine Bosselut, Omer Levy, Ari Holtzman, Corin Ennis, Dieter Fox, and Yejin Choi.
This is not your standard NLP task. This work tries to predict which entities chang...
53 - Classical Structured Prediction Losses for Sequence to Sequence Learning, with Sergey and Myle from 2018-03-21T23:18:09
NAACL 2018 paper, by Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc'Aurelio Ranzato, from Facebook AI Research
In this episode we continue our theme from last episode on s...
52 - Sequence-to-Sequence Learning as Beam-Search Optimization, with Sam Wiseman from 2018-03-15T21:56:25
EMNLP 2016 paper by Sam Wiseman and Sasha Rush.
In this episode we talk with Sam about a paper from a couple of years ago on bringing back some ideas from structured prediction into neural...
51 - A Regularized Framework for Sparse and Structured Neural Attention, with Vlad Niculae from 2018-03-12T21:29:04
NIPS 2017 paper by Vlad Niculae and Mathieu Blondel.
Vlad comes on to tell us about his paper. Attentions are often computed in neural networks using a softmax operator, which maps scalar...
50 - Cardinal Virtues: Extracting Relation Cardinalities from Text, with Paramita Mirza from 2018-02-14T19:18:48
ACL 2017 paper, by Paramita Mirza, Simon Razniewski, Fariz Darari, and Gerhard Weikum.
There's not a whole lot of work on numbers in NLP, and getting good information out of numbers expres...
49 - A Joint Sequential and Relational Model for Frame-Semantic Parsing, with Bishan Yang from 2018-02-05T19:01:53
EMNLP 2017 paper by Bishan Yang and Tom Mitchell.
Bishan tells us about her experiments on frame-semantic parsing / semantic role labeling, which is trying to recover the predicate-argumen...
48 - Incidental Supervision: Moving Beyond Supervised Learning, with Dan Roth from 2018-01-29T19:42:39
AAAI 2017 paper, by Dan Roth.
In this episode we have a conversation with Dan about what he means by "incidental supervision", and how it's related to ideas in reinforcement learning and r...
47 - Dynamic integration of background knowledge in neural NLU systems, with Dirk Weißenborn from 2018-01-24T16:54:49
How should you incorporate background knowledge into a neural net? A lot of people have been thinking about this problem, and Dirk Weissenborn comes on to tell us about his work in this area. Pape...
Listen
46 - Parsing with Traces, with Jonathan Kummerfeld from 2018-01-08T21:30:10
TACL 2017 paper by Jonathan K. Kummerfeld and Dan Klein.
Jonathan tells us about his work on parsing algorithms that capture traces and null elements in sentence structure. We spend the f...
45 - Build It, Break It workshop, with Allyson Ettinger and Sudha Rao from 2018-01-02T19:52:20
How robust is your NLP system? High numbers on common datasets can be misleading, as most systems are easily fooled by small modifications that would not be hard for humans to understand.
Ally...
44 - Truly Low Resource NLP, with Anders Søgaard from 2017-12-07T22:50:55
Anders talks with us about his line of work on doing NLP in languages where you have no linguistic resources other than a Bible translation or other religious works. He and his students have devel...
Listen
43 - Reinforced Video Captioning with Entailment Rewards, with Ramakanth and Mohit from 2017-12-04T20:19:55
EMNLP 2017 paper by Ramakanth Pasunuru and Mohit Bansal
Ram and Mohit join us to talk about their work, which uses reinforcement learning to improve performance on a video captioning task....
42 - Generating Sentences by Editing Prototypes, with Kelvin Guu from 2017-11-30T20:02:53
Paper is by Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, and Percy Liang
In this episode, Kelvin tells us how to build a language model that starts from a prototype sentence instead o...
41 - Cross-Sentence N-ary Relation Extraction with Graph LSTMs, with Nanyun (Violet) Peng from 2017-11-10T00:28:03
TACL 2017 paper, by Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih.
Most relation extraction work focuses on binary relations, like (Seattle, located in, Washi...
40 - On the State of the Art of Evaluation in Neural Language Models, with Gábor Melis from 2017-11-07T19:58:16
Recent arxiv paper by Gábor Melis, Chris Dyer, and Phil Blunsom.
Gábor comes on the podcast to tell us about his work. He performs a thorough comparison between vanilla LSTMs and recurren...
39 - Organizing the SemEval task on scientific information extraction, with Isabelle Augenstein from 2017-11-01T18:53:43
Isabelle Augenstein was the lead organizer of SemEval 2017 task 10, on extracting keyphrases and relations from scientific publications. In this episode we talk about her experience organizing the...
Listen
38 - A Corpus of Natural Language for Visual Reasoning, with Alane Suhr from 2017-10-30T18:07:36
ACL 2017 best resource paper, by Alane Suhr, Mike Lewis, James Yeh, and Yoav Artzi
Alane joins us on the podcast to tell us about the dataset, which contains images paired with natural lan...
37 - On Statistical Significance, Training Variance, and Why Reporting Score Distributions Matters from 2017-10-24T15:58:34
In this episode we talk about a couple of recent papers that get at the issue of training variance, and why we should not just take the max from a training distribution when reporting results. Sad...
Listen
36 - Attention Is All You Need, with Ashish Vaswani and Jakob Uszkoreit from 2017-10-23T19:36:52
NIPS 2017 paper.
We dig into the details of the Transformer, from the "attention is all you need" paper. Ashish and Jakob give us some motivation for replacing RNNs and CNNs with a more p...
35 - Replicability Analysis for Natural Language Processing, with Roi Reichart from 2017-10-19T17:28:26
TACL 2017 paper by Rotem Dror, Gili Baumer, Marina Bogomolov, and Roi Reichart.
Roi comes on to talk to us about how to make better statistical comparisons between two methods when there a...
34 - Translating Neuralese, with Jacob Andreas from 2017-10-17T18:23:59
ACL 2017 paper by Jacob Andreas, Anca D. Dragan, and Dan Klein.
Jacob comes on to tell us about the paper. The paper focuses on multi-agent dialogue tasks, where two learning systems need...
33 - Entity Linking via Joint Encoding of Types, Descriptions, and Context, with Nitish Gupta from 2017-10-16T21:04:36
EMNLP 2017 paper by Nitish Gupta, Sameer Singh, and Dan Roth.
Nitish comes on to talk to us about his paper, which presents a new entity linking model that both unifies prior sources of in...
32 - The Effect of Different Writing Tasks on Linguistic Style, with Roy Schwartz from 2017-10-10T16:55:30
CoNLL 2017 paper, by Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A. Smith.
Roy comes on to talk to us about the paper. They analyzed the ROCStories corp...
31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling from 2017-10-06T15:53:28
ICLR 2017 paper by Hakan Inan, Khashayar Khosravi, Richard Socher, presented by Waleed.
The paper presents some tricks for training better language models.
It introduces a modified l...
30 - Probabilistic Typology: Deep Generative Models of Vowel Inventories from 2017-10-05T21:51:23
Paper by Ryan Cotterell and Jason Eisner, presented by Matt.
This paper won the best paper award at ACL 2017. It's also quite outside the typical focus areas that you see at NLP conferenc...
29 - Neural machine translation via binary code prediction, with Graham Neubig from 2017-07-14T14:43:34
ACL 2017 paper, by Yusuke Oda and others (including Graham Neubig) at Nara Institute of Science and Technology (Graham is now at Carnegie Mellon University).
Graham comes on to talk to us ...
28 - Data Programming: Creating Large Training Sets, Quickly from 2017-07-11T22:46:50
NIPS 2016 paper by Alexander Ratner and coauthors in Chris Ré's group at Stanford, presented by Waleed.
The paper presents a method for generating labels for an unlabeled dataset by combin...
27 - What do Neural Machine Translation Models Learn about Morphology?, with Yonatan Belinkov from 2017-07-05T17:08:19
ACL 2017 paper by Yonatan Belinkov and others at MIT and QCRI.
Yonatan comes on to tell us about their work. They trained a neural MT system, then learned models on top of the NMT represe...
26 - Structured Attention Networks, with Yoon Kim from 2017-06-30T14:26:06
ICLR 2017 paper, by Yoon Kim, Carl Denton, Luong Hoang, and Sasha Rush.
Yoon comes on to talk with us about his paper. The paper shows how standard attentions can be seen as an expected f...
25 - Neural Semantic Parsing over Multiple Knowledge-bases from 2017-06-28T16:13:53
ACL 2017 short paper, by Jonathan Herzig and Jonathan Berant.
This is a nice, obvious-in-hindsight paper that applies a frustratingly-easy-domain-adaptation-like approach to semantic parsi...
24 - Improving Hypernymy Detection with an Integrated Path-based and Distributional Method from 2017-06-27T16:10:04
ACL 2016 outstanding paper, by Vered Shwartz, Yoav Goldberg and Ido Dagan.
Waleed presents this paper, discussing hypernymy detection and the methods used in the paper. It's pretty simila...
23 - Get To The Point: Summarization with Pointer-Generator Networks from 2017-06-26T15:30:48
ACL 2017 paper by Abigail See, Peter Liu, and Chris Manning.
Matt presents the paper, describing the task (summarization on CNN/Daily Mail), the model (the standard copy + generate model t...
22 - Deep Multitask Learning for Semantic Dependency Parsing, with Noah Smith from 2017-06-16T14:12:49
An interview with Noah Smith.
Noah tells us about his work with his students Hao Peng and Sam Thomson. We talk about what semantic dependency parsing is, the model that they used to appro...
21 - Contextual Explanation Networks, with Maruan Al-Shedivat from 2017-06-15T14:46:47
https://arxiv.org/abs/1705.10301
Maruan, Avinava Dubey and Eric Xing essentially put the post-hoc decision boundary explanations from the "Why Should I Trust You?" paper* as a core compone...
20 - A simple neural network module for relational reasoning from 2017-06-14T15:55:45
The recently-hyped paper that got "superhuman" performance on FAIR's CLEVR dataset.
https://arxiv.org/abs/1706.01427
19 - End-to-end Differentiable Proving, with Tim Rocktäschel from 2017-06-12T16:15:03
An interview with Tim Rocktäschel.
https://arxiv.org/abs/1705.11040
18 - Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema from 2017-06-09T16:23:03
https://www.semanticscholar.org/paper/Generalizing-to-Unseen-Entities-and-Entity-Pairs-w-Verga-Neelakantan/7dd8b958632b07e41979337c71d847a3f39df456
Listen
17 - pix2code: Generating Code from a Graphical User Interface Screenshot from 2017-06-08T14:56:33
https://arxiv.org/abs/1705.07962
Listen
16 - Arc-swift: A Novel Transition System for Dependency Parsing from 2017-06-07T16:07:45
https://www.semanticscholar.org/paper/Arc-swift-A-Novel-Transition-System-for-Dependency-Qi-Manning/56fc1372a41a46f777ac77859219bb4b76bfd098
Listen
15 - Attention and Augmented Recurrent Neural Networks from 2017-06-06T15:46:40
http://distill.pub/2016/augmented-rnns/
Listen
14 - Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning from 2017-06-05T16:14:15
https://arxiv.org/abs/1705.00557
Listen
13 - Question Answering from Unstructured Text by Retrieval and Comprehension from 2017-06-02T15:56:44
https://www.semanticscholar.org/paper/Question-Answering-from-Unstructured-Text-by-Retri-Watanabe-Dhingra/89d06c2996f379c602e64b4243f026cd164400d7
Listen
12 - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data from 2017-06-01T20:38:53
https://arxiv.org/abs/1705.02364
Listen
11 - Relation Extraction with Matrix Factorization and Universal Schemas from 2017-05-29T15:03:10
https://www.semanticscholar.org/paper/Relation-Extraction-with-Matrix-Factorization-and-Riedel-Yao/52b5eab895a2d9ae24ea72ca72782974a52f90c4
Listen
10 - A Syntactic Neural Model for General-Purpose Code Generation from 2017-05-26T14:55:49
https://www.semanticscholar.org/paper/A-Syntactic-Neural-Model-for-General-Purpose-Code-Yin-Neubig/c8d0e13de2eaa09a928eff36b99d63f494c2f5ec
Listen
09 - Learning to Generate Reviews and Discovering Sentiment from 2017-05-25T15:36:50
https://www.semanticscholar.org/paper/Learning-to-Generate-Reviews-and-Discovering-Senti-Radford-Jozefowicz/664ec878de4b7170712baae4a7821fc2602bba25
https://blog.openai.com/unsupervised-se...
08 - Finding News Citations for Wikipedia from 2017-05-24T14:18:25
https://www.semanticscholar.org/paper/Finding-News-Citations-for-Wikipedia-Fetahu-Markert/526acf565190d843758b89d37acf281639cb90e2
Listen
07 - Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks from 2017-05-23T13:32:27
https://www.semanticscholar.org/paper/Capturing-Semantic-Similarity-for-Entity-Linking-w-Francis-Landau-Durrett/1c9aca60f7ac5edcceb73d612806704a7d662643
Listen
06 - Design Challenges for Entity Linking from 2017-05-22T14:17:49
https://www.semanticscholar.org/paper/Design-Challenges-for-Entity-Linking-Ling-Singh/aa2a7ac7bfa9a0201d4faddd4e7bb26299a5e0be
Listen
05 - Transition-Based Dependency Parsing with Stack Long Short-Term Memory from 2017-05-19T15:00:11
https://www.semanticscholar.org/paper/Transition-Based-Dependency-Parsing-with-Stack-Lon-Dyer-Ballesteros/396b7932beac62a72288eaea047981cc9a21379a
Listen
04 - Recurrent Neural Network Grammars, with Chris Dyer from 2017-05-12T00:40:03
An interview with Chris Dyer.
https://www.semanticscholar.org/paper/Recurrent-Neural-Network-Grammars-Dyer-Kuncoro/1594d954abc650bce2db445c52a76e49655efb0c
03 - FastQA: A Simple and Efficient Neural Architecture for Question Answering from 2017-05-12T00:38:45
https://www.semanticscholar.org/paper/FastQA-A-Simple-and-Efficient-Neural-Architecture-Weissenborn-Wiese/7c1576b96a1e246d77b30f7b80cec63be96fa698
Listen
02 - Bidirectional Attention Flow for Machine Comprehension from 2017-05-12T00:37:43
https://www.semanticscholar.org/paper/Bidirectional-Attention-Flow-for-Machine-Seo-Kembhavi/007ab5528b3bd310a80d553cccad4b78dc496b02
Listen
01 - A Comparative Study of Word Embeddings for Reading Comprehension from 2017-05-12T00:31:53
https://www.semanticscholar.org/paper/A-Comparative-Study-of-Word-Embeddings-for-Reading-Dhingra-Liu/3ec37205c9201fc891ab51da200e361fdc34bfb3
Listen
00 - Intro to the podcast from 2017-05-12T00:30:22
In this episode we briefly say what we're up to with the podcast. No technical content, just a description of what each episode will look like, and why we're doing this.
Listen