Since Fall Semester of 2016, Rebecca J. Passonneau and David Reitter introduced a new series of talks to Penn State: the Joint CSE/IST Colloquium in Natural Language Processing! Starting in the Fall of 2019, this colloquium is jointly hosted by Rebecca J. Passonneau (CSE), Shomir Wilson (IST) and Kenneth Huang (IST)!
2024-2025
Date: September 6th 2024
Format: Zoom
Time: 2:00-3:00 pm
Speaker: Heng Ji
Title: Making Large Language Model’s Knowledge More Accurate, Organized, Up-to-date and Fair
Abstract: Large language models (LLMs) have demonstrated remarkable performance on knowledge reasoning tasks, owing to their implicit knowledge derived from extensive pretraining data. However, their inherent knowledge bases often suffer from disorganization and illusion, bias towards common entities, and rapid obsolescence. Consequently, LLMs frequently make up untruthful information, exhibit resistance to updating outdated knowledge, or struggle with generalizing across multiple languages. In this talk I will aim to answer the following questions: (1) Where and How is Knowledge Stored in LLM? (2) Why does LLM Lie? (3) How to Update LLM’s Dynamic Knowledge? (4) How to Reach LLM’s Knowledge Updating Ripple Effect? and (5) What can knowledge + LLM do for us? I will present a case study on “SmartBook” – situation report generation. Our investigations reveal several underlying causes. First, LLMs acquire implicit knowledge primarily through attention-weighted associations between words, rather than explicit understanding of concepts, entities, attributes, relations, events, semantic roles, and logics. We will investigate where various types of knowledge are stored inside LLMs. Second, frequent word associations overshadow uncommon ones due to training data imbalance and wide context, particularly in contexts involving dynamic events. Third, counter-intuitive updating behaviors are elucidated through a novel gradient similarity metric. Fourth, LLMs are often unaware of real-world events occurring after their pretraining phase, complicating the anchoring of related knowledge updates. While existing methods focus largely on updating entity attributes, our research underscores the necessity of updating factual knowledge—such as participants, semantic roles, time, and location—based on real-world events. We propose a novel framework for knowledge updating in LLMs that leverages event-driven signals to identify factual errors preemptively and introduce a training-free self-contrastive decoding approach to mitigate inference errors.
Bio: Heng Ji is a professor at Computer Science Department, and an affiliated faculty member at Electrical and Computer Engineering Department and Coordinated Science Laboratory of University of Illinois Urbana-Champaign. She is an Amazon Scholar. She is the Founding Director of Amazon-Illinois Center on AI for Interactive Conversational Experiences (AICE). She received her B.A. and M. A. in Computational Linguistics from Tsinghua University, and her M.S. and Ph.D. in Computer Science from New York University. Her research interests focus on Natural Language Processing, especially on Multimedia Multilingual Information Extraction, Knowledge-enhanced Large Language Models and Vision-Language Models. The awards she received include Outstanding Paper Award at ACL2024, two Outstanding Paper Awards at NAACL2024, “Young Scientist” by the World Laureates Association in 2023 and 2024, “Young Scientist” and a member of the Global Future Council on the Future of Computing by the World Economic Forum in 2016 and 2017, “Women Leaders of Conversational AI” (Class of 2023) by Project Voice, “AI’s 10 to Watch” Award by IEEE Intelligent Systems in 2013, NSF CAREER award in 2009, PACLIC2012 Best paper runner-up, “Best of ICDM2013” paper award, “Best of SDM2013” paper award, ACL2018 Best Demo paper nomination, ACL2020 Best Demo Paper Award, NAACL2021 Best Demo Paper Award, Google Research Award in 2009 and 2014, IBM Watson Faculty Award in 2012 and 2014 and Bosch Research Award in 2014-2018. She was invited to testify to the U.S. House Cybersecurity, Data Analytics, & IT Committee as an AI expert in 2023. She was selected to participate in DARPA AI Forward in 2023. She was invited by the Secretary of the U.S. Air Force and AFRL to join Air Force Data Analytics Expert Panel to inform the Air Force Strategy 2030, and invited to speak at the Federal Information Integrity R&D Interagency Working Group (IIRD IWG) briefing in 2023. She is the lead of many multi-institution projects and tasks, including the U.S. ARL projects on information fusion and knowledge networks construction, DARPA ECOLE MIRACLE team, DARPA KAIROS RESIN team and DARPA DEFT Tinker Bell team. She has coordinated the NIST TAC Knowledge Base Population task 2010-2020. She is the Chief Editor of Data Intelligence Journal, and served as the associate editor for IEEE/ACM Transaction on Audio, Speech, and Language Processing, and the Program Committee Co-Chair of many conferences including NAACL-HLT2018 and AACL-IJCNLP2022. She was elected as the North American Chapter of the Association for Computational Linguistics (NAACL) secretary 2020-2023. Her research has been widely supported by the U.S. government agencies (DARPA, NSF, DoE, ARL, IARPA, AFRL, DHS) and industry (Amazon, Google, Bosch, IBM, Disney).
Date: September 27th 2024
Format: Zoom
Time: 2:00-3:00 pm
Speaker: Lu Wang
Title: Moral Event Extraction and Temporal Graph Generation for Narrative Understanding
Abstract:
Narrative understanding is key to grasping events, character roles, and their interactions. In particular, the values embedded in these narratives can shape readers’ perceptions and influence the public’s opinions. We are interested in studying media narratives. While news media often aim to reduce the use of explicit moral language, most news articles remain dense with moral values embedded in the events being reported. However, detecting moral values reflected in the complex interactions between entities and the corresponding moral events presents a significant challenge for many NLP systems, including large language models (LLMs). To address this, we introduce a new dataset, Moral Events, containing news articles from diverse US media outlets across the political spectrum. We then propose MOKA, a moral event extraction framework that uses MOral Knowledge Augmentation. Our experiments demonstrate that MOKA surpasses competitive baselines across three tasks related to moral event understanding. We also show that mainstream media often engage in selective reporting of moral events.
In the second part of this talk, I will discuss our recent work on investigating LLMs’ capabilities of reasoning about time and temporal relations, which is critical to understanding event relations in narratives. While LLMs excel in many tasks, temporal reasoning remains a significant challenge due to its complexity. We focus on temporal graph generation to assess the global reasoning capabilities of LLMs and show that even powerful models like GPT-3.5/4 struggle with this task. To address this without the need for expensive model training or fine-tuning, we introduce a prompting technique that converts events into a Python class and guides LMs to generate temporally grounded narratives, which are used to support temporal graph generation. Experiments show significant improvements on various types of events.
Bio:
Lu Wang is an Associate Professor of Computer Science and Engineering at University of Michigan. Previously until 2020, she was at Khoury College of Computer Sciences, Northeastern University. She completed her Ph.D. in the Department of Computer Science at Cornell University, under supervision of Professor Claire Cardie. Lu’s research is focused on natural language processing, computational social science, and machine learning. More specifically, Lu aims to build trustworthy models that produce factual content and are calibrated according to their knowledge. She works on problems of summarization, generation, reasoning, argument mining, as well as create novel applications to understand narratives and media bias, and to support education. Her work won area chair award at ACL 2024, best paper honorable mention award at CHI 2023, outstanding paper award at ACL 2017, and best paper nomination award at SIGDIAL 2012.
Date: November 1st 2024
Format: Zoom
Time: 12:00-1:00 pm
Speaker: Chris Callison-Burch
Title: Using Large Language Models to Build Explainable Classifiers
Abstract: This presentation discusses research on using large language models (LLMs) to build explainable classifiers. It will show off work from my PhD students and collaborators on several recent research directions:
● Image classification with explainable features (arxiv.org/abs/2211.11158)
● Text classification with explainable features (work in progress)
● The importance of faithfulness in explanations (arxiv.org/abs/2209.11326)
● A faithful “chain of thought” LLM reasoner that produces code in its explanations (arxiv.org/abs/2301.13379)
The talk will cover joint work with: Adam Stein, Ajay Patel, Ansh Kothary, Artemis Panagopoulou, Daniel Jin, Delip Rao, Eric Wong, Harry Li Zhang, Kathleen McKeown, Marianna Apidianaki, Mark Yatskar, Shenghao Zhou, Shreya Havaldar, Veronica Qing Lyu, Yue Yang, and others.
Bio: Chris Callison-Burch is a Professor of Computer and Information Science at the University of Pennsylvania. His course on Artificial Intelligence has one of the highest enrollments at the university with over 500 students taking the class each Fall. He is best known for his research into natural language processing. His current research is focused on applications of large language models to long-standing challenges in artificial intelligence. His PhD students joke that now whenever they ask him anything his first response is “Have you tried GPT for that?”. Prof Callison-Burch has more than 150 publications, which have been cited over 25,000 times. He is a Sloan Research Fellow, and he has received faculty research awards from Google, Microsoft, Amazon, Facebook, and Roblox, in addition to funding from DARPA, IARPA, and the NSF.
Date: November 15th 2024
Format: In-person and Zoom
Time: 2:00-3:00 pm
Speaker: Sasha Rush
Title: SSMs Change The Foundation Model Design Space
Abstract: Recent work has shown that linear RNN models such as SSMs show competitive scaling properties with Transformers in several modalities. Much of the conversation has focused on whether these models can “beat” Transformers in a variety of different tasks. Instead, in this talk we consider two applications where SSM architectures open up new model design possibilities. We first give a quick tutorial of the architecture, focusing on its scaling. Next we discuss the implications of these results, focusing on building efficient byte-level language models. Finally we discuss SSMs for non-language foundation models using long-range linearization in image generation.
Bio: Alexander “Sasha” Rush is an Associate Professor at Cornell Tech and a researcher at Hugging Face. His research interest is in the study of language models with applications in controllable text generation, efficient inference, and applications in summarization and information extraction. In addition to research, he has written several popular open-source software projects supporting NLP research, programming for deep learning, and virtual academic conferences.
2023-2024
Date: September 8th 2023
Format: Zoom
Time: 2:00-3:00
Speaker: Joyce Y. Chai
Title: Communicating with Embodied Agents
Abstract: Despite recent advances, language communication in embodied AI still faces many challenges. Human language not only needs to ground to agents’ perception and action but also needs to facilitate collaboration between humans and agents. To address these challenges, I will introduce several efforts in my lab that study pragmatic communication with embodied agents. I will talk about cognitively motivated grounded language learning that facilitates fast mapping. I will discuss task learning by following language instructions and highlight the importance of physical commonsense reasoning. I will further present explicit modeling of partners’ goals, beliefs, and abilities (i.e., theory of mind) and discuss its role in language communication for situated collaborative tasks.
Bio: Joyce Chai is a Professor in the Department of Electrical Engineering and Computer Science at the University of Michigan. She holds a Ph.D. in Computer Science from Duke University. Her research interests span from natural language processing and embodied AI to human-AI collaboration. She is fascinated by how experience with the world and how social pragmatics shape language learning and language use; and is excited about developing language technology that is sensorimotor grounded, pragmatically rich, and cognitively motivated. Her current work explores the intersection between language, perception, and action to enable situated communication with embodied agents. She served on the executive board of NAACL and as Program Co-Chair for multiple conferences – most recently ACL 2020. She is a recipient of the NSF Career Award and several paper awards with her students (e.g., Best Long Paper Award at ACL 2010, Outstanding Paper Awards at EMNLP 2021 and ACL 2023). She is a Fellow of ACL.
Date: October 6th 2023
Format: Zoom
Time: 2:00-3:00
Speaker: Katharina Kann
Title: Towards Universal Natural Language Processing
Abstract: Natural language processing (NLP) plays an increasingly important role in everyday life, and many people are familiar with products such as Google Translate, Alexa, Siri, or ChatGPT. However, NLP systems currently only exist for a small fraction of the world’s approximately 7000 languages. This is undesirable for many reasons. For instance, only speakers of high-resource languages are able to benefit from the abundance of information available on the internet, which reinforces already existing inequalities. It also limits the ability of NLP to support language documentation and revitalization efforts. NLP systems further perform poorly for many domains, which limits their applicability, e.g., in healthcare or for educational purposes. In this talk, I will present a couple of my recent lines of work: I will first discuss how we can leverage NLP systems to speed up language documentation efforts. I will then talk about how models can be trained for or adapted to various high-level NLP tasks. I will end with an outlook on remaining challenges and questions for both low-resource languages and low-resource domains.
Bio: Katharina von der Wense is an Assistant Professor of Computer Science at University of Colorado Boulder, USA, and a Juniorprofessor at the Johannes Gutenberg University Mainz, Germany. She leads the VDW Natural Language Processing Group (NALA). She received her PhD from LMU Munich in 2019 and was a postdoc at New York University until she moved to Boulder in 2020. Her work is centered around deep learning for NLP, with a special focus on multilingual NLP and transfer learning, computational morphology, language grounding, and NLP for medical and educational applications.
Date: October 20th 2023
Format: Zoom
Time: 2:00-3:00
Speaker: Mohit Bansal
Title: Multimodal (Generative) LLMs: Unification, Efficiency, Interpretability
Abstract: In this talk, I will present our journey of large-scale multimodal pretrained (generative) models across various modalities (text, images, videos, audio, layouts, etc.) and enhancing important aspects such as unification, efficiency, and interpretability. We will start by discussing early cross-modal vision-and-language pretraining models (LXMERT) and visually-grounded text models with image/video knowledge distillation (Vokenization, VidLanKD). We will then present early unified models (VL-T5) to combine several multimodal tasks (such as visual QA, referring expression comprehension, visual entailment, visual commonsense reasoning, captioning, and multimodal translation) by treating all these tasks as text generation. We will also look at recent unified models (with joint objectives and architecture) such as textless video-audio transformers (TVLT), vision-text-layout transformers for universal document processing (UDOP), composable any-to-any multimodal generation (CoDi), as well as consistent multi-scene video generation (VideoDirectorGPT). Second, we will look at further parameter/memory efficiency via adapter (VL-Adapter), ladder-sidetuning (LST), sparse sampling (ClipBERT), and audio replacement (ECLIPSE) methods. I will conclude with interpretability and evaluation aspects of image generation models, based on fine-grained skill and bias evaluation (DALL-Eval) and based on interpretable and controllable visual programs (VPGen+VPEval).
Bio: Dr. Mohit Bansal is the John R. & Louise S. Parker Professor and the Director of the MURGe-Lab (UNC-NLP Group) in the Computer Science department at the University of North Carolina (UNC) Chapel Hill. Prior to this, he was a research assistant professor (3-year endowed position) at TTI-Chicago. He received his Ph.D. in 2013 from the University of California at Berkeley (where he was advised by Dan Klein) and his B.Tech. from the Indian Institute of Technology at Kanpur in 2008. His research expertise is in natural language processing and multimodal machine learning, with a particular focus on grounded and embodied semantics, language generation and Q&A/dialogue, and interpretable and generalizable deep learning. He is a recipient of DARPA Director’s Fellowship, NSF CAREER Award, Google Focused Research Award, Microsoft Investigator Fellowship, Army Young Investigator Award (YIP), DARPA Young Faculty Award (YFA), and outstanding paper awards at ACL, CVPR, EACL, COLING, and CoNLL. His service includes ACL Executive Committee, ACM Doctoral Dissertation Award Committee, CoNLL Program Co-Chair, ACL Americas Sponsorship Co-Chair, and Associate/Action Editor for TACL, CL, IEEE/ACM TASLP, and CSL journals.
Date: December 1st 2023
Format: Zoom
Time: 2:00-3:00
Speaker: Sameer Singh
Title: Lipstick on a Pig: Using Language Models as Few-Shot Learners
Abstract: Language models provide representations that can be adapted to many NLP tasks with minimal effort. In particular, language modeling has provided exceptional few-shot natural language understanding and reasoning performance by framing these tasks as prompts. These capabilities improve significantly with larger models and datasets, making the direct use of language models the dominant approach for NLP applications. However, the goals of language modeling are not precisely the same as what we need from few-shot learners, and it is vital to understand this gap.
In this talk, we will discuss the basics of neural networks, the text corpus, and the training pipeline that enables language models to behave as these general-purpose AI agents. However, we will show how this very paradigm of language modeling also introduces fundamental limitations in this technology. We will characterize these vulnerabilities in language models and discuss how they affect end-use applications. The results presented suggest language modeling may not be sufficient to learn robust reasoners and that we need to take the pretraining data into account when interpreting few-shot evaluation results.
2022-2023
Date: September 2nd, 2022
Format: Zoom
Time: 2:00-3:00
Speaker: Yejin Choi
Title: David V.S. Goliath: the Art of Leaderboarding in the Era of Extreme-Scale Neural Models
Abstract: Scale appears to be the winning recipe in today’s leaderboards. And yet, extreme-scale neural models are still brittle to make errors that are often nonsensical and even counterintuitive. In this talk, I will argue for the importance of knowledge, especially common sense knowledge, as well as inference-time reasoning algorithms, and demonstrate how smaller models developed in academia can still have an edge over larger industry-scale models, if powered with knowledge and/or reasoning algorithms. First, I will introduce “symbolic knowledge distillation”, a new framework to distill larger neural language models into smaller common sense models, which leads to a machine-authored KB that wins, for the first time, over a human-authored KB in all criteria: scale, accuracy, and diversity. Next, I will highlight how we can make better lemonade out of neural language models by shifting our focus to unsupervised, inference-time reasoning algorithms. I will demonstrate how unsupervised models powered with algorithms can match or even outperform supervised approaches on hard reasoning tasks such as nonmonotonic reasoning (such as counterfactual and abductive reasoning), or complex language generation tasks that require logical constraints. Finally, I will introduce a new (and experimental) conceptual framework, Delphi, toward machine norms and morality, so that the machine can learn to reason that “helping a friend” is generally a good thing to do, but “helping a friend spread fake news” is not.
Bio: Yejin Choi is a Professor at Paul G. Allen School of Computer Science & Engineering at University of Washington and Allen Institute for Artificial Intelligence (AI2). Her primary research interests are in the fields of Natural Language Processing, Machine Learning and Artificial Intelligence, with broader interests in Computer Vision and Digital Humanities.
Date: September 16th, 2022
Format: Zoom
Time: 2:00-3:00
Speaker: Luke Zettlemoyer
Title: Large Language Models: Will they keep getting bigger? And, how will we use them if they do?
Abstract: The trend of building ever larger language models has dominated much research in NLP over the last few years. In this talk, I will discuss our recent efforts to (at least partially) answer two key questions in this area: Will we be able to keep scaling? And, how will we actually use the models, if we do? I will cover our recent efforts on learning new types of sparse mixtures of experts (MoEs) models. Unlike model-parallel algorithms for learning dense models, which are very difficult to further scale with existing hardware, our sparse approaches have significantly reduced cross-node communication costs and could possibly provide the next big leap in performance, although finding a version that scales well in practice remains an open challenge. I will also present our recent work on prompting language models that better controls for surface form variation, to improve performance of models that are so big we can only afford to do inference, with little to no task-specific fine tuning. Finally, time permitting, I will discuss work on new forms of supervision for language model training, including learning from the hypertext and multi-modal structure of web pages to provide new signals for both learning and prompting the model. Together, these methods present our best guesses for how to keep the scaling trend alive as we move forward to the next generation of NLP models. This talk describes work done at the University of Washington and Meta, primarily led by Armen Aghajanyan, Ari Holtzmann, Mike Lewis, Sewon Min, and Peter West.
Bio: Luke Zettlemoyer is a Professor in the Allen School of Computer Science & Engineering at the University of Washington, and also a Research Scientist at Meta. His honors include multiple paper awards, and being named an PECASE Awardee and an Allen Distinguished Investigator. Previously, he did postdoctoral research at the University of Edinburgh and earned a Ph.D. at MIT. His current research is in the intersections of natural language processing, machine learning, and decision making under uncertainty. He is particularly interested in designing learning algorithms for recovering representations of the meaning of natural language text.
Date: October 7th, 2021
Format: In-person, W201 Cybertorium
Time: 2:00-3:00
Speaker: Yonatan Bisk
Title: Following Instructions and Asking Questions
Abstract: As we move towards the creation of embodied agents that understand natural language, several new challenges and complexities arise for grounding (e.g. complex state spaces), planning (e.g. long horizons), and social interaction (e.g. asking for help or clarifications). In this talk, I’ll discuss several recent results both on improvements to embodied instruction following within ALFRED and initial steps towards building agents that ask questions or model theory-of-mind.
Bio: Yonatan Bisk is an Assistant Professor at Carnegie Mellon University. His research area is Natural Language Processing (NLP) with a focus on grounding and embodiment. In particular, his interests are broadly: Modeling the semantics of the physical world, and Connecting language to perception and control.
2021-2022
Date: September 3rd, 2021
Format: In Person
Time: 2:00-3:00
Location: Cybertorium, Westgate Bldg
Recorded Talk and Presentation slides
Speaker: Nathan Schneider
Title: Squirrel or Skunk? NLP Models Face the Long Tail of Language
Abstract: Natural language is chock-full of rare events, which can challenge NLP models based on supervised learning. Dr. Schneider will present advances in modeling and evaluation of “long tail” phenomena in grammar and meaning: retrieving rare senses of words; tagging words with complex syntactic categories (TACL 2021); and calibrating model confidence scores for sparse tagsets.
Bio: Dr. Nathan Schneider is an annotation schemer and computational modeler for natural language. As Assistant Professor of Linguistics and Computer Science at Georgetown University, he looks for synergies between practical language technologies and the scientific study of language. He specializes in broad-coverage semantic analysis: designing linguistic meaning representations, annotating them in corpora, and automating them with statistical natural language processing techniques. A central focus in this research is the nexus between grammar and lexicon as manifested in multiword expressions and adpositions/case markers. He has inhabited UC Berkeley (BA in Computer Science and Linguistics), Carnegie Mellon University (Ph.D. in Language Technologies), and the University of Edinburgh (postdoc). Now a Hoya and leader of NERT, he continues to play with data and algorithms for linguistic meaning.
Date: October 15th, 2021
Format: Zoom
Time: 2:00-3:00
Speaker: Rada Mihalcea
Title: The Ups and Downs of Word Embeddings
Abstract: Word embeddings have largely been a “success story” in our field. They have enabled progress in numerous language processing applications, and have facilitated the application of large-scale language analyses in other domains, such as social sciences and humanities. While less talked about, word embeddings also have many shortcomings — instability, lack of transparency, biases, and more. In this talk, I will review the “ups” and “downs” of word embeddings, discuss tradeoffs, and chart potential future research directions to address some of the downsides of these word representations.
Bio: Rada Mihalcea is the Janice M. Jenkins Collegiate Professor of Computer Science and Engineering at the University of Michigan and the Director of the Michigan Artificial Intelligence Lab. Her research interests are in computational linguistics, with a focus on lexical semantics, computational social sciences, and multimodal language processing. She serves or has served on the editorial boards of the Journals of Computational Linguistics, Language Resources and Evaluations, Natural Language Engineering, Journal of Artificial Intelligence Research, IEEE Transactions on Affective Computing, and Transactions of the Association for Computational Linguistics. She was a program co-chair for EMNLP 2009 and ACL 2011, and a general chair for North American ACL 2015 and *SEM 2019. She directs multiple diversity and mentorship initiatives, including Girls Encoded and the ACL Year-Round Mentorship program. She currently serves as ACL President. She is the recipient of a Presidential Early Career Award for Scientists and Engineers awarded by President Obama (2009), and was named an ACM Fellow (2019) and an AAAI Fellow (2021). In 2013, she was made an honorary citizen of her hometown of Cluj-Napoca, Romania.
Date: November 5th, 2021
Format: Zoom
Time: 2:00-3:00
Speaker: Jacob Eisenstein
Title: Trustworthy NLP: A Causal Perspective
Abstract: Natural language processing systems have achieved high levels of accuracy on many benchmark datasets, yet it is unclear whether these same systems can be trusted in high-stakes settings. This talk identifies some key desiderata for more trustworthy natural language processing, and argues that progress can be made by treating learning and inference as causal systems. In particular, I will focus on counterfactual invariance – the notion that predictions should be invariant to interventions on causally-irrelevant variables, which can be formalized in checklist-style evaluations. I will present a method for achieving counterfactual invariance by designing regularizers around the causal structure of the data-generating process. For some causal structures, the resulting counterfactually-invariant predictors are optimal with respect to a class of “causally-compatible” perturbations of the data generating process. Empirically, these predictors avoid spurious correlations and transfer well to new data distributions. I will also touch on other connections between causality and natural language processing highlighted in the recent survey by Feder et al (2021).
Bio: Jacob Eisenstein is a research scientist at Google, where he is focused on making language technology more robust and trustworthy. He was previously on the faculty of the Georgia Institute of Technology, where he supervised six successful doctoral dissertations, received the NSF CAREER Award for research on computational sociolinguistics, and wrote a textbook on natural language processing. He completed his Ph.D. at MIT with a dissertation on computational models of speech and gesture.
2020-2021
September 25th, 2020
Speaker: Lyle Ungar
Recorded Zoom: Link
Bio: Dr. Lyle Ungar is a Professor of Computer and Information Science at the University of Pennsylvania. He received a B.S. from Stanford University and a Ph.D. from MIT. Dr. Ungar directed Penn’s Executive Masters of Technology Management (EMTM) Program for a decade, and served as Associate Director of the Penn Center for BioInformatics (PCBI). He has published over 250 articles and holds ten patents. His current research focuses on statistical natural language processing, deep learning, and the use of social media to understand the psychology of individuals and communities. Lyle has consulted for companies ranging from start-ups to Fortune 500 companies on strategic use of information technology in areas including data mining, business process automation, online auction design, and chatbots.
Title: Measuring Well-Being Using Social Media Language
Abstract: Social media such as Twitter and Facebook provide a rich, if imperfect, portal into people’s lives. We analyze tens of millions of Facebook posts and billions of tweets to study variation in language use with age, gender, personality, and mental and physical well-being. Word clouds provide insights into depression, empathy and stress, correlations between language use and county-level health data suggest connections between health and happiness, including potential psychological causes of heart disease. Similar analyses are useful in many fields.
October 23rd, 2020
Speaker: Natalie Parde
Recorded Zoom: Link
Bio: Natalie Parde is an Assistant Professor in the Department of Computer Science at the University of Illinois at Chicago, where she also co-directs UIC’s Natural Language Processing Laboratory. Her research interests are in natural language processing, with emphases in healthcare applications, interactive systems, multimodality, and creative language. She serves on the program committees for EMNLP, ACL, and NAACL, and the Senior Program Committee for AAAI, among other conferences and workshops. In her spare time, Dr. Parde enjoys engaging in mentorship and outreach for underrepresented CS students.
Title: Supporting Cognitive Wellness through Spoken Language Analysis
Abstract: Natural language processing can be used to facilitate cognitive wellness in many ways, including through automated cognitive health assessment. In this talk I discuss two novel strategies for assessing cognitive health directly from individuals’ free speech, focusing on the problem at different levels of granularity. First, we design a neural model that leverages semantic and psycholinguistic features to classify patients as belonging to coarse-grained dementia or control groups. Then, we examine the utility of a variety of lexicosyntactic, psycholinguistic, discourse-based, and acoustic features to predict fine-grained, continuous scores indicative of cognitive health status. We achieve high performance surpassing existing benchmarks for both tasks. I conclude by introducing some intriguing directions for future work.
November 20th, 2020
Speaker: Katrin Erk
Recorded Zoom: Link
Bio: Katrin Erk is a professor in the Department of Linguistics at the University of Texas at Austin. Her research expertise is in the area of computational linguistics, especially semantics. Her work is on distributed, flexible approaches to describing word meaning, and on combining them with logic-based representations of sentences and other larger structures. At the word level, she is studying flexible representations of meaning in context, independent of word sense lists. At the sentence level, she is looking into probabilistic frameworks that can draw weighted inferences from combined logical and distributed representations. Katrin Erk completed her dissertation on tree description languages and ellipsis at Saarland University in 2002.
Title: How to Marry A Star: Probabilistic Constraints for Meaning in Context
Abstract: Context has a large influence on word meaning. When we look closely, there is not simply a single contextual constraint, but multiple interacting and sometimes competing constraints, including selectional constraints and discourse topic. We develop a probabilistic generative model of utterance understanding that takes into account different interacting contextual influences on word meaning, and combines word meaning with a formal logic account of sentence meaning in a single model. The model characterizes utterance understanding as probabilistically describing the situation underlying the utterance. We present small-scale experiments with the model, and discuss directions for extending its scale.
2019-2020
September 27, 2019
Speaker: Elijah Mayfield
Bio: Elijah Mayfield is an Entrepreneur-in-Residence at Carnegie Mellon University and Lecturer at the University of Pennsylvania. Previously, he was Vice President of New Technologies at Turnitin, managing machine learning and NLP research for educational products used by over 30 million students globally. He joined Turnitin when they acquired LightSide Labs, which he founded as CEO with support from the Gates Foundation, the US Department of Education, and others. Mayfield has coauthored more than 40 peer-reviewed publications on language technologies and human-computer interaction, receiving awards including a Siebel Scholarship, an IBM Ph.D. Fellowship, and Forbes 30 under 30 in Education.
Title: Explainable Humans: Algorithmic Decision-Making and Education Policy
The high stakes of automated decision-making has produced a need for explainable, equitable machine learning. This talk will explore what it means to give satisfactory explanations for algorithmic judgment calls, and where those explanations come from, with a special focus on natural language processing problems in topics relevant to educators and schools. We focus on two domains, investigating how group debates shape the content deemed notable for inclusion in Wikipedia, and how automated essay scoring learns standards of quality writing from human assessment and rubrics, and how machine learning algorithms encode existing biases in both domains into predictive models.
October 11th, 2019
Speaker: Prof. Karthik Narasimhan
Bio:Karthik Narasimhan is an assistant professor in the Computer Science department at Princeton University. His research spans the areas of natural language processing and reinforcement learning, with a focus on building intelligent agents that learn to operate in the world through both experience and existing human knowledge (e.g. text). Karthik received his PhD from MIT in 2017, and spent a year as a visiting research scientist at OpenAI prior to joining Princeton. His work has received a best paper award at EMNLP 2016 and an honorable mention for best paper at EMNLP 2015.
Title: Addressing long-term dependencies for language generation
Building accurate language models that capture meaningful long-term dependencies is a core challenge in natural language processing. Towards this end, I will first present our recent work on a calibration-based approach to measure long-term discrepancies between a generative sequence model and the true distribution. Empirically, we show that state-of-the-art language models, including LSTMs and Transformers, are miscalibrated: the entropy rates of their generations drift dramatically upward over time. We provide provable methods to mitigate this phenomenon. We also show how this calibration-based approach can be used to measure the amount of memory used by language models. Second, I will describe a new algorithm for multi-objective reinforcement learning that allows for effective optimization over multiple (potentially conflicting) long-term goals. Our method demonstrates faster learning as well as greater adaptability on several sequential prediction problems, including task-oriented dialog.
October 25th, 2019
Speaker: Prof. Kathleen McKeown
Bio:Kathleen R. McKeown is the Henry and Gertrude Rothschild Professor of Computer Science at Columbia University and the Founding Director of the Data Science Institute. She served as Director from 2012 to 2017. In earlier years, she served as Department Chair 1(998-2003) and as Vice Dean for Research for the School of Engineering and Applied Science (2010-2012). A leading scholar and researcher in the field of natural language processing, McKeown focuses her research on big data; her interests include text summarization, question answering, natural language generation, social media analysis and multilingual applications. She has received numerous honors and awards, including AAAI Fellow, a Founding Fellow of the Association for Computational Linguistics and an ACM Fellow. Early on she received the National Science Foundation Presidential Young Investigator Award, and a National Science Foundation Faculty Award for Women. In 2010, she won both the Columbia Great Teacher Award—an honor bestowed by the students—and the Anita Borg Woman of Vision Award for Innovation.
Title: Where Natural Language Processing Meets Societal Needs
The large amount of language available online today makes it possible to think about how to learn from this language to help address needs faced by society. In this talk, I will describe research in our group on summarization and social media analysis that addresses several different challenges. We have developed approaches that can be used to help people live and work in today’s global world, approaches to help determine where problems lie following a disaster, and approaches to identify when the social media posts of gang-involved youth in Chicago express either aggression or loss.
December 13th, 2019
Speaker: Prof. Mona Diab
Title: TBD
2018-2019
October 5, 2018
Speaker: Prof. Robert Frank
Bio:
Robert Frank is currently Professor and Chair of Linguistics at Yale University. He received his PhD at the University of Pennsylvania (Computer and Information Science) and has taught at Johns Hopkins University (Cognitive Science) and the University of Delaware (Linguistics). His research explores models of language learning and processing, as well as the role of computationally constrained grammar formalisms in linguistic explanation.
Title: Inductive Bias in Language Acquisition: UG vs. Deep Learning
Abstract:
Generative approaches to language acquisition emphasize the need for language-specific inductive bias, Universal Grammar (UG), to guide the hypotheses learners make in the face of limited data. In contrast, computational models of language learning, particularly those rooted in contemporary neural network models, have yielded significant advances in the performance of practical NLP systems, largely without the imposition of any such bias. While UG-based approaches have led to important insights into the stages and processes underlying language acquisition, they have not yielded concrete models of language. At the same time, existing practical computational models have not been widely tested with respect to their ability to extract linguistically significant generalizations from training data. As a result their ability to face the challenges posed by UG-based approaches remains unproven. In this talk, I will review a number of experiments that explore the ability of network models to take on such challenges. Looking at question formation and complementizer-trace effects, we find that (certain) network architectures are capable of learning significant grammatical generalizations through gradient descent learning, suggesting that the architectures themselves impose some of the necessary bias, often assumed to require UG. Inadequacies remain in the generalizations acquired, however, which points to the need for hybrid models that integrate language specific information into network models.
October 19, 2018
Speaker: Prof. Carolyn Penstein Rosé
Bio:
Dr. Carolyn Rosé is a Professor of Language Technologies and Human-Computer Interaction in the School of Computer Science at Carnegie Mellon University. Her research program is focused on better understanding the social and pragmatic nature of conversation, and using this understanding to build computational systems that can improve the efficacy of conversation between people, and between people and computers. In order to pursue these goals, she invokes approaches from computational discourse analysis and text mining, conversational agents, and computer supported collaborative learning. Her research group’s highly interdisciplinary work, published in over 200 peer reviewed publications, is represented in the top venues in 5 fields: namely, Language Technologies, Learning Sciences, Cognitive Science, Educational Technology, and Human-Computer Interaction, with awards in 3 of these fields. She serves as Past President and Inaugural Fellow of the International Society of the Learning Sciences, Senior member of IEEE, Chair of the International Alliance to Advance Learning in the Digital Era, Executive Editor of the International Journal of Computer-Supported Collaborative Learning, and Associate Editor of the IEEE Transactions on Learning Technologies.
Title: Who is the Bridge Between the What and the How
Abstract:
This talk reports on over a decade of research where theoretical foundations motivate computational models that produce real world impact in online spaces. Both the earliest philosophers of language and the most recent researchers in computational approaches to social media analysis have acknowledged the distinction between the what of language, namely its propositional content, and the how of language, or its form, style, or framing. What bridges between these realms are social processes that motivate the linguistic choices that result in specific realizations of propositional content situated within social interactions, designed to achieve social goals. These insights allow researchers to make sense of the connection between discussion processes and outcomes from those discussions. These findings motivate on the one hand design of computational approaches to real time monitoring of discussion processes and on the other hand the design of interventions that support interactions in online spaces with the goal of increasing desired outcomes, including learning, health, and wellbeing. As an example, in this talk we probe into a specific quality of discussion referred to as Transactivity. Transactivity is the extent to which a contribution articulates the reasoning of the speaker, that of an interlocutor, and the relation between them. In different contexts, and within very distinct theoretical frameworks, this construct has been associated with solidarity, influence, expertise transfer, and learning. Within the construct of Transactivity, the cognitive and social underpinnings are inextricably linked such that modeling the who enables prediction of the connection between the what and the how.
November 2, 2018
Speaker: Dr. Srinivas Bangalore
Bio:
Dr. Srinivas Bangalore (CV) is the Director of AI Research technologies at Interactions LLC. He was a Lead Inventive Scientist at Interactions (2015-2017) and a Principal Research Scientist at AT&T Labs–Research (1997-2014). He has a PhD in Computer Science from University of Pennsylvania and has made significant contributions to many areas of natural language processing including Spoken Language Translation, Multimodal Understanding, Language Generation and Question-Answering. He has co-edited three books on Supertagging, Natural Language Generation, and Language Translation, has authored over a 100 research publications and holds over 100 patents in these areas. Dr. Bangalore has been an adjunct associate professor at Columbia University (2005), a visiting professor at Princeton University (2008-present) and Otto Monstead Professor at Copenhagen Business School (2013). He has been awarded the Morris and Dorothy Rubinoff award for outstanding dissertation, the AT&T Outstanding Mentor Award, in recognition of his support and dedication to AT&T Labs Mentoring Program and the AT&T Science & Technology Medal for technical leadership and innovative contributions in Spoken Language Technology and Services. He has served on the editorial board of Computational Linguistics Journal, Computer, Speech and Language Journal and on program committees for a number of ACL and IEEE Speech Conferences.
Title: Opportunities and Challenges in Enterprise Conversational Virtual Agents
Abstract:
The field of language technologies continues to be transformed, fueled by ever increasing size of corpora that are processed on resource-demanding computing platforms with compute-intensive algorithms implemented in open source software. With the availability of shared tools, methods to solve disparate language problems are being unified, sidelining the idiosyncrasies of each problem. Tasks such as speech recognition, language translation, dialog management are increasingly formulated as end-to-end transductions that question the need for intermediate representations. It is in this context, we present the practical challenges and opportunities arising in a large-scale, speech-driven conversational agent deployed on a human-augmented platform that services over a billion calls a year for brand-name enterprises.
2017-2018
October 13, 2017
Speaker: Prof. Ani Nenkova
Bio:
Ani Nenkova is an Associate Professor of Computer and Information science at the University of Pennsylvania. Her main areas of research are computational linguistics and artificial intelligence, with emphasis on developing computational methods for analysis of text quality and style, discourse, affect recognition and summarization. She obtained her PhD degree in computer science from Columbia University.
Title: Style Analysis for Practical Semantic Interpretation of Text
Abstract:
Traditionally, natural language processing practitioners work under the assumption that the direct goal of text analysis is to ultimately derive a semantic interpretation of text. We explore alternatives to this tradition and instead focus on detecting style differences first, deferring or entirely foregoing semantic interpretation. This “style, then semantics if need be” approach to understanding reflects typical human behavior and may prove beneficial for many practical
applications of language processing. Under style we hope to capture how content is conveyed rather than exactly what facts are being communicated or what truth values one ought to assign to the expressed statements.
Main challenges in style analysis are the lack of clear definition of the required stylistic dimensions and firm understanding of the granularity on which style should be analyzed. Here we present initial task-dependent style analysis in the context of automatic summarization. We present results on word-, sentence- and paragraph-level and show first results connecting style analysis on each of these levels and the performance of an automatic summarizer.
These results are part of a long-term research agenda aiming to establish style analysis as an integral area of computational linguistics research and to elucidate the specific mechanisms via which style modulates and enhances the semantic interpretation of text.
November 3, 2017
Speaker: Prof. Diane Litman
Bio:
Diane is a Professor of Computer Science, and a Senior Scientist with the Learning Research and Development Center (LRDC), at the University of Pittsburgh. She received her Ph.D. and M.S. in Computer Science from the University of Rochester, and A.B. in Mathematics and Computer Science from the College of William and Mary in Virginia. Her research interests are in the areas of artificial intelligence and education, computational linguistics, knowledge representation and reasoning, natural language learning, spoken language, and user modeling.
Title: Argument Mining from Text and its Application in Education
Abstract:
The written arguments of students are educational data that can be automatically mined for purposes of student instruction and assessment. This talk will illustrate some of the opportunities and challenges in educationally-oriented argument mining from text. I will first describe how we are using natural language processing to develop argument mining systems that are being embedded in educational technologies for essay grading, peer review, and writing revision analysis. I will then present the results of empirical evaluations of these technologies, using argumentative writing data obtained from elementary, high school, and university students.
December 1, 2017
Speaker: Prof. Julia Hirschberg
Bio:
Julia Hirschberg, whose talk will be sponsored by the James F. Kelly Distinguished Lecture series, is Percy K. and Vida L. W. Hudson Professor of Computer Science and Chair of the Computer Science Department at Columbia University. She received her PhD in Computer Science from the University of Pennsylvania. She has numerous honors and awards, including a (founding) ACL Fellow since 2011, an ACM Fellow since 2016, a AAAI fellow since 1994, an ISCA Fellow since 2008, an IEEE Fellow since 2017, and a member of the National Academy of Engineering since 2017. She was elected to the American Philosophical Society in 2014 and as an Honorary Member of the Association for Laboratory Phonology in the same year. Her research interests in Natural Language Processing are in empirical and corpus-based studies of intonation (prosody) and discourse; automatic detection of emotion, charisma, and deception in speech; hedging behavior in text and speech; spoken dialogue systems; speech search for low resource languages; code-switching; text-to-speech synthesis; and annotation standards for speech corpora.
Title: Prosodic Entrainment in Dialogue: Language and Social Impact
Abstract:
In conversation, speakers often adapt aspects of their speaking style to the style of their conversational partner. This phenomenon goes by many names, including entrainment, adaptation, and alignment. In this talk, I will describe results from experiments on English and Mandarin prosodic entrainment in the Columbia Games Corpus and in the Tongji Games Corpus, large corpora of speech recorded from subjects playing a series of computer games. I also will discuss experiments relating entrainment to several social dimensions, including likeability and dominance, and its relationship to higher level prosodic features. Finally, I will describe an experiment with systems that entrain to their users in a set of “Go-Fish” games created in English, Porteño Spanish, and Slovak, as well as other ongoing research studying entrainment in deceptive speech and in linguistic code-switching. This is joint work with Štefan Beňuš, Nishmar Cestero, Agustín Gravano, Rivka Levitan, Sarah Ita Levitan, and Zhihua Xia.
January 26, 2018
Speaker: Prof. Jason Eisner
Bio:
Jason Eisner is Professor of Computer Science at Johns Hopkins University, where he is also affiliated with the Center for Language and Speech Processing, the Machine Learning Group, the Cognitive Science Department, and the national Center of Excellence in Human Language Technology. His goal is to develop the probabilistic modeling, inference, and learning techniques needed for a unified model of all kinds of linguistic structure. His 100+ papers have presented various algorithms for parsing, machine translation, and weighted finite-state machines; formalizations, algorithms, theorems, and empirical results in computational phonology; and unsupervised or semi-supervised learning methods for syntax, morphology, and word-sense disambiguation. He is also the lead designer of Dyna, a new declarative programming language that provides an infrastructure for AI research. He has received two school-wide awards for excellence in teaching.
Title: Recovering Syntactic Structure from Surface Features
Abstract:
We show how to predict the basic word-order facts of a novel language, and even obtain approximate syntactic parses, given only a corpus of part-of-speech (POS) sequences. We are motivated by the longstanding challenge of determining the structure of a language from its superficial features. While this is usually regarded as an unsupervised learning problem, there are good reasons that generic unsupervised learners are not up to the challenge. We do much better with a supervised approach where we train a system — a kind of language acquisition device — to predict how linguists will annotate a language. Our system uses a neural network to extract predictions from a large collection of numerical measurements. We train it on a mixture of real treebanks and synthetic treebanks obtained by systematically permuting the real trees, which we can motivate as sampling from an approximate prior over possible human languages.
2016-2017
November 11, 2016
Speaker: Prof. Barbara di Eugenio
Bio:
Barbara di Eugenio is a Professor of Computer Science at the University of Illinois at Chicago. Her main area of research is Natural Language Processing (NLP), and its application to human-computer interaction, educational technology, human-robot interaction, and multimedia systems. Her goal is to use NLP to support both education and instruction, and collaboration between human or artificial agents. The theoretical aspects of her research concern the linguistic analysis, and the knowledge representation and reasoning that support the understanding and generation of NL discourse and dialogue. Her research has its empirical foundations in both qualitative and quantitative corpus analysis, including data mining techniques.
Title: Towards a Dialogue System that supports Rich Visualizations of Data
Abstract:
The goal of the Articulate system is that of supporting a full-fledged conversation between a user and a system that transforms language queries into visualizations of data on large displays. So far, we have collected a corpus where users explore crime data from the city of Chicago via visualizations; we annotated the corpus for user intentions; and we have developed the core natural language-to-visualization pipeline which is able to process a sequence of requests, create the corresponding visualizations, position them on the screen, and manage them. The pipeline starts by classifying the input request into one of six types, then parsing it and applying semantic role labeling; first a logical form, then a SQL query and finally a visualization specification is derived.
December 2, 2016
Speaker: Prof. Roger Levy
Bio:
Roger Levy is an Associate Professor in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. His research focuses on theoretical and applied questions in the processing and acquisition of natural language. Linguistic communication involves the resolution of uncertainty over a potentially unbounded set of possible signals and meanings. How can a fixed set of knowledge and resources be deployed to manage this uncertainty? And how is this knowledge acquired? His research combines computational modeling, psycholinguistic experimentation, and analysis of large naturalistic language datasets. This work furthers our understanding of the cognitive underpinning of language processing and acquisition, and helps us design models and algorithms that will allow machines to process human language.
Title: Bayesian Pragmatics: Lexical Uncertainty, Compositionality, and the Typology of Conversational Implicature
Abstract:
A central scientific challenge for our understanding of human cognition is how language simultaneously achieves its unbounded, yet highly context-dependent expressive capacity. In constructing theories of this capacity, it has been productive to distinguish between strictly semantic content, or the “literal” meanings of atomic expressions (e.g., words) and the rules of meaning composition and pragmatic enrichment, by which speakers and listeners can rely on general principles of cooperative communication to take understood communicative intent far beyond literal content. Major open questions remain, however, of how to formalize pragmatic inference and characterize its relationship with semantic composition. Here I describe recent work within a Bayesian framework of interleaved semantic composition and pragmatic inference.
First, I show how two major principles of Levinson’s typology of conversational implicature fall out of our models: Q(uantity) implicature, in which utterance meaning is refined through exclusion of the meanings of alternative utterances; and I(nformativeness) implicature, in which utterance meaning is refined by strengthening to the prototypical case. Q and I are often in tension; I show that the Bayesian approach derives quantitative predictions regarding their relative strength in interpretation of a given utterance and present evidence supporting these predictions from a large-scale experiment on interpretation of utterances such as “I slept in a car” (was it my car, or someone else’s car?). I then turn to questions of compositionality, focusing on two of the most fundamental building blocks of semantic composition, the words “and” and “or”. Canonically, these words are used to coordinate expressions whose semantic content is least partially disjoint (“friends and enemies,” “sports and recreation”), but they can also be used to coordinate expressions whose literal semantic content is in a one-way inclusion relation (“boat or canoe” — c.f. Hurford, 1974; “roses and flowers”) or even in a two-way inclusion relation, or total semantic equivalence (“oenophile or wine-lover.”)
But why are these latter coordinate expressions used and how are they understood? Each class of these latter expressions falls out as a special case of our general framework, in which their prima facia inefficiency for communicating their literal content triggers a pragmatic inference that enriches the expression’s meaning in the same ways that we see in human interpretation. More broadly, these results illustrate the value of integrating recursive probabilistic models with formal semantic theories in the study of linguistic meaning and communication.