Publications (selected)

2024

·       Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers.  Krzysztof Choromanski, Shanda Li, Valerii Likhosherstov, Kumar Avinava Dubey, Shengjie Luo, Di He, Yiming Yang, Tamás Sarlós, Thomas Weingarten, Adrian Weller. AISTATS 2024: 2278-2286

·       Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLM.  Ruohong Zhang, Yau-Shian Wang, Yiming Yang. EACL (1) 2024: 659-673

·       Functional Interpolation for Relative Positions improves Long Context Transformers. Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontañón, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli. ICLR 2024

·       Learning Performance-Improving Code Edits. Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob R. Gardner, Yiming Yang, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, Amir Yazdanbakhsh. ICLR 2024

·       SALMON: Self-Alignment with Instructable Reward Models. Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Daniel Cox, Yiming Yang, Chuang Gan. ICLR 2024

·       Representation Learning and Information Retrieval. (video on YoutubeYiming Yang: SIGIR 2024 KEYNOTE.

·       Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination Syeda Nahida Akter, Aman Madaan, Sangwu Lee, Yiming Yang, Eric Nyberg. CoRR abs/2401.08025 (2024)

·       In-Context Principle Learning from MistakesTianjun Zhang, Aman Madaan, Luyu Gao, Steven Zheng, Swaroop Mishra, Yiming Yang, Niket Tandon, Uri Alon.  CoRR abs/2402.05403 (2024)

·       HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wild. Zhiying Zhu, Zhiqing Sun, Yiming Yang. CoRR abs/2403.04307 (2024)

·       Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, Chuang Gan (to appear ??? 2024)

·       Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward. Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, Yiming Yang. CoRR abs/2404.01258 (2024)

·       Self-Play Preference Optimization for Language Model Alignment. Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming YangQuanquan Gu: CoRR abs/2405.00675 (2024)

·       Few-shot Personalization of LLMs with Mis-aligned Responses. Jaehyung Kim, Yiming Yang. CoRR abs/2406.18678 (2024)

·       Learning to Correct for QA Reasoning with Black-box LLMs.  Jaehyung Kim, Dongyoung Kim, Yiming Yang. CoRR abs/2406.18695 (2024)

·       Lean-STaR: Learning to Interleave Thinking and Proving. Haohan Lin, Zhiqing Sun, Yiming Yang, Sean WelleckCoRR abs/2407.10040 (2024)

2023

§  PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification. Yau-Shian Wang, Ta-Chung Chi, Ruohong Zhang, .  ACL 14897-14911

§  PAL: Program-aided Language Models. Luyu GaoAman MadaanShuyan Zhou, Uri Alon, Pengfei Liu, Graham Neubig. ICML 10764-10799

§  A Neural PDE Solver with Temporal Stencil Modeling. Zhiqing SunShinjae Yoo. ICML 33135-33155

§  Retrieval-Enhanced Generative Model for Large-Scale Knowledge Graph Completion. Donghan Yu. SIGIR 2334-2338

§  Learning Performance-Improving Code Edits. Aman MadaanAlexander Shypula, Uri Alon, Milad Hashemi, Parthasarathy Ranganathan, Graham Neubig, Amir Yazdanbakhsh. Deep Learning for Code Workshop at ICLR 2023.

§  Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers. Krzysztof Marcin Choromanski, Shanda Li, Valerii Likhosherstov, Kumar Avinava Dubey, Shengjie Luo, Di He, Tamás Sarlós, Thomas Weingarten, Adrian Weller.  (Under Review)

§  DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization. Zhiqing Sun. NeurIPS 2023 (spotlight)

§  Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision. Zhiqing SunYikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David D. Cox, Chuang Gan. NeurIPS 2023 (spotlight)

§  Self-Refine: Iterative Refinement with Self-Feedback. Aman MadaanNiket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha DziriShrimai PrabhumoyeSean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, Peter Clark. NeurIPS 2023

§  Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-tuned GPT. Ruohong ZhangYau-Shian Wang. CoRR abs/2304.11872 ()

§  Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning with LLMs. Pranjal Aggarwal. EMNLP 2023.

§  Active Retrieval Augmented Generation. Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Jamie Callan, Graham Neubig. EMNLP 2023.

§  CompleQA: Benchmarking the Impacts of Knowledge Graph Completion Methods on Question Answering. Donghan Yu, Yu GuChenyan XiongYiming Yang. Findings of EMNLP 2023

§  Accelerating Diffusion-based Combinatorial Optimization Solvers by Progressive Distillation. Junwei HuangYiming Yang.  ICML’2023 workshop.

§  Recitation-Augmented Language Models. Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, Denny Zhou.  ICLR 2023.

§  Long-tailed Extreme Multi-label Text Classification with Generated Pseudo Label Descriptions.  Ruohong Zhang Lei: EACL 2023.

2022

§  DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problems.  Ruizhong Qiu*, *, : NeurIPS 2022.

§  Sparse Attention with Learning to Hash. Zhiqing SunYiming YangShinjae Yoo.  ICLR 

§  Exploiting Local and Global Features in Transformer-based Extreme Multi-label Text Classification. Ruohong Zhang, Yau-Shian Wang, Yiming Yang, Tom Vu, Likun Lei.  CoRR abs/2204.00933 (

§  Memory-assisted prompt editing to improve GPT-3 after deployment. Aman Madaan*, Niket Tandon*, Peter Clark, and Yiming Yang: EM-NLP 2022.

§  Conditional set generation using Seq2seq model. Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, and Antoine Bosselu: EM-NLP 2022.

§  Language Models of Code are Few-Shot Commonsense Learners (CoCoGen). Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, Graham Neubig: EM-NLP 2022.

§  FLOWGEN: Fast and slow graph generation.  Aman MadaanYiming Yang.  Dynamic Neural Networks Workshop at ICML 2022.

§  Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback. Niket Tandon*, Aman Madaan*, Peter Clark, and Yiming Yang: NAACL 2022 (Findings)

§  PAL: Program-aided Language Models. Luyu GaoJamie Callan, Graham Neubig. CoRR abs/2211.10435 ()

§  JAKET: Joint Pre-training of Knowledge Graph and Language Understanding.  Donghan Yu, Chenguang Zhu, Michael Zeng.  AAAI 2022: 11630-11638

§  KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering. Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Michael Zeng:  ACL (1) 20224961-4974

2021

§  Rethinking Transformer-based Set Prediction for Object Detection. [pdf] Zhiqing Sun, Shengcao Cao, Yiming Yang, Kris Kitani.  ICCV 2021.

§  Enhancing Summarization with Text Classification via Topic Consistency [pdf] Jingzhou Liu and Yiming Yang. ECML 2021.

§  Unsupervised Extractive Text Summarization with Distance-Augmented Sentence Graphs. [pdf] Jingzhou Liu, Dominic J. D. Hughes, . SIGIR 2313-2317

§  Could you give me a hint? Generating inference graphs for defeasible reasoning. [pdf] Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Eduard H. Hovy. ACL/IJCNLP (Findings) 5138-5147.

§  Meta Back-translation [pdf] Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig. ICLR 2021.

§  Generalized Multi-Relational Graph Convolution Networks [pdf]  Donghan Yu, Yiming Yang, Ruohong Zhang, Yuexin Wu. WWW 2021.

§  Neural language modeling for contextualized temporal graph generation [pdf] Aman Madaan and Yiming Yang.  NAACL 2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2005

2004

2003

2002

2001

2000

1999

1998

1997

1996

1995

1994

1993

1992