Bingbin Liu

PhD Student at Carnegie Mellon University
bingbinl [at] cs [dot] cmu [dot] edu
LinkedIn / Github / Google Scholar / CV (Feb 2024)



Welcome to my website! ^ ^ I am a postdoc research fellow at Simons Institute, participating in Special Year of Language Models and Modern Paradigm of Generalization. I'm broadly interested in bridging theoretical and empirical/scientific understanding of machine learning, often motivated by findings in synthetic ``sandbox'' settings. Some keywords for my work include reasoning, language models, and self-supervised learning.

I recently graduated from the Machine Learning Department of Carnegie Mellon University co-advised by Prof. Andrej Risteski and Prof. Pradeep Ravikumar. Previously, I was a master student in the Stanford Vision and Learning Lab, where I worked on video understanding and its applications to healthcare under the supervision of Prof. Fei-Fei Li, Prof. Juan Carlos Niebles, and Prof. Serena Yeung.

Here are some slides from my recent talk which may provide more information about my work.

Preprints and Publications

(Un)interpretability of Transformers: a case study with bounded Dyck grammars
Kaiyue Wen, Yuchen Li, Bingbin Liu, Andrej Risteski
NeurIPS23

Exposing Attention Glitches with Flip-Flop Language Modeling
Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang
NeurIPS23 (spotlight) [arxiv]

Transformers Learn Shortcuts to Automata
Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang
ICLR23 (oral) [arxiv / OpenReview / project page]

Masked prediction tasks: a parameter identifiability view
Bingbin Liu, Daniel Hsu, Pradeep Ravikumar, Andrej Risteski
NeurIPS 2022

Analyzing and improving the optimization landscape of noise-contrastive estimation
Bingbin Liu, Elan Rosenfeld, Pradeep Ravikumar, Andrej Risteski
ICLR 2022 (spotlight) [blog post]

Contrastive learning of strong-mixing continuous-time stochastic processes
Bingbin Liu, Pradeep Ravikumar, Andrej Risteski
AISTATS 2021

Generalized Boosting
Arun Sai Suggala, Bingbin Liu, Pradeep Ravikumar
NeurIPS 2020

Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction
Bingbin Liu, Ehsan Adeli, Zhangjie Cao, Kuan-Hui Lee, Abhijeet Shenoi, Adrien Gaidon, Juan Carlos Niebles
IEEE-RAL + ICRA 2020

A Computer Vision System to Detect Bedside Patient Mobilization
Serena Yeung*, Francesca Rinaldo*, Jeffrey Jopling, Bingbin Liu, Rishab Mehra, Lance Downing, Michelle Guo, Gabriel Bianconi, Alexandre Alahi, Julia Lee, Brandi Campbell, Kayla Deru, William Beninati, Li Fei-Fei, Arnold Milstein.
Nature Digital Medicine, 2019

Temporal Modular Networks for Retrieving Complex Compositional Activities in Videos
Bingbin Liu, Serena Yeung, Edward Chou, De-An Huang, Li Fei-Fei, Juan Carlos Niebles
European Conference on Computer Vision (ECCV), 2018
Also presented at Women in Computer Vision (WiCV) workshop.
Paper Project page Poster Video

Learning to Decompose and Disentangle Representations for Video Prediction
Jun-Ting Hsieh, Bingbin Liu, De-An Huang, Li Fei-Fei, Juan Carlos Niebles
Neural Information Processing Systems (NeurIPS), 2018
Paper Github Poster Video

3D Point Cloud-Based Visual Prediction of ICU Mobility Care Activities
Bingbin Liu*, Michelle Guo*, Edward Chou, Rishab Mehra, Serena Yeung, N. Lance Downing, Francesca Salipur, Jeffrey Jopling, Brandi Campbell, Kayla Deru, William Beninati, Arnold Milstein,
Machine Learning for Health Care (MLHC), 2018
Paper

Misc

I want to keep a list of good advice I've received from various friends and mentors. Please let me know if you have recommendations (especially since I'm doing a horrible job at gender balance here)!