Mor Geva Pipek

Mor Geva Pipek

I am an Assistant Professor (Senior Lecturer) at the School of Computer Science at Tel Aviv University and a Visiting Researcher at Google Research. My research is in the field of Natural Language Processing. I work on understanding the inner workings of large language models, their limitations and capabilities. I am also interested in leveraging insights on how models operate for practical applications and for developing transparent and robust models. I completed a Ph.D. in Computer Science and a B.Sc. in Bioinformatics at Tel Aviv University, and was a postdoctoral researcher at Google DeepMind and at the Allen Institute for AI (AI2).


Checkpoint 255
morgeva@tauex.tau.ac.il

Students

Daniela Gottesman
Ohav Barbi
Amit Elhelo
Yoav Gur Arieh

Alumni

Daniela Gottesman M.Sc. 2024 → to start Ph.D.
Amit Arnold Levy Guest Student 2024
Dana Ramati Project Student 2024

Publications

Towards Interpreting Visual Information Processing in Vision-Language Models
Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, Fazl Barez. 2024.
CoverBench: A Challenging Benchmark for Complex Claim Verification
Alon Jacovi, Moran Ambar, Eyal Ben-David, Uri Shaham, Amir Feder, Mor Geva, Dror Marcus, Avi Caciularu. 2024.
When Can Transformers Count to n?
Gilad Yehudai, Haim Kaplan, Asma Ghandeharioun, Mor Geva, Amir Globerson. 2024.
From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty
Maor Ivgi, Ori Yoran, Jonathan Berant, Mor Geva. 2nd Workshop on Attributing Model Behavior at Scale, NeurIPS 2024.
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries
Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, Amir Globerson. EMNLP 2024.
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP
Marius Mosbach, Vagrant Gautam, Tomás Vergara-Browne, Dietrich Klakow, Mor Geva. EMNLP 2024.
Intrinsic Evaluation of Unlearning Using Parametric Knowledge Traces
Yihuai Hong, Lei Yu, Haiqin Yang, Shauli Ravfogel, Mor Geva. 2024.
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
Shahar Katz, Yonatan Belinkov, Mor Geva, Lior Wolf. EMNLP 2024.
RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations
Jing Huang, Zhengxuan Wu, Christopher Potts, Mor Geva, Atticus Geiger. ACL 2024.
Do Large Language Models Latently Perform Multi-Hop Reasoning?
Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva*, Sebastian Riedel*. ACL 2024.
The Hidden Space of Transformer Language Adapters
Jesujoba O. Alabi, Marius Mosbach, Matan Eyal, Dietrich Klakow, Mor Geva. ACL 2024.
A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, Mor Geva. ACL 2024.
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models
Asma Ghandeharioun*, Avi Caciularu*, Adam Pearce, Lucas Dixon, Mor Geva. ICML 2024.
Jump to Conclusions: Short-Cutting Transformers With Linear Transformations
Alexander Yom Din, Taelin Karidi, Leshem Choshen, Mor Geva. LREC-COLING 2024.
The Hidden Language of Diffusion Models
Hila Chefer, Oran Lang, Mor Geva, Volodymyr Polosukhin, Assaf Shocher, Michal Irani, Inbar Mosseri, Lior Wolf. ICLR 2024.
Evaluating the Ripple Effects of Knowledge Editing in Language Models
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, Mor Geva. TACL 2024.
In-Context Learning Creates Task Vectors
Roee Hendel, Mor Geva, Amir Globerson. Findings of EMNLP 2023.
CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks
Mete Ismayilzada, Debjit Paul, Syrielle Montariol, Mor Geva, Antoine Bosselut. EMNLP 2023.
A Comprehensive Evaluation of Tool-Assisted Generation Strategies
Alon Jacovi, Avi Caciularu, Jonathan Herzig, Roee Aharoni, Bernd Bohnet, Mor Geva. Findings of EMNLP 2023.
LM vs LM: Detecting Factual Errors via Cross Examination
Roi Cohen, May Hamri, Mor Geva, Amir Globerson. EMNLP 2023.
Dissecting Recall of Factual Associations in Auto-Regressive Language Models
Mor Geva, Jasmijn Bastings, Katja Filippova, Amir Globerson. EMNLP 2023.
Analyzing Transformers in Embedding Space
Guy Dar, Mor Geva, Ankit Gupta, Jonathan Berant. ACL 2023
Crawling the Internal Knowledge-Base of Language Models
Roi Cohen, Mor Geva, Jonathan Berant, Amir Globerson. Findings of EACL 2023.
Understanding Transformer Memorization Recall Through Idioms
Adi Haviv, Ido Cohen, Jacob Gidron, Roei Schuster, Yoav Goldberg, Mor Geva. EACL 2023.
Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models
I was fortunate to be part of this work by 442 contributors across 132 institutions. TMLR 2023.
Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions
Mihir Parmar*, Swaroop Mishra*, Mor Geva, Chitta Baral. EACL 2023.
Inferring Implicit Relations with Language Models
Uri Katz, Mor Geva, Jonathan Berant. Findings of EMNLP 2022.
LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models
Mor Geva, Avi Caciularu, Guy Dar, Paul Roit, Shoval Sadde, Micah Shlain, Bar Tamir, Yoav Goldberg. System Demonstrations Track, EMNLP 2022.
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
Mor Geva*, Avi Caciularu*, Kevin Ro Wang, Yoav Goldberg. EMNLP 2022.
SCROLLS: Standardized CompaRison Over Long Language Sequences
Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, Omer Levy. EMNLP 2022.
What's in your Head? Emergent Behaviour in Multi-Task Transformer Models
Mor Geva, Uri Katz, Aviv Ben-Arie, Jonathan Berant. EMNLP 2021.
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, Jonathan Berant. TACL 2021.
Transformer Feed-Forward Layers Are Key-Value Memories
Mor Geva, Roei Schuster, Jonathan Berant, Omer Levy. EMNLP 2021.
Injecting Numerical Reasoning Skills into Language Models
Mor Geva*, Ankit Gupta*, Jonathan Berant. ACL 2020.
Break It Down: A Question Understanding Benchmark
Tomer Wolfson, Mor Geva, Ankit Gupta, Matt Gardner, Yoav Goldberg, Daniel Deutch, Jonathan Berant. TACL 2020.
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion
Mor Geva, Eric Malmi, Idan Szpektor, Jonathan Berant. NAACL 2019.
Emergence of Communication in an Interactive World with Consistent Speakers
Ben Bogin, Mor Geva, Jonathan Berant. Emergent Communication Workshop, NIPS 2018.

Patents

Training Set Sufficiency for Custom Face Recognition , joint with Oron Nir. Issued May 11, 2018, us 404234-US-NP.
Media Management System for Video Data Processing and Adaptation Data Generation . June, 2018, us 403684-US-NP.
Methods for Consolidating OCR Detection in Video , joint with Oron Nir. June, 2018, us 403687-US-PSP.

Teaching

Natural Language Processing
Spring 2024/25
Seminar on Interpretability of Large Language Models
Fall 2024/25
Seminar on Interpretability of Large Language Models
Spring 2023/24
Introduction to Machine Learning
Teaching Assistant, Fall 2018/19
NVIDIA DLI Workshop
University Ambassador, 2018/19