Timezone: »
How meaning is represented in the brain is still one of the big open questions in neuroscience. Does a word (e.g., bird) always have the same representation, or does the task under which the word is processed alter its representation (answering can you eat it?" versus
can it fly?")? The brain activity of subjects who read the same word while performing different semantic tasks has been shown to differ across tasks. However, it is still not understood how the task itself contributes to this difference. In the current work, we study Magnetoencephalography (MEG) brain recordings of participants tasked with answering questions about concrete nouns. We investigate the effect of the task (i.e. the question being asked) on the processing of the concrete noun by predicting the millisecond-resolution MEG recordings as a function of both the semantics of the noun and the task. Using this approach, we test several hypotheses about the task-stimulus interactions by comparing the zero-shot predictions made by these hypotheses for novel tasks and nouns not seen during training. We find that incorporating the task semantics significantly improves the prediction of MEG recordings, across participants. The improvement occurs 475-550ms after the participants first see the word, which corresponds to what is considered to be the ending time of semantic processing for a word. These results suggest that only the end of semantic processing of a word is task-dependent, and pose a challenge for future research to formulate new hypotheses for earlier task effects as a function of the task and stimuli.
Author Information
Mariya Toneva (Carnegie Mellon University)
Otilia Stretcu (Carnegie Mellon University)
Barnabas Poczos (Carnegie Mellon University)
Leila Wehbe (Carnegie Mellon University)
Tom Mitchell (CMU)
More from the Same Authors
-
2020 Session: Orals & Spotlights Track 35: Neuroscience/Probabilistic »
Leila Wehbe · Francisco Ruiz -
2020 Poster: Robust Density Estimation under Besov IPM Losses »
Ananya Uppal · Shashank Singh · Barnabas Poczos -
2020 Spotlight: Robust Density Estimation under Besov IPM Losses »
Ananya Uppal · Shashank Singh · Barnabas Poczos -
2019 Workshop: Sets and Partitions »
Nicholas Monath · Manzil Zaheer · Andrew McCallum · Ari Kobren · Junier Oliva · Barnabas Poczos · Ruslan Salakhutdinov -
2019 Poster: Nonparametric Density Estimation & Convergence Rates for GANs under Besov IPM Losses »
Ananya Uppal · Shashank Singh · Barnabas Poczos -
2019 Poster: Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain) »
Mariya Toneva · Leila Wehbe -
2019 Oral: Nonparametric Density Estimation & Convergence Rates for GANs under Besov IPM Losses »
Ananya Uppal · Shashank Singh · Barnabas Poczos -
2019 Poster: Learning Data Manipulation for Augmentation and Weighting »
Zhiting Hu · Bowen Tan · Russ Salakhutdinov · Tom Mitchell · Eric Xing -
2019 Poster: Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels »
Simon Du · Kangcheng Hou · Russ Salakhutdinov · Barnabas Poczos · Ruosong Wang · Keyulu Xu -
2019 Poster: Graph Agreement Models for Semi-Supervised Learning »
Otilia Stretcu · Krishnamurthy Viswanathan · Dana Movshovitz-Attias · Emmanouil Platanios · Sujith Ravi · Andrew Tomkins -
2019 Poster: Inducing brain-relevant bias in natural language processing models »
Dan Schwartz · Mariya Toneva · Leila Wehbe -
2019 Poster: Neural Taskonomy: Inferring the Similarity of Task-Derived Representations from Brain Activity »
Aria Wang · Michael Tarr · Leila Wehbe -
2019 Poster: Game Design for Eliciting Distinguishable Behavior »
Fan Yang · Liu Leqi · Yifan Wu · Zachary Lipton · Pradeep Ravikumar · Tom M Mitchell · William Cohen -
2019 Poster: Learning Local Search Heuristics for Boolean Satisfiability »
Emre Yolcu · Barnabas Poczos -
2018 Workshop: Learning by Instruction »
Shashank Srivastava · Igor Labutov · Bishan Yang · Amos Azaria · Tom Mitchell -
2018 Poster: Nonparametric Density Estimation under Adversarial Losses »
Shashank Singh · Ananya Uppal · Boyue Li · Chun-Liang Li · Manzil Zaheer · Barnabas Poczos -
2018 Poster: Learning Pipelines with Limited Data and Domain Knowledge: A Study in Parsing Physics Problems »
Mrinmaya Sachan · Kumar Avinava Dubey · Tom Mitchell · Dan Roth · Eric Xing -
2018 Poster: Neural Architecture Search with Bayesian Optimisation and Optimal Transport »
Kirthevasan Kandasamy · Willie Neiswanger · Jeff Schneider · Barnabas Poczos · Eric Xing -
2018 Spotlight: Neural Architecture Search with Bayesian Optimisation and Optimal Transport »
Kirthevasan Kandasamy · Willie Neiswanger · Jeff Schneider · Barnabas Poczos · Eric Xing -
2017 Oral: Deep Sets »
Manzil Zaheer · Satwik Kottur · Siamak Ravanbakhsh · Barnabas Poczos · Ruslan Salakhutdinov · Alexander Smola -
2017 Poster: Hypothesis Transfer Learning via Transformation Functions »
Simon Du · Jayanth Koushik · Aarti Singh · Barnabas Poczos -
2017 Poster: MMD GAN: Towards Deeper Understanding of Moment Matching Network »
Chun-Liang Li · Wei-Cheng Chang · Yu Cheng · Yiming Yang · Barnabas Poczos -
2017 Poster: Deep Sets »
Manzil Zaheer · Satwik Kottur · Siamak Ravanbakhsh · Barnabas Poczos · Ruslan Salakhutdinov · Alexander Smola -
2017 Poster: Gradient Descent Can Take Exponential Time to Escape Saddle Points »
Simon Du · Chi Jin · Jason D Lee · Michael Jordan · Aarti Singh · Barnabas Poczos -
2017 Spotlight: Gradient Descent Can Take Exponential Time to Escape Saddle Points »
Simon Du · Chi Jin · Jason D Lee · Michael Jordan · Aarti Singh · Barnabas Poczos -
2017 Poster: Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approach »
Emmanouil Platanios · Hoifung Poon · Tom M Mitchell · Eric Horvitz -
2016 Poster: Variance Reduction in Stochastic Gradient Langevin Dynamics »
Kumar Avinava Dubey · Sashank J. Reddi · Sinead Williamson · Barnabas Poczos · Alexander Smola · Eric Xing -
2016 Poster: The Multi-fidelity Multi-armed Bandit »
Kirthevasan Kandasamy · Gautam Dasarathy · Barnabas Poczos · Jeff Schneider -
2016 Poster: Finite-Sample Analysis of Fixed-k Nearest Neighbor Density Functional Estimators »
Shashank Singh · Barnabas Poczos -
2016 Poster: Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization »
Sashank J. Reddi · Suvrit Sra · Barnabas Poczos · Alexander Smola -
2016 Poster: Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations »
Kirthevasan Kandasamy · Gautam Dasarathy · Junier B Oliva · Jeff Schneider · Barnabas Poczos -
2016 Poster: Efficient Nonparametric Smoothness Estimation »
Shashank Singh · Simon Du · Barnabas Poczos -
2015 Poster: Nonparametric von Mises Estimators for Entropies, Divergences and Mutual Informations »
Kirthevasan Kandasamy · Akshay Krishnamurthy · Barnabas Poczos · Larry Wasserman · james m robins -
2015 Poster: On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants »
Sashank J. Reddi · Ahmed Hefny · Suvrit Sra · Barnabas Poczos · Alexander Smola -
2014 Workshop: 4th Workshop on Automated Knowledge Base Construction (AKBC) »
Sameer Singh · Fabian M Suchanek · Sebastian Riedel · Partha Pratim Talukdar · Kevin P Murphy · Christopher RĂ© · William Cohen · Tom Mitchell · Andrew McCallum · Jason E Weston · Ramanathan Guha · Boyan Onyshkevych · Hoifung Poon · Oren Etzioni · Ari Kobren · Arvind Neelakantan · Peter Clark -
2014 Poster: Exponential Concentration of a Density Functional Estimator »
Shashank Singh · Barnabas Poczos -
2011 Poster: Group Anomaly Detection using Flexible Genre Models »
Liang Xiong · Barnabas Poczos · Jeff Schneider -
2010 Poster: Estimation of Renyi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs »
David Pal · Barnabas Poczos · Csaba Szepesvari -
2009 Poster: Zero-shot Learning with Semantic Output Codes »
Mark M Palatucci · Dean Pomerleau · Geoffrey E Hinton · Tom Mitchell -
2008 Workshop: Parallel Implementations of Learning Algorithms: What have you done for me lately? »
Robert Thibadeau · Dan Hammerstrom · David S Touretzky · Tom Mitchell -
2008 Workshop: Parallel Implementations of Learning Algorithms: What have you done for me lately? »
Robert Thibadeau · David S Touretzky · Dan Hammerstrom · Tom Mitchell -
2006 Workshop: New directions on decoding mental states from fMRI data »
John-Dylan Haynes · Tom Mitchell · Francisco Pereira