firstbacksecondback
58 Results
|
Workshop
|
Approximations may be all you need: Towards Pre-training LLMs with Low-Rank Decomposition and Optimizers Namrata Shivagunde · Mayank Kulkarni · Giannis Karamanolakis · Jack FitzGerald · Yannick Versley · Saleh Soltan · Volkan Cevher · Jianhua Lu · Anna Rumshisky |
||
|
Poster
|
Fri 16:30 |
FouRA: Fourier Low-Rank Adaptation Shubhankar Borse · Shreya Kadambi · Nilesh Pandey · Kartikeya Bhardwaj · Viswanath Ganapathy · Sweta Priyadarshi · Risheek Garrepalli · Rafael Esteves · Munawar Hayat · Fatih Porikli |
|
|
Workshop
|
GaLore-mini: Low Rank Gradient Learning with Fewer Learning Rates WH Huang · Zhenyu Zhang · Yushun Zhang · Zhiquan Luo · Ruoyu Sun · Zhangyang "Atlas" Wang |
||
|
Poster
|
Wed 16:30 |
Metric Transforms and Low Rank Representations of Kernels for Fast Attention Timothy Chu · Josh Alman · Gary L. Miller · Shyam Narayanan · Mark Sellke · Zhao Song |
|
|
Workshop
|
Rethinking Fine-tuning Through Geometric Perspective Krishna Sri Ipsit Mantri · Moshe Eliasof · Carola-Bibiane Schönlieb · Bruno Ribeiro |
||
|
Poster
|
Wed 11:00 |
Fast Tree-Field Integrators: From Low Displacement Rank to Topological Transformers Krzysztof M Choromanski · Arijit Sehanobish · Somnath Basu Roy Chowdhury · Han Lin · Kumar Avinava Dubey · Tamas Sarlos · Snigdha Chaturvedi |
|
|
Poster
|
Thu 11:00 |
Compressing Large Language Models using Low Rank and Low Precision Decomposition Rajarshi Saha · Naomi Sagan · Varun Srivastava · Andrea Goldsmith · Mert Pilanci |
|
|
Workshop
|
StructMoE : Structured Mixture of Experts Using Low Rank Experts Zain Sarwar · Ashwinee Panda · Benjamin Thérien · Stephen Rawls · Anirban Das · Kartik Balasubramaniam · Berkcan Kapusuzoglu · Shixiong Zhang · Sambit Sahu · MILIND NAPHADE · Supriyo Chakraborty |
||
|
Workshop
|
Slaying the HyDRA: Parameter-Efficient Hyper Networks with Low-Displacement Rank Adaptation Xiangyu Chen · Ye Wang · Matthew Brand · Perry Wang · Jing Liu · Toshiaki Koike-Akino |
||
|
Poster
|
Thu 11:00 |
SLTrain: a sparse plus low rank approach for parameter and memory efficient pretraining Andi Han · Jiaxiang Li · Wei Huang · Mingyi Hong · Akiko Takeda · Pratik Kumar Jawanpuria · Bamdev Mishra |