Data Size Recovery from Lora Weights

Data Size Recovery from Lora Weights

Mohammad Salama, Jonathan Kahana, Eliahu Horwitz, Yedid Hoshen
The Hebrew University of Jerusalem
Paper Code arXiv 🤗 Dataset

Data Size Recovery (DSiRe)


We introduce DSiRe, a new method to determine the dataset size used to LoRA fine-tune a model based on its weights by extracting the singular values of each LoRA matrix and training layer-specific nearest-neighbor classifiers.

Abstract

Model inversion and membership inference attacks aim to reconstruct and verify the data which a model was trained on. However, they are not guaranteed to find all training samples as they do not know the size of the training set. In this paper, we introduce a new task: dataset size recovery, that aims to determine the number of samples used to train a model, directly from its weights. We then propose DSiRe, a method for recovering the number of images used to fine-tune a model, in the common case where fine-tuning uses LoRA. We discover that both the norm and spectrum of the LoRA matrices are closely linked to the fine-tuning dataset size; we leverage this finding to propose a simple yet effective prediction algorithm. To evaluate dataset size recovery of LoRA weights, we develop and release a new benchmark, LoRA-WiSE, consisting of over 25,000 weight snapshots from more than 2,000 diverse LoRA fine-tuned models. Our best classifier can predict the number of fine-tuning images with a mean absolute error of 0.36 images, establishing the feasibility of this attack.

Dataset Size Recovery

In this paper, We introduce the "Dataset Size Recovery" task and tackles it in cases where fine-tuning was performed via LoRA. Our initial findings reveal a strong correlation between the Frobenius norm and singular values of LoRA matrices with the fine-tuning dataset size.

Figure a
Figure b

Norm and Spectrum of Fine-Tuning Weights vs. Dataset Size Analaysis shows that the Frobenius norm and singular values of LoRA matrices have a clear negative correlation with the fine-tuning dataset size.

DSiRe

To solve this task we present DSiRe (Dataset Size Recovery), a method that recovers the dataset size directly from the LoRA weights. DSiRe leverages the spectrum of LoRA matrices to predict the number of samples used for fine-tuning. Our approach is simple yet highly effective, achieving high accuracy across a diverse datasets and models.

experience

DSiRe Confusion Matrix for Medium Data Range in a single experiment. Illustrating DSiRes accuracy in the range of 1-50 samples, shows that most of the errors are near misses, highlighting DSiRe's precision in dataset size recovery.

LoRa-WISE Benchmark

We present the LoRA Weight Size Evaluation (LoRA-WiSE) benchmark, a comprehensive benchmark specifically designed to evaluate LoRA dataset size recovery methods, for generative models. LoRA-WiSE is spanning different dataset sizes, backbones, ranks, and personalization sets

lora_wise

LoRA WISE Overview. The dataset comprises over 25,000 weights checkpoints drawn from more than 2000 independent LoRA models, spanning different dataset sizes, backbones, ranks, and personalization sets.

BibTeX

@article{salama2024dataset,
        title={Dataset Size Recovery from LoRA Weights},
        author={Salama, Mohammad and Kahana, Jonathan and Horwitz, Eliahu and Hoshen, Yedid},
        journal={arXiv preprint arXiv:2406.19395},
        year={2024}
      }