SPEAR: Exploiting ReLU-induced Sparsity to Break Linear Layer Gradient Aggregation

The code accompanying our ICML 2024 submission

Prerequisites

• Install Anaconda.
• Create the conda environment:
  

conda env create -f env.yml

• Enable the created environment:
  

conda activate sparsity_leakage

Main experiments (at initalization)

Call main.py

Parameters

• --B - Batch size to be attacked e.g 20.
• --L - Number of layers in the FCNN e.g 6.
• --W - The widht of the layers in the FCNN e.g 200.
• --N - Maximum number of samples to be used e.g 10000000000.
• --dataset - The dataset to be used. Must be one of MNIST, CIFAR10, TinyImgNet.

See more in by running:

python main.py --help

Example Command

• To run our attack on TinyImageNet you can run the following command:
  

python main.py --B 20 --W 200 --L 6 --N 2000000000 --cond early --sigma_tol 1e-7 --sigma_treshold 0.98 --dataset TinyImgNet --st 0 --en 100 --sparsity_tol 1e-5

Experiments during training

Call train.py

Parameters

• --B - Batch size to be attacked e.g 20.
• --L - Number of layers in the FCNN e.g 6.
• --W - The widht of the layers in the FCNN e.g 200.
• --N - Maximum number of samples to be used e.g 10000000000.
• --dataset - The dataset to be used. Must be one of MNIST, CIFAR10, TinyImgNet.
• --ds_type - Whether to attack the training set or test set. Must be one of test or train.
• --steps - How many steps to train the model for before attacking it e.g. 5000.

See more in by running:

python train.py --help

Example Command

• To run our attack on the test set after 5000 training steps you can run the following command:
  

python train.py --B 10 --W 200 --L 6 --N 500000 --cond early --sigma_tol 1e-8 --sigma_treshold 0.99 --dataset MNIST steps --st 0 --en 100 --sparsity_tol 1e-5 --par-SVD 10000 --steps 5000 --pFN 0.4 --ds_type test