Saddle-to-Saddle Dynamics with Simplicity Bias

# Saddle-to-Saddle Dynamics with Simplicity Bias ## Description This project investigates the hypothesis that saddle-to-saddle dynamics with simplicity bias can enhance the training efficiency of neural networks by dynamically adjusting learning rates during training. ## Research Hypothesis Saddle-to-saddle dynamics with simplicity bias can be leveraged to improve the training efficiency of neural networks by dynamically adjusting learning rates during training. ## Implementation Approach The implementation involves developing a dynamic learning rate scheduler that integrates saddle-to-saddle dynamics. Experiments will be conducted on various datasets and neural network architectures to evaluate the impact on training efficiency and generalization. ## Setup Instructions 1. Clone the repository. 2. Install the required dependencies using `pip install -r requirements.txt`. 3. Prepare the datasets (CIFAR-10, MNIST) in the data directory. ## Usage Run the training script: ```bash python src/train.py --dataset CIFAR-10 --model resnet18 ``` Evaluate the model: ```bash python src/evaluate.py --model checkpoints/model.pth ``` ## Expected Results The project aims to demonstrate improved training efficiency and generalization performance compared to traditional learning rate schedules. ## References - Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures [arXiv:2512.20607v1](http://arxiv.org/abs/2512.20607v1)