Reinforcement Learning-driven adaptive compression rates for SparseLoCo will prevent training divergence in volatile public-internet environments compared to static low-bandwidth configurations.

Motivation

The source paper assumes low but stable bandwidth. In real-world 'volunteer computing' or spot-instance scenarios, bandwidth fluctuates and packet loss varies. A static sparsity threshold leads to either under-utilization or training instability (divergence) when network conditions degrade rapidly.

Proposed Method

Implement a lightweight RL agent that monitors network jitter, latency, and packet loss in real-time. The agent dynamically adjusts the gradient sparsity threshold and pipeline buffer size of the Heterogeneous Low-Bandwidth protocol. Test this on a cluster with artificially induced network chaos (using `tc` or similar tools) to measure training stability (loss spikes) and effective throughput.

Expected Contribution

A robust protocol enabling LLM pre-training over unstable, decentralized consumer hardware networks (e.g., similar to Folding@Home but for LLMs).

Required Resources

Cluster of consumer-grade GPUs (e.g., RTX 3090s/4090s), network simulation tools, expertise in RL and distributed systems.

Source Paper

Heterogeneous Low-Bandwidth Pre-Training of LLMs

View Paper Details →