Replacing the 'difficulty-alignment' objective with an 'Information Gain' objective (maximizing the reduction in agent uncertainty) will accelerate convergence rates for complex reasoning tasks compared to standard performance-based curriculum learning.

Motivation

GenEnv currently aligns difficulty based on performance (pass/fail). However, a task can be 'hard' (50% pass rate) purely due to aleatoric uncertainty (randomness) rather than epistemic uncertainty (lack of knowledge). Targeting tasks where the agent is *uncertain* about the reasoning path (high entropy in generation) is theoretically more efficient for learning.

Proposed Method

Modify the Environment Simulator's reward function. Instead of targeting a specific success rate, reward the Simulator for generating tasks that maximize the KL-divergence between the Agent's current belief state and the posterior after attempting the task (or simply maximizing prediction entropy). Compare the sample efficiency of this 'Active Co-Evolution' against the original GenEnv on complex reasoning benchmarks like MATH or HumanEval.

Expected Contribution

A more theoretically grounded objective for environment co-evolution that distinguishes between 'hard because random' and 'hard because unknown,' leading to faster training.

Required Resources

Standard GenEnv compute setup, plus additional compute overhead for estimating agent uncertainty (e.g., via dropout ensembles or log-prob analysis).

Source Paper

GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators

View Paper Details →