Loading content…
Independent (NUCES-FAST)
Neuro-Evolution for Effective Decision (NEED) — master's final year project at NUCES-FAST training and optimizing RL agents with minimal parameters (e.g. Lunar Lander in only 96 params) without backpropagation. Currently developing v2 for formal publication.
PythonPyTorchJAXEvoTorchRayGymnasiumEvolutionary AlgorithmsNeuroevolutionReinforcement LearningGradient-Free Optimization
NEED (Neuro-Evolution for Effective Decision) is an independent research framework developed during Master of Science in Data Science studies at NUCES-FAST. The project explores evolutionary algorithms to train and optimize compact neural policies from noise, completely gradient-free without backpropagation.
While backpropagation is optimal for supervised and unsupervised learning due to smooth, differentiable loss landscapes, it frequently struggles in reinforcement learning with sparse reward signals, high gradient variance, and deceptive local optima. NEED demonstrates that evolutionary algorithms excel in RL by directly searching the parameter space without gradient calculations, value estimators, or reward differentiability constraints.
Key Highlights & Achievements:
Traditional reinforcement learning relies on backpropagation, requiring massive compute, dense reward engineering, and millions of parameters that often fail under deceptive reward landscapes.
Engineered an evolutionary policy search engine leveraging PyTorch, JAX, EvoTorch, and Ray. Evaluates diverse populations across parallel Gymnasium environments, combining novelty search with fitness gating without gradient updates. Establishes the theoretical and empirical advantages of evolutionary optimization over backpropagation in reinforcement learning.
Achieved perfect autonomous Lunar Lander landings with only 96 parameters and top average scores. Recorded 500+ policy runs across 8 bodies; deployed live research portal at need.nomanali.online/v1 with v2 in active development for academic publication.
