Akarsh Kumar

These are research projects I am currently working on or will do in future. Contact me if you want to collaborate or see my written detailed formalization of these projects.

Evolving/Meta-Learning Symbolically Extrapolative Generalization in NNs

My NSF GRFP Proposal

  • Symbolism was the first frontier of AI until connectionism with NNs took over.

  • Training NNs with SGD exploits statistical regularities in the training data to effectively solve the task.

  • However, the NN never learns the true structure in the data as humans do.

  • Human brains emerge symbolism from connectionism to achieve System 2 reasoning and understanding of data.

  • Can we meta-learn a learning algorithm to emerge symbols in an NN to understand the true structure in the data?

Evolving/Meta-Learning Optimal Sparsity and Weight Sharing in NNs

  • Currently, neural architecture search builds NNs from hard coded building blocks.

  • Can we instead learn the optimal building blocks themselves?

  • All NN building blocks (Conv, Attention, MLP-Mixer, etc.) carefully balance weight sparsity and weight sharing.

  • In this project, I formalize finding the best sparsity and weight sharing as an optimization problem.

  • This project will give insights into the next generation of NN architectures.

Go-Explore with Tabula Rasa Intelligent Exploration

Here is an idea allowing the agent to intelligently explore based on its past exploration successes.

  • We run normal Go-Explore early on and collect the archive of cells.

  • This gives a dataset where we can learn “what kinds of cells were promising/productive and led to new promising cells/rewards” and “what kinds of cells were unproductive and useless”.

  • Train a classifier to predict the productiveness of given cells.

  • Intrinsically motivate the agent to get to states/cells which the classifier thinks are productive.

  • The classifier may learn what is promising/productive on a global scale, and this global information may be used to achieve intelligent local exploration of the agent!

  • Also, by intelligently exploring locally, we will ignore many useless states, and thus our future archive will be smaller, making this algorithm scalable to large state/cell spaces.

A Goal Switching Approach to Multi-Task/Objective Learning.

  • Parameterize multi-task/objective problem as a grid of tasks (each cell is a task/problem).

  • Run MAP-Elites as you optimize solutions for specific tasks but also goal switch between tasks.

  • Can find a pseudo-optimal curriculum for each of the tasks.

  • If multi-task problem is really a multi-objective problem, it may find better pareto optimal solutions.

Evolving/Meta-Learning the Optimal Optimization Algorithm

  • SGD is a powerful optimization algorithm, and almost always beats EAs in differentiable problems.

  • However, QD has shown us that EAs with diversity is nessesary for to solve some deceptive problems.

  • Can we evolve/learn the optimal optimization algorithm?

  • Specifically, our search space should contain simple SGD, simple GAs (truncation selection), etc.

  • Will our final evolutionary controller be naturally QD and outperform SGD?

Two Man SGD

  • SGD keeps one point and follows its gradient.

  • What if we had two points and their gradients and created a clever crossover mechanism?

  • This new optimization algorithm has strictly more information than SGD and thus should be able to strictly outperform SGD.

  • This point-gradient crossover could be used in a GA as well.

  • I have developed a crossover mechanism that outperforms SGD in early parts of NN training.

Taming POET for Automatic Curriculum Learning

  • POET has shown us the importance of goal switching and QD populations in RL environments.

  • However, POET is Open-Ended (i.e. has no goal but to build complexity)

  • Can we tame POET to keep its Open-Endedness to explore early on, but later exploit its findings to create a curriculum for a target distribution of tasks?

Meta-Learning a Neuroevolution Crossover Mechanism

  • Crossover in neuroevolution is hard because of the competing conventions problem.

  • However, two different NNs which have learned different useful representations may be combined into a single NN which takes the best of both NNs.

  • Can we meta-learn a crossover mechanism NNs?