Akarsh Kumar

These are research projects I am currently working on or will do in future. Contact me if you want to collaborate or see my written detailed formalization of these projects.

Evolving/Meta-Learning Symbolically Extrapolative Generalization in NNs

My NSF GRFP Proposal

• Symbolism was the first frontier of AI until connectionism with NNs took over.

• Training NNs with SGD exploits statistical regularities in the training data to effectively solve the task.

• However, the NN never learns the true structure in the data as humans do.

• Human brains emerge symbolism from connectionism to achieve System 2 reasoning and understanding of data.

• Can we meta-learn a learning algorithm to emerge symbols in an NN to understand the true structure in the data?

Evolving/Meta-Learning Optimal Sparsity and Weight Sharing in NNs

• Currently, neural architecture search builds NNs from hard coded building blocks.

• Can we instead learn the optimal building blocks themselves?

• All NN building blocks (Conv, Attention, MLP-Mixer, etc.) carefully balance weight sparsity and weight sharing.

• In this project, I formalize finding the best sparsity and weight sharing as an optimization problem.

• This project will give insights into the next generation of NN architectures.

Go-Explore with Tabula Rasa Intelligent Exploration

Here is an idea allowing the agent to intelligently explore based on its past exploration successes.

• We run normal Go-Explore early on and collect the archive of cells.

• This gives a dataset where we can learn “what kinds of cells were promising/productive and led to new promising cells/rewards” and “what kinds of cells were unproductive and useless”.

• Train a classifier to predict the productiveness of given cells.

• Intrinsically motivate the agent to get to states/cells which the classifier thinks are productive.

• The classifier may learn what is promising/productive on a global scale, and this global information may be used to achieve intelligent local exploration of the agent!

• Also, by intelligently exploring locally, we will ignore many useless states, and thus our future archive will be smaller, making this algorithm scalable to large state/cell spaces.

A Goal Switching Approach to Multi-Task/Objective Learning.

• Parameterize multi-task/objective problem as a grid of tasks (each cell is a task/problem).

• Run MAP-Elites as you optimize solutions for specific tasks but also goal switch between tasks.

• Can find a pseudo-optimal curriculum for each of the tasks.

• If multi-task problem is really a multi-objective problem, it may find better pareto optimal solutions.

Evolving/Meta-Learning the Optimal Optimization Algorithm

• SGD is a powerful optimization algorithm, and almost always beats EAs in differentiable problems.

• However, QD has shown us that EAs with diversity is nessesary for to solve some deceptive problems.

• Can we evolve/learn the optimal optimization algorithm?

• Specifically, our search space should contain simple SGD, simple GAs (truncation selection), etc.

• Will our final evolutionary controller be naturally QD and outperform SGD?

Two Man SGD

• SGD keeps one point and follows its gradient.

• What if we had two points and their gradients and created a clever crossover mechanism?

• This new optimization algorithm has strictly more information than SGD and thus should be able to strictly outperform SGD.

• This point-gradient crossover could be used in a GA as well.

• I have developed a crossover mechanism that outperforms SGD in early parts of NN training.

Taming POET for Automatic Curriculum Learning

• POET has shown us the importance of goal switching and QD populations in RL environments.

• However, POET is Open-Ended (i.e. has no goal but to build complexity)

• Can we tame POET to keep its Open-Endedness to explore early on, but later exploit its findings to create a curriculum for a target distribution of tasks?

Meta-Learning a Neuroevolution Crossover Mechanism

• Crossover in neuroevolution is hard because of the competing conventions problem.

• However, two different NNs which have learned different useful representations may be combined into a single NN which takes the best of both NNs.

• Can we meta-learn a crossover mechanism NNs?