New Policy Gradient Method Uses Hypervectors for Discrete Actions.

Researchers have developed a new reinforcement learning approach called Vector-Symbolic Policy Gradient (VSPG) for agents that perform discrete actions. This method represents each possible action using a high-dimensional vector, known as a hypervector, and determines the best action by comparing its hypervector to the encoded state of the environment. The VSPG framework proves that its learning update mechanism is equivalent to an advantage-weighted bundling of hypervectors, which allows it to leverage standard advantage estimation techniques. Each trained action hypervector effectively acts as a compressed memory, storing information about past states and transferring learning based on similarity. This design promises more sample-efficient learning without increasing the memory footprint during inference, and it offers a quantitative guarantee of robustness against minor data corruptions.