Robert Farrell, Rajarshi Das, et al.
AAAI-SS 2010
We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel and enables exact memory retrieval with exponential capacity without requiring exponential separation functions. Uniquely, it introduces abundant additional emergent local minima while preserving perfect pattern recovery--a characteristic previously unseen in DenseAM literature. Empirical results show LSR generates significantly more local minima and produces samples with higher log-likelihood than LSE-based models, making it promising for both memory storage and generative tasks.
Robert Farrell, Rajarshi Das, et al.
AAAI-SS 2010
Gabriele Dominici, Pietro Barbiero, et al.
ICLR 2025
Chen-chia Chang, Wan-hsuan Lin, et al.
ICML 2025
Seamus Somerstep, Felipe Maia Polo, et al.
ICLR 2025