I can see why Mordvintsev et al are up to what they are doing, but to be honest I'm struggling with understanding the point of using a neural-net to 'emulate' CAs like OP seems to be doing (and as far as I can gather, only totalistic ones too?).
It sounds a bit like swatting a fly using an H-bomb tbh, but maybe someone who knows more about the project can share some of the underlying rationale?
friedchips 21 hours ago [-]
I'm not involved in this project, but I partially replicated the results from Mordvintsev et al. a few years ago because I found the idea interesting. The key idea for me was learning the possibly unknown rules of a CA from training examples. This sounded to me like something that could be useful in science, to learn about spatially distributed processes. Or in ML as a new idea for image classification or segmentation. The hope was always that a CA could be learned which would have a simple discrete representation which could then be used in inference with much lower computational needs than a full neural net. But unfortunately we never managed to succeed here, and I have the impression that this area is not as active anymore as it was some years ago.
I suppose the idea of this project is the same: show the correspondence between both in order to understand them better.
I think the biggest advantage NNs have over CA is the fact that most CA only provide localized computation. It can take a large number of fixed iterations before information propagates to the appropriate location in the 1d/2d/3d/etc. space. Contrast this with arbitrary NN topology where instant global connectivity is possible between any elements.
Tzt 24 hours ago [-]
CNNs are CA if you don't insert fully connected layers, actually.
evilmathkid 21 hours ago [-]
You also need to make the CNN recurrent, allow it to unfold over many steps, ensure input and output grid are same size and avoid non-local stuff like global pooling, certain norms, etc.
Either way, parent comment is correct. An arbit NN is better than a CA at learning non-local rules unless the global rule can be easily described as a composition of local rules. (They still can learn any global rule though, its just harder and you run into vanishing gradient problems for very distant rules)
They are pretty cool with emergent behaviors and sometimes they generalise very well
https://www.neuralca.org/
https://google-research.github.io/self-organising-systems/di...
https://google-research.github.io/self-organising-systems/is...
I can see why Mordvintsev et al are up to what they are doing, but to be honest I'm struggling with understanding the point of using a neural-net to 'emulate' CAs like OP seems to be doing (and as far as I can gather, only totalistic ones too?).
It sounds a bit like swatting a fly using an H-bomb tbh, but maybe someone who knows more about the project can share some of the underlying rationale?
I suppose the idea of this project is the same: show the correspondence between both in order to understand them better.
Anyway, some interesting papers from back then:
Cellular automata as convolutional neural networks: http://arxiv.org/abs/1809.02942
Image segmentation via Cellular Automata: http://arxiv.org/abs/2008.04965
It's Hard for Neural Networks To Learn the Game of Life: http://arxiv.org/abs/2009.01398
Either way, parent comment is correct. An arbit NN is better than a CA at learning non-local rules unless the global rule can be easily described as a composition of local rules. (They still can learn any global rule though, its just harder and you run into vanishing gradient problems for very distant rules)
They are pretty cool with emergent behaviors and sometimes they generalise very well
It seems like a passion project and a niche interest by the author.
I clicked in the hope that it would tell me something about how CAs can be 'trained' and 'used' to make useful predictions somehow.
Instead, I got a neural network which is trained to predict the t+3 step of a CA based on an initial state.
Am I missing something?
IMHO this is semi interesting because having ANNs predict the outcome of deterministic dynamical systems may help with some planning tasks.