Did you investigate other search processes besides SGD? I'm thinking of those often termed "biologically plausible" (e.g. forward-forward, FA). Are their internal representations closer to the fractured or unified representations?
This is an interesting line of research but missing a key aspect: there's (almost) no references to the linear representation hypothesis. Much work on neural network interpretability lately has shown individual neurons are polysemantic, and therefore practically useless for explainability. My hypothesis is fitting linear probes (or a sparse autoencoder) would reveal linearly semantic attributes.
It is unfortunate because they briefly mention Neel Nanda's Othello experiments, but not the wide array of experiments like the NeurIPS Oral "Linear Representation Hypothesis in Language Models" or even golden gate Claude.
That is addressing the incomprehensibility of PCA and applying a transformation to the entire latent space. I've never found PCA to be meaningful for deep learning. As far as I can tell, polysemous issue with neurons cannot be addressed with a single linear transformation. There is no sparse analysis (via linear probes or SAEs) and hence the unaddressed issue.
Not quite. For an underlying semantic concept (e.g., smiling face), you can go from a basis vector [0,1,0,...,0] to the original latent space via a single rotation. You could then induce said concept by manipulating the original latent point by traversing along that linear direction.
I think we are saying the same thing. Please correct me though where I am wrong. You could look at the maps in some way but instead of the basis being one hot dimensions (the standard basis), it could be rotated.
I am glad they evaluated this hypothesis using weight decay which is primarily thought of to induce a structured representation. My first thought was that the entire paper was useless if they didn't do this experiment.
I find it rather interesting that the structured representations go from sparse to full to sparse as a function of layer depth. I have noticed that applying weight decay penalty as an exponential function of layer depth gives improved results over using a global weight decay.
> Much of the excitement in modern AI is driven by the observation that scaling up existing systems leads to better performance.
Scaling up almost always leads to better performance. If you're only getting linear gains though then there is absolutely nothing to be excited about. You are in a dead end.
"I looked at the representations of a network and I don't like them".
Great! There's no mathematical definition of what a fractured representation is. It's whatever art preferences you have.
Our personal preferences aren't a good predictor of which network will work well. We wasted decades with classical AI and graphical models encoding our aesthetic into models. Just to find out that the results are totally worthless.
Can we stop please? I get it. I too like beautiful things. But we can't hold on to things that don't work. Entire fields like linguistics are dying because they refuse to abandon this nonsense.
I believe they just mean that you should edit the comment where you added the links to mention that you are the author, to add that additional context.
I just meant 'it's good for people to know one of the authors is in the thread because it makes for more interesting conversation'. Clearly did not figure out how to do that without starting a bunch of meta!
I believe this could (or should) have been a Show HN, which would have allowed you to include explanatory text. See the top of this page for the rules.
I am interested in doing research like this. Is there any way I can be a part of it or a similar group? I have been fighting for funding from DoD for many years but to no avail so I largely have to do this research on my own time or solve my current grant's problems so that i can work on this. In my mind, this kind of research is the most interesting and important right now in the deep learning field. I am a hard worker and a high-throughput thinking... how can i get connected to otherwise with a similar mindset?
Did you investigate other search processes besides SGD? I'm thinking of those often termed "biologically plausible" (e.g. forward-forward, FA). Are their internal representations closer to the fractured or unified representations?
This is an interesting line of research but missing a key aspect: there's (almost) no references to the linear representation hypothesis. Much work on neural network interpretability lately has shown individual neurons are polysemantic, and therefore practically useless for explainability. My hypothesis is fitting linear probes (or a sparse autoencoder) would reveal linearly semantic attributes.
It is unfortunate because they briefly mention Neel Nanda's Othello experiments, but not the wide array of experiments like the NeurIPS Oral "Linear Representation Hypothesis in Language Models" or even golden gate Claude.
We mention this issue exactly in the fourth paragraph in Section 4 and in Appendix F!
That is addressing the incomprehensibility of PCA and applying a transformation to the entire latent space. I've never found PCA to be meaningful for deep learning. As far as I can tell, polysemous issue with neurons cannot be addressed with a single linear transformation. There is no sparse analysis (via linear probes or SAEs) and hence the unaddressed issue.
Is what your saying imply that there is a rotation matrix you can apply to each activation output to make it less entangled?
Not quite. For an underlying semantic concept (e.g., smiling face), you can go from a basis vector [0,1,0,...,0] to the original latent space via a single rotation. You could then induce said concept by manipulating the original latent point by traversing along that linear direction.
I think we are saying the same thing. Please correct me though where I am wrong. You could look at the maps in some way but instead of the basis being one hot dimensions (the standard basis), it could be rotated.
We mention this issue exactly in the fourth paragraph in Section 4 and in Appendix F!
Isn't this simply mirroronic gravitation?
I am glad they evaluated this hypothesis using weight decay which is primarily thought of to induce a structured representation. My first thought was that the entire paper was useless if they didn't do this experiment.
I find it rather interesting that the structured representations go from sparse to full to sparse as a function of layer depth. I have noticed that applying weight decay penalty as an exponential function of layer depth gives improved results over using a global weight decay.
> Much of the excitement in modern AI is driven by the observation that scaling up existing systems leads to better performance.
Scaling up almost always leads to better performance. If you're only getting linear gains though then there is absolutely nothing to be excited about. You are in a dead end.
"I looked at the representations of a network and I don't like them".
Great! There's no mathematical definition of what a fractured representation is. It's whatever art preferences you have.
Our personal preferences aren't a good predictor of which network will work well. We wasted decades with classical AI and graphical models encoding our aesthetic into models. Just to find out that the results are totally worthless.
Can we stop please? I get it. I too like beautiful things. But we can't hold on to things that don't work. Entire fields like linguistics are dying because they refuse to abandon this nonsense.
Tweet: https://x.com/kenneth0stanley/status/1924650124829196370 Arxiv: https://arxiv.org/abs/2505.11581
Sounds like you're one of the co-authors? Probably worth mentioning if the case so people know they can discuss the work with one of the work-doers.
I mentioned that in the original post, but I don't see that text here anymore (thats why I added links via comment)... I am new to hackernews
I believe they just mean that you should edit the comment where you added the links to mention that you are the author, to add that additional context.
I just meant 'it's good for people to know one of the authors is in the thread because it makes for more interesting conversation'. Clearly did not figure out how to do that without starting a bunch of meta!
I believe this could (or should) have been a Show HN, which would have allowed you to include explanatory text. See the top of this page for the rules.
https://news.ycombinator.com/show
Welcome to the site. There are a lot of features which are less obvious, which you’ll discover over time.
Reading material usually can't be a Show HN but you can just post your work without that and say you're involved.
The repo includes runnable code.
> Show HN is for something you've made that other people can play with… On topic: things people can run on their computers or hold in their hands
A lot of writing includes runnable code and isn't a Show HN. It's a comparatively narrow category.
I am interested in doing research like this. Is there any way I can be a part of it or a similar group? I have been fighting for funding from DoD for many years but to no avail so I largely have to do this research on my own time or solve my current grant's problems so that i can work on this. In my mind, this kind of research is the most interesting and important right now in the deep learning field. I am a hard worker and a high-throughput thinking... how can i get connected to otherwise with a similar mindset?