This is irrelevant, and it's very frustrating that computer scientists think it is relevant.

If you give a universal function approximator the task of approximating an abstract function, you will get an approximation.

Eg.,

    def circle(radius): ... return points()
    aprox_cricle = neuralnetwork(sample(circle()))
    
    if is_model_of(samples(aprox_circle), circle)): print("OF COURSE!")

This is irrelevant: games, rules, shapes, etc. are all abstract. So any model of samples of these is a model of them.

The "world model" in question is a model of the world. Here "data" is not computer science data, ie., numbers its measurements of the world, ie., the state of a measuring device causally induced by the target of measurement.

Here there is no "world" in the data, you have to make strong causal assumptions about what properties of the target cause the measures. This is not in the data. There is no "world model" in measurement data. Hence the entirety of experimental science.

No result based on one mathematical function succeeding in approximating another is relevant whether measurement data "contains" a theory of the world which generates it: it does not. And of course if your data is abstract, and hence constitutes the target of modelling (only applies to pure math), then there is no gap -- a model of "measures" (ie., the points on a circle) is the target.

No model of actual measurement data, ie., no model in the whole family we call "machine learning", is a model of its generating process. It contains no "world model".

Photographs of the night sky are compatible with all theories of the solar system in human history (including, eg., stars are angels). There is no summary of these photographs which gives information about the world over and above just summarising patterns in the night sky.

The sense in which any model of measurement data is "surface statistics" is the same. Consider plato's cave: pots, swords, etc. on the outside project shadows inside. Modelling the measurement data is taking cardboard and cutting it out so it matches the shadows. Modelling the world means creating clay pots to match the ones passing by.

The latter is science: you build models of the world and compare them to data, using the data to decide between them.

The former is engineering (, pseudoscience): you take models of measures and reply these models to "predict" the next shadow.

If you claim the latter is just a "surface shortcut" you're an engineer. If you claim its a world model you're a pseudoscientist.

Do Large Language Models learn world models or just surface statistics? (2023)