As long as the problem is dominated by heavy computational density — matrix multiplication is the poster child of this — GPUs or the specialized neural units will do very well. If the problem becomes ...

Keep in mind all the agent "sees" is a sequence of tokens and unless the model is specifically trained to be able to interpret that specific kind of spatial reasoning -- to line those up in columns -- ...