Can Transformers Learn Logical Reasoning from Scratch?

In this research, we address the more general question of what kinds of goals are learnable from scratch by examining whether Transformer architectures can learn and construct new syllogisms by leveraging existing ones. Despite having Turing-complete expressivity, Transformers' ability to learn effectively is still limited.