 How do we go from recurrent neural networks to sentence transformers? Recurrent neural networks can translate vectors to sequences, sequence to vectors, and sequence to sequences. Transformers replace recurrent neural networks that sequence to sequence tasks since they are easier to train and handle long sequences pretty well. BERT and GPT are language models that specifically focus on solving NLP tasks like question answering and text summarization. They can be pre-trained to understand language at a fundamental level, and then fine-tuned on any specific language task. To solve a class of problems where we want to compare multiple inputs in order to generate an output for natural language tasks, sentence transformers become useful. Examples are like determining sentence similarity to rank responses to a certain question like you'd see on Quora for example.