What are the newest great techniques to improve NLP/NLU transformer model performances?

Question

Accepted Answer

I have heard about the benefits of removing stop words in text processing is one of the important steps. It needs knowledge of the problem statement and expertise of NLP too. Also, in essence, this is AI, and for NLP applications in finance, a high-quality team and the right architecture are important

Pascal de Buren · Answer

I personally love the ideas around adding an external memory to transformers since it's a simple way to add new vocabulary without training a 100bn+ transformer model. I found this paper by Google on Memorizing Transformers especially insightful: https://arxiv.org/abs/2203.08913

What are the newest great techniques to improve NLP/NLU transformer model performances?

Replies