How do you fix chatbot errors?
Enis Gayretli
3 replies
Do you have any handling tips?
Replies
Leif Andreas Rudlang@leif_from_emberly
Emberly
What kind of errors, is this related to the ChatGPT API?
Share
@enisgayretli A-ha, that is actually a tough one!
If the chatbot is based on a LLM like ChatGPT is, this can never fully be resolved due to the way it works. However, there seems to be a understanding among users that the answers can be incorrect, and that is okay.
To understand why this is, a good tool is to think of the LLM as a fancy autocorrect, it does not think like a human does, rather it takes in some text, and then guesses what comes next based on previous examples. Its just statistics.
If the chatbot has to be correct, one possible approach is using good "old fashioned" NLP to try to guess what the user is looking for, then returning "hardcoded" answers.
This can be spiced up with an LLM for example, you can take the users question, along with your pre-approved answers, and ask the LLM to return the answer that is most likely to fulfill the users query.
If there is margin for error, you can do a lot of things to reduce the error rate (but it will be never be 0), such as providing the information you want the answer to be based on in the prompt, and telling the chatbot to only use that information to generate the answer.