How Generative AI as a "writing aid" will transform businesses
Introducing AnswerChatAI - an OpenAI powered app that finds "needles in information haystacks" including multilingual Q&A, summarization, and paraphrasing - over custom documents
Unless your stuck under a rock, you’ve probably heard of ChatGPT, and the excitement as well as controversy surrounding it’s release and integration with Bing. In fact, the head of AI at Meta, said this about the LLM hype:
“current LLMs are useful as writing aids, but they make stuff up and are superficially impressive but very stupid.”
Let’s say that all that LLMs are useful only as impressive reading/writing aids. In of it self in my opinion, this is a major breakthrough. Think about what search engines do — they help us essentially find the right resources, instead of going through millions of web pages on the Internet. LLMs like ChatGPT solve this even further, by pinpointing key needles in document haystacks. So let’s take a look at the capabilities of an app AnswerChatAI that uses GPT under the hood to extract document specific information.
Information Extraction:
Say I’m scrolling through this article I wrote about how I transitioned from Academia to a Data Science role and you want to figure out who the author is. You can ask AnswerChatAI this question:
How I Transitioned from Academia To the Data Science Industry
One day I realized it was time for a new adventure. Here’s how small consistent efforts laid the foundations for my…towardsdatascience.com
It does a pretty good job of answering who I am, from the blog article context.
You can also try this out on research papers. This is where it gets really powerful if you wanted to obtain specific information about research outcomes. The paper introduces ClinicalBERT — basically pre-training the architecture of the BERT LLM model, using clinical notes. It does a pretty good job of answering specific questions like how does the new research model outperform the baseline model discussed in the paper.
Multilingual Q&A
Another really useful feature is for multilingual Q&A. I’ve noticed that if you feed in an article in a non-English language, it will reply with an answer corresponding to the language of the question.
In the example below, I ask the question Dónde nació el? (Where was he born) which is in Spanish and it returns the correct answer.
Summaries and Bullet Points
You can also use AnswerChatAI to summarize articles as below. You can also play with different prompts e.g. summarize in 3 sentences, or summarize prioritizing certain types of content, etc.
Another modification of summarizing that you can do is asking AnswerChatAI to list the key bullet points as in the book review below:
Room for improvement
All this is very promising but I would be remiss if I say there is no room or need for improvement. In the simple example below, the orginal article contains the sentence “I’m thirsty for friends.” So I would expect GPT to get the right answer when I ask it “Do I want friends?.”
It does get the correct answer however, when I ask “Do we need friends.” Here I must mention that the app is using OpenAI’s latest version of GPT available as an API (which is GPT-3), so not ChatGPT. But when the ChatGPT API is out, it might perform much better.
One big point of such LLM models raised is that of hallucination. Because these models are trained on pretty much the entire Internet, and are not restricted in how they can answer their questions, often their answers might not originate entirely from the contents of documents. The way we get around this in most cases is by prompting the LLM “Answer the following question based on the context below. If you are unsure, say ‘I don’t know’ ” or some similar prompt. But often, the LLM is like a 9 year old kid that you give directions to. They might follow orders most of the time, but other times they might just do whatever they prefer.
In the example above, the correct answer is given based on the context. But even thought the answer is factually correct, it is not present in the context. This in of itself is not an issue — maybe it is like a smart Alec student that gives an answer to a teacher that they did not ask for or jumps to conclusions. But the danger is if it generates specific outdated information like old financial records.
Takeaways
I believe LLMs are going to revolutionize how we understand, query, and process documents. Even if this might sound boring as compared to the latest hype over ChatGPT taking over software jobs, becoming conscious, or whatever the latest fad is — the underlying technology will lead to significant improvements across multiple industries. Companies will save precious time and resources by using a machine to do a bunch of traditionally labor intensive work. Employees will be more productive as they can use an AI tool to help find information and connect the dots across multiple company databases in a much quicker way than searching by keywords and then poring through documents.
However, the road towards ultimately every company using AI to power their document information tasks is non-trivial. First, these AI models have hundreds of billions of parameters and thus have latency issues — basically taking a few seconds or more per search, which is not ideal. There are multiple possible solutions from using fast vector databases, to caching documents and questions appropriately.
Second, generative AI that has been trained on wide amounts of data has the tendency to hallucinate. The last thing we want is a very confident QA bot telling us things that are not in the document in question, but from the general Internet.
The only way to know whether or not generative AI querying your custom documents works or gives incorrect answers — is to test it. If it works 80% or more of the time, that’s a good starting point. If it is 60%, then maybe you are better of not using this in your application. But unlike the hype around us, I don’t think it is a 1 or 0 — either ChatGPT or LLMs in general are mankinds saviour, or they will destroy civilization. The smart ones will realize the business value there is, and the chance to improve society — maybe incrementally at first.
https://www.answerchatai.com/ — our QA engine using generative AI to answer questions and extract key knowledge from custom text is now live! Answer domain specific questions 3 easy steps!
Upload a URL or paste a text and hit the search button
Ask a question specific to the context and hit query
Get your answer!
Feel free to use and let me know your feedback or if you want to apply this to your specific domain!