I programmed an AI chatbot! It is a retrieval augmented generation (RAG) model. It has been trained on the content of my professional blog here at koplowiczandsons.com. It can now answer questions based on the blog’s information.
I actually wrote the bot a year ago, and hooked it up to a Discord server, but never took it further than that. The challenge here was to hook up the bot to a website instead of Discord. It's a completely different type of challenge, because Discord handles all the api calls. With my website, I had to handle api calls myself. That means a third server:
I recently upgraded koplowiczandsons.com to the latest version of Drupal, and used the opportunity to turn it into a headless Drupal site, with a React frontend. As part of this process, I upgraded the web server from Apache to Nginx (which I already blogged about). So now I have a LEMP stack - but that doesn't get me the frontend, or the chatbot.
The frontend requires node, which needs to be compiled after it is deployed. I solve this with some CI/CD, courtesy of Github Actions. (Github makes this process easy, since it's where my code is stored anyway.) So now, I can push the frontend and backend simultaneously, and the CI takes the following steps:
- For the backend:
- composer install
- drush deploy
- And for the frontend:
- npm install
- npm run build
All I have to do is a simple "git push," and it deploys everything for me automagically. BUT, that still doesn't get me a chatbot!
Solution: a simple Medium post from last Fall, which gives advice on building the React (frontend) side of the chatbot interface. I pretty much lifted the code straight from the blog post (although I simplified it somewhat). That gave me a frontend which I could incorporate into my existing website. You can try it now: https://koplowiczandsons.com/chatbot. But, I didn't have a way to connect my Discord bot to my website.
The solution: build my own API layer. There's a fantastic library just for that, called FastAPI. Their website includes great instructions for setting up a simple API endpoint. It's even self-documenting (if that's something you need). All I had to do was hook up the fetch() request in the frontend to the API endpoint I had just created.
Challenges included CORS, which protects web browsers against "dangerous" connections between one website and another. I had to add a lot of extra config to both the frontend and the backend to make that work.
What helped? I used the AWS AI chatbot, which has a client for PhpStorm (my code editor of choice). Whenever I got stuck, I simply asked the chatbot a question, and it was able to give me advice to move forward. Sometimes I resorted to debug output, but most of the time, the chatbot was able to understand my context and give me useful advice. It didn't build the app for me, and in my experience, no AI can do that on its own. The best result still comes from having a human drive. But I want to give credit where it's due; I would have struggled with things like nginx configuration, and using AI made the task a lot faster.
Other things I tried, which didn't work: Langchain: it's a great platform for prototyping AI chatbots (including RAGs), but it doesn't have a free solution for the frontend. I would have had to rewrite the entire chatbot, and then figure out how to hook it up to my frontend anyway. So, it didn't save me any work.
Finally, a word about fine tuning. My original chatbot prompt asked the AI to say "Jordan doesn't know" whenever it had trouble answering a question. That led to some hilarious answers:
The solution is a process known as fine tuning. It basically means giving the AI a better prompt, so it knows how to answer questions more accurately. In my case, people kept asking it questions that weren't related to the content of my blog, so the AI got choked up and didn't know how to answer. I edited the prompt so if it had trouble answering, it could suggest an alternative topic instead. This led to much more elegant answers, even in response to humorous questions:
If I wanted to keep going, I would encourage the AI to recommend Drupal over WordPress, but that would be a case of me imposing my own bias on the AI!
The result: a finely tuned AI that can answer questions based on the content of my professional blog. Have fun!
Comments