Ever since chatGPT was released last year, I’ve been fascinated by generative AI and wanted to explore building my own chatbot using available API. As a complete beginner without any programming experience, this was an ambitious goal. However, I was determined to push myself to gain hands-on experience working with AI systems. My motivation was to go beyond just using these tools, and start developing a deeper understanding of how they are built. I decided to leverage my existing content by creating a chatbot that could answer questions about the posts on my blog. Building this chatbot from scratch has been an invaluable learning experience, although it still has a long way to go. In this post, I’ll share some of the key lessons I’ve learned so far on this journey as a coding novice.
Table of Contents
- So what were my challenges?
- 1. ChatGPT 4 can fulfill most of the basic/intermediate coding tasks
- 2. Take fundamental coding/IT lessons/courses
- 3. Complete these FREE, short courses on Building Systems with the ChatGPT API
- 4. Read the actual OpenAI documentation by yourself
- 5. ChatGPT gets lost easily when the conversation is too long or when you start/stop after a few hours
- 6. GPT 4 API is expensive to use so try GPT 3.5 as much as possible
- 7. OpenAI Cookbook is your friend
- 8. Context window length is tricky to work with for beginner
- 9. Version control, version control and version control
- Share this with a friend
- Conclusion
- One week after
So what were my challenges?
- For starters, I didn’t know programming. I didn’t know much (if any) HTML (beyond the use of <h1> or <h2>), let alone Python or any other programming languages.
- Of course, I didn’t know how to deploy an application on the cloud, never heard of Docker, Cloud Run, etc… before.
- I couldn’t even migrate this website from Bluehost to GCP because I didn’t know about MySQL, virtual instance, etc…
- I didn’t know SSH, public key, private key or anything like that T.T
- I also didn’t have any background in machine learning or advanced math.
- And I am not a spring chicken 😐 I am 40 years old, with a full-time job.
But after about 5 months, here is version 0.1 of the chatbot that I built, using OpenAI GPT API and a lot of ChatGPT 4’s help. I am not kidding when I say version 0.1 because it is still far (very far) from what I expected but well it is “working” (sort of). What it can do:
- If you ask the chatbot a question about previous topics that I wrote before, the chatbot can synthesize the content across multiple articles and provide you a response.
- This point is important to me because previously I wrote about implementing off-the-shelf solutions but the common issue is that those chatbots do not synthesize the content across posts. The chatbot “feels” like a search function more than a chatbot.
- It is designed to answer questions about my blog posts only so if you ask it to answer other topics, it is likely that it will say no.
- Its knowledge includes all the posts that I published over the past 17 years, up until the end of Sep 2023. That is about 500 posts.
There are so many things that the chatbot can NOT do Yet, like:
- It can’t recall accurately a specific number in a post that I wrote.
- It can’t answer questions about published dates accurately.
- I am sure it has a lot of security issues T.T (sad, sad, sad face)
- and many more
Update Feb 2024:
Version 2.1 of the chatbot is available and you can check out the post about it “Chatbot v2.10 Unveiled: Elevating User Experience with Enhanced Speed, Scalability, and Simplicity.“
So what have I learned through the first phase of this journey?
1. ChatGPT 4 can fulfill most of the basic/intermediate coding tasks
if you know what you are looking for and be specific with your prompts/context-sharing.
The key here is knowing what to ask the machine to do. It can do “almost all” of the coding tasks given as long as the tasks are specific enough. I say “almost all” on purpose because it repeatedly fails to follow some examples given.
The code chatGPT 4 generates is often modular enough that it can be reused later. The machine often includes specific comments throughout the code too.
2. Take fundamental coding/IT lessons/courses
If you were in my shoes, you wouldn’t know to ask what you didn’t know. A few ways to overcome this:
- Explain the overall objective of the project to chatGPT and ask it to give you an overview of all the steps. This will roughly give you an idea of areas that you will need to learn more about
- If you just blindly follow what chatGPT tells you to do, it may not make sense. (I tried this :P)
- These fundamental courses are super helpful to me. They give me broad (but not deep) knowledge so that I know how different puzzles are supposed to fit together.
3. Complete these FREE, short courses on Building Systems with the ChatGPT API
Two courses about chatGPT API will save you a lot of time:
While these courses are for beginners, when I first came across them a few months ago, I didn’t have enough fundamental knowledge to even follow what was said. Hence I stopped watching them after about 10 mins. Then after I tried to work with the OpenAI API and different promts for a while, got stuck and then I remembered these courses. Basically they gave me all the answers I needed to build this version 0.1
The prompt engineering course is particularly helpful because it shows me how to instruct chatGPT to output in a certain way. For example, I would not think of this type of system message without watching the course:
“Respond in a friendly and helpful tone, with concise answers if possible. \
Use HTML-compatible bullet point format and line breaks (<br>) for long answers where necessary. \
Make sure to ask the user relevant follow-up questions.”
The HTML-compatible bullet point format and line breaks help to make it a lot easier for users of the chat bot when the answers are long.
4. Read the actual OpenAI documentation by yourself
I am lazy (:D) so initially I didn’t bother to read the documentation from OpenAI. I simply copied and pasted the documentation to chatGPT and then asked it to use as reference sources and guided me with actual codes. It didn’t work out very well and I wasted quite a lot of time before I decided to read the documentation myself and then provided more targeted asks to chatGPT
5. ChatGPT gets lost easily when the conversation is too long or when you start/stop after a few hours
I guess I should not be surprised by this because similar to human, chatGPT gets lost easily when the conversation is getting too long. What do I mean by this?
- It starts to recommend codes that don’t make any sense
- Or couldn’t recall another piece of codes/advices that it recently gives me
Also when you stop working with it for a few hours (or overnight) and then come back, even after asking chatGPT to read the entire conversation history and confirm that it has done so, similar to human, I have to provide a lot more recent context and code base to it again. Otherwise, it seems to completely “forget” many details that it told me before.
6. GPT 4 API is expensive to use so try GPT 3.5 as much as possible
For example, I wasted $20 using GPT 4 API for summarization task that GPT 3.5 can do well. Also make sure that you test the code on a few files first before you “unleash” it on hundreds of files because well, the GPT API cost can rack up fast.
7. OpenAI Cookbook is your friend
Again, I made the mistake of not bothering (or more precisely not knowing) to check OpenAI sample cookbook before hand. Over there, you can find:
- How to count token using tiktoken: this is important because if you just ask chatGPT to do this, it will rely on old methodologies for GPT 3 and not the most up to date approach
- Question answering using embeddings-based search: I basically use the same approach to build version 0.1 of the chatbot.
8. Context window length is tricky to work with for beginner
Different GPT models have different max token limits and modify the code to make sure that you stay within the token limit is not easy for novices (like myself). I had to spend so much time on this part because I have many blog posts that are very very long.
Also for the chatbot, I want it to synthesize the content across multiple posts before giving an answer so I faced token limit often.
9. Version control, version control and version control
This is probably super intuitive for experts but for coding novice like myself, I have to learn the hard way. And I also have to learn the right git commit frequency that works for this project, since I am the sole contributor/coder.
Share this with a friend
If you enjoyed this article and found it valuable, I’d greatly appreciate it if you could share it with your friends or anyone else who might be interested in this topic. Simply send them the link to this post, or share it on your favorite social media platforms. Your support helps me reach more readers and continue providing valuable content.
Conclusion
In summary, this chatbot-building journey as a beginner has taught me the importance of learning the fundamentals, leveraging ChatGPT for targeted tasks, studying the API documentation, and using version control. While there is still significant room for improvement, I now have a basic prototype that can synthesize information across my blog posts to answer questions. Moving forward, my priorities are to expand the chatbot’s knowledge base to cover new content, improve its ability to retrieve precise details from posts, and optimize its responses to stay within token limits. I also plan to continue leveling up my skills in areas like prompt engineering and cloud deployment.
That’s it from me. I will continue to update on the lessons learnt and share when the new version is available.
Let me know what you think about the lesson and the chatbot after trying it out.
One week after
After 1 week sprint, I have completed additional updates to the chatbot. To read more about it, go here “Evolving My Chatbot: A One-Week Sprint to Smarter Interactions.”
Chandler