Update Feb 2024: I disabled the generative agent from Google because I developed my own chatbot. You can check out the post introducing it here.
Yup, you read it right. The generative AI agent is live and you can interact with it at the bottom right-hand corner of my website. This agent is trained using the content from my website, using Google Cloud Gen App Builder. Give it a try and let me know what you think.
Here are my initial impressions of the setup/integration process
- The entire process is fairly quick and intuitive.
- Verify ownership: I need to verify with Google that I own the website so that Google’s crawlers can crawl the publically available content. I like this step because it prevents abuse, in which people create agents using other people’s websites.
- Free money is welcome! Google gave me:
- $1000 free credit to try Gen App Builder.
- $600 free credit to try Dialogflow CX trial.
- They are both needed to build the chat app.
- The credit was automatically given without any ask from me so it is Much appreciated and a nice surprise! Thank you, Google.
- Time to go live is relatively short: my website content is not a lot so it takes less than 2 hours for the crawler to complete indexing my site
- Super easy to integrate to an existing website: literally just copy and paste the code. I don’t bother to change the look and feel of the agent much at this point because the default one is good enough for me.
ML settings
I went ahead and customized the Generative AI setting to make sure that it meets my needs.
Grounding confidence
I am switching back and forth between High confidence, Medium confidence, and Low confidence because I am unsure which one is best. Because I have written about the same topic over time, what I am after is that the chatbot can synthesize the content written and provide nuanced answers. I am not sure which setting is good for this purpose?
Agent Identity
I use what I have under custom instructions from chatGPT “thoughtful, creative, provide nuanced answers.”
My initial impressions about the agent’s answers
I tried to ask the agent some questions and the quality of its answers was mixed. I am not sure if the answer quality will improve over time.
For example, I asked the below and the answers were not great.
- Tell me about the latest 10 articles on this website?
- Answer: I’m sorry, I don’t have access to the last 10 articles on this website. How can I help you today?
- Chandler’s comment: the answer is very weird because the AI has access to all of the content.
- What does Chandler think about Kevin Rudd?
- Answer: While I don’t have a good frame of reference to agree or disagree strongly with what Kevin shared, I find his talk insightful.
- Chandler’s comment: this is somewhat an OK answer. However, the reference link provided is not representative.
- Does Chandler write about Ray Dalio often?
- Answer: Yes, chandler nguyen blog has written about Ray Dalio.
- Chandler’s comment: I expect the AI to say a lot more about what I have written about Ray Dalio, or stuff like I write about his sharing twice every quarter, etc…
However, when I asked more specific questions like:
- what is Chandler’s advice about high-yield saving accounts for expats?
- how about health insurance?
- How about treasury direct website?
The answers to these questions seem a lot better.
My impression after a few days
Now that I have asked the bot more questions, my overall impression is that the quality of its answers can be improved. The Gen AI chatbot doesn’t seem to “synthesize” the content on my site and then provide the answer but simply tries to find certain phrases from previous writing and provide the replies. This is my educated guess based on the chat that I have with the bot thus far.
I guess this approach is fine for industries that want to provide specific answers from a database and the content doesn’t change much through time or there is a “source of truth”. However, for a blog, I wish that the bot would actually synthesize the content across the entire site and then provide nuanced answers based on different articles, published through time. Perhaps I am asking too much? But well, that is my expectation.
Also, the bot doesn’t seem to keep up to date with the new content. For example, I changed how I wrote about myself a couple of months ago but when you ask “Who is Chandler Nguyen?”, the answer is still very much based on the previous content, with my previous job title.
Last but not least, I also exported all the published posts over the past 15 years and uploaded that as a data source successfully to the chat bot app. I am not sure if that helps. It would be great to know if it helps through the UI though.
That’s it from me.
Chandler