升級我而家嘅Chatbot

更新（2026年）： Sydney自呢次Weaviate實驗以嚟已經行咗好遠。試過FAISS、Weaviate同其他之後，我最終揀咗Supabase pgvector做production版本。Sydney而家run喺Claude上，有streaming responses同tool use。下面嘅hybrid search同query structuring ideas直接影響咗Sydney今日嘅運作方式。

問Sydney →

---

2024年5月嘅原文保留如下作為context。

好似我喺最近嘅文章提到，我做緊一個self-evaluating金融Chatbot。咁點解detour去update而家嘅chatbot Sydney？Well，有幾個原因：

• End to end test Weaviate vector store：
  • 我嘗試用Weaviate作為金融chatbot嘅main vector store，但遇到scaling/memory size嘅問題。具體嚟講，僅僅過去10年約20間公司嘅10K同10Q filings，weaviate collection嘅size已經2GB。所以用而家嘅approach，成個S&P 500公司意味住約50+ GB嘅collection。太大喇，run/maintain會cost好多錢。所以我嘗試test唔同嘅Product Quantization（PQ）parameters。
  • 試過好多其他vector stores之後，我傾向Weaviate因為佢能夠以快速度做hybrid search with metadata filters。
  • 當我做Weaviate嘅PQ嗰陣，我有個問題係點deploy Weaviate喺production environment，即係Weaviate Cloud定用AWS/Google Cloud？Deploy會唔會困難？要幾多錢？
  • 因為呢啲問題，我決定為而家嘅chatbot Sydney deploy Weaviate。基本上我會用Weaviate replace FAISS。
  • 而家嘅版本，我用緊Weaviate Cloud
• Implement query translation同query structuring，同時確保output係Json格式等佢哋可以被使用。
  • Query translation嘅目標係將input分解成一組specific嘅sub-problems或sub-questions，可以independently回答。
  • Query structuring：我好care generate唔止係適合hybrid search嘅search term/phrases，仲要有適合嘅metadata for filtering。
    • 呢個超重要因為我想金融chatbot正確咁filter year、industry等。
• 點樣做hybrid-search with multiple filters同return唔止content仲有metadata。

如你所見，我需要以上所有嘢work先至能create金融chatbot，咁點解唔喺chatbot Sydney度以更細嘅scale試先呢 :)

我好開心同你講你可以而家試Sydney。佢有以上所有功能。你可以問以下呢類問題，chatbot應該會返回相關答案加上specific blog posts嘅links。

• Chandler喺2020年寫咗乜關於Kevin Rudd？
• 話我知Chandler由2020年到而家所有寫過關於Ray Dalio嘅嘢
• Chandler喺2022年寫咗乜關於Health Savings Accounts？
• Chandler喺2015年做咗啲乜？

暫時就係咁。我要返去做金融chatbot嘞 :P

如果你試過Weaviate或者hybrid search with metadata filters，我好想聽你有咩work到。

祝好，

Chandler

2024年9月更新

Sydney而家係一個multi-talented agent，能夠：

• 回答關於而家S&P 500公司嘅問題，包括佢哋過去10年提交俾SEC嘅嘢。
• 提供我15年blog內容嘅insights。

喺呢度睇吓。

P.S：以下係我用嚟做query translation同query structuring嘅prompt類型嘅examples。

"You are a helpful assistant that generates multiple sub-questions related to an input question. "
             "The current year is 2024."
             "The goal is to break down the input into a set of specific sub-problems / sub-questions that can be answered in isolation. "
             "Each specific sub-question will be used to retrieve relevant content from a vector store, using similarity search with score. "
             "Phrase the wording of the questions appropriately for this purpose.\n"
             "This vector store includes all of the published blog posts from Chandler Nguyen's blog from 2007 to 2024.\n\n"
             "Original question: \{query\}\n\n"
             "Generate the minimum number of sub-questions necessary to answer the original question. "
             "Your response should be formatted as a JSON array of strings, where each string represents a sub-question. "
             "Do not include any additional words, characters, or explanations in the response.\n\n"
             "Example response:\n"
             '[\n'
             '  "sub-question 1",\n'
             '  "sub-question 2"\n'
             ']'

"""You are a helpful assistant that generates a structured query related to an input question.
The goal is to break down the input into a structured query that can be used to retrieve relevant content from a vector store, using similarity search with score.
This vector store includes all of the published blog posts from Chandler Nguyen's blog from 2007 to 2024.
Original question: \{query\}

You must generate a response in JSON format as described below without any additional words or characters:
[
    "content_search": Similarity search query used to apply to the content of the Chandler Nguyen published blog posts to find similar documents related to the sub-question(s). Ensure the content_search query is not too broad or too specific, and strikes a balance between relevance and completeness. \n
    "start_date": optional field, the start date to search for blog posts that are relevant to the sub-question(s) in YYYY-MM-DD format. If the sub-question(s) do not specify a time frame, leave this field blank or set it to the earliest possible date (e.g., 2007-01-01) to cover a broader range. \n
    "end_date": optional field, the end date to search for blog posts that are relevant to the sub-question(s) in YYYY-MM-DD format. If the sub-question(s) do not specify a time frame, leave this field blank or set it to the latest possible date (e.g., 2024-12-31) to cover a broader range. \n
]

If the sub-question(s) include multiple years or a specific time range, generate 1 response for each year or time range, enclosed in separate JSON objects within the outer array.

Example responses:

For an open-ended sub-question without a specific time frame:
Sub-questions: ["What are the key insights Chandler wrote about Health Savings Accounts (HSA)?"]
[
    "content_search": "Chandler Nguyen blog posts about Health Savings Accounts",
    "start_date": "2007-01-01",
    "end_date": "2024-12-31"
]

For a sub-question specifying a year:
Sub-questions: ["What blog posts did Chandler write in 2018?", "Which blog posts written by Chandler in 2018 mention Kevin Rudd?"]
[
    "content_search": "Chandler Nguyen blog posts in 2018",
    "start_date": "2018-01-01",
    "end_date": "2018-12-31"
],
[
    "content_search": "Kevin Rudd mentioned in Chandler Nguyen blog posts in 2018",
    "start_date": "2018-01-01",
    "end_date": "2018-12-31"
]

For a sub-question specifying a time range:
Sub-questions: ["What did Chandler write about Ray Dalio in 2020?", "What did Chandler write about Ray Dalio in 2021?", "What did Chandler write about Ray Dalio in 2022?"]
[
    "content_search": "Chandler Nguyen blog posts about Ray Dalio in 2020",
    "start_date": "2020-01-01",
    "end_date": "2020-12-31"
],
[
    "content_search": "Chandler Nguyen blog posts about Ray Dalio in 2021",
    "start_date": "2021-01-01",
    "end_date": "2021-12-31"
],
[
    "content_search": "Chandler Nguyen blog posts about Ray Dalio in 2022",
    "start_date": "2022-01-01",
    "end_date": "2022-12-31"
]
"""

P.P.S：我知道front end仲係好慢，所以我可能需要學多啲關於前端開發。