Hello, I am following the GraphAcademy, ' Build a Neo4j-backed Chatbot using Python. When running the movies database, I can return source documents and they seem to get presented to the LLM. When I create my own database with my own text chunks in there, the source_documents is and the response from the LLM is like it hasn't seen those source documents. Here is my code:
neo4jvector = Neo4jVector.from_existing_index(
embeddings,
url=st.secrets["NEO4J_URI"],
username=st.secrets["NEO4J_USERNAME"],
password=st.secrets["NEO4J_PASSWORD"],
index_name="BestPracticeContent",
node_label="TextChunk",
text_node_property=["text"],
embedding_node_property="embedding",
)
retriever = neo4jvector.as_retriever(k=15)
kg_qa = RetrievalQA.from_chain_type(
llm = llm,
chain_type="stuff", # using 'prompt stuffing'
retriever=retriever,
verbose=True,
return_source_documents = True,
)
tag::tools
tools = [
#Tool.from_function(
# name="General Chat",
# description="For general chat not covered by other tools",
# func=llm.invoke,
# return_direct=True
#),
Tool.from_function(
name="Vector Search Index",
description="Provides information about Best Practice using Vector Search",
func = kg_qa,
return_direct=False
),
Tool.from_function(
name="Cypher QA",
description="Provide information about Best Practice questions using Cypher",
func = run_cypher,
return_direct=True
),
]
memory = ConversationBufferWindowMemory(
memory_key='chat_history',
k=5,
return_messages=True,
)
agent_prompt = PromptTemplate.from_template("""
You are a medical expert providing information to medical professionals based solely on the specific documents or outputs from tools provided. It is crucial to follow these instructions carefully:
- ONLY use information from the provided tools or documents in your responses.
- DO NOT rely on or include your pre-trained knowledge for medical advice.
- be helpful and polite
- If the information is not available in the documents or tool outputs, clearly state that the required information is not available.
- Example of Correct Use: "Based on the information provided by documents from BMJ Best Practice, the best practice for treating condition X is..."
- Example of Incorrect Use: "Generally, condition X is treated with..."
- DO NOT state the necessity to consult a doctor/nurse/healthcare provider/medical professional
Remember, the accuracy and relevance of your responses depend entirely on the use of the provided information. Any deviation from these instructions is unacceptable and compromises the quality of the advice given to healthcare professionals. These are healthcare professionals, so don't ask them to contact any healthcare professional.
TOOLS:
You have access to the following tools:
{tools}
If asked a medical question, always use a tool, please use the following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here]
Begin!
Previous conversation history:
{chat_history}
New input: {input}
{agent_scratchpad}
""")
agent = create_react_agent(llm, tools, agent_prompt)
agent_executor = AgentExecutor(
agent=agent,
tools=tools,
memory=memory,
verbose=True,
handle_parsing_errors=True
)
def generate_response(prompt):
response = agent_executor.invoke({"input": prompt})
return response['output']
And this is the response I get:
Entering new AgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: Vector Search Index
Action Input: treatment options for COPD
Entering new RetrievalQA chain...
Finished chain.
{'query': 'treatment options for COPD', 'result': "Treatment options for Chronic Obstructive Pulmonary Disease (COPD) often involve a combination of medication, lifestyle changes, and potentially surgery. Medications can include bronchodilators, inhaled steroids, combination inhalers, and oral steroids. Pulmonary rehabilitation programs can help manage the disease through exercise, disease management training, and nutritional advice. Oxygen therapy may also be necessary in severe cases. In extreme cases, surgeries like a lung transplant or lung volume reduction surgery might be considered. It's important to consult with a healthcare provider for personalized treatment plans.", 'source_documents': []}Do I need to use a tool? No
Final Answer: Based on the information obtained from the Vector Search Index, treatment options for Chronic Obstructive Pulmonary Disease (COPD) often involve a combination of medication, lifestyle changes, and potentially surgery. Medications can include bronchodilators, inhaled steroids, combination inhalers, and oral steroids. Pulmonary rehabilitation programs can help manage the disease through exercise, disease
management training, and nutritional advice. Oxygen therapy may also be necessary in severe cases. In extreme cases, surgeries like a lung transplant or lung volume reduction surgery might be considered.
Finished chain.
I know I haven't refined a 'retrieval_query' in the neo4jvector, but I thought at least something should be returned?