Training a Custom LLM with Your Data Using LLaMA and LangChain

 In this blog, we’ll walk through a step-by-step guide to building an AI application that can intelligently answer questions based on your website’s content using LangChain, Ollama, and ChromaDB. This approach leverages Retrieval-Augmented Generation (RAG), enabling you to use a pre-trained language model while grounding its responses in your own data — without needing to fine-tune the model.

What We'll Use

• LangChain – for chaining together data loading, embedding, retrieval, and prompt logic

• Ollama – to run open-source LLMs (like LLaMA or Mistral) locally

• ChromaDB – as an efficient vector store

• WebsiteLoader – to extract data directly from your website

• RecursiveTextSplitter – for clean and structured chunking of long web content

Step 1: Install Dependencies

pip install langchain chromadb beautifulsoup4 unstructured requests tiktoken

Step 2: Load Website Content

from langchain_community.document_loaders import WebBaseLoader

urls = [

    "https://your-website.com/page1",

    "https://your-website.com/page2"

]

loader = WebBaseLoader(urls)

documents = loader.load()

Step 3: Split the Text with RecursiveTextSplitter

from langchain.text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(

    chunk_size=500,

    chunk_overlap=50

)

docs = text_splitter.split_documents(documents)

Step 4: Generate Embeddings using Ollama

from langchain.embeddings import OllamaEmbeddings

embedding = OllamaEmbeddings(model="nomic-embed-text")

Step 5: Store Embeddings in ChromaDB

from langchain.vectorstores import Chroma

vectorstore = Chroma.from_documents(

    documents=docs,

    embedding=embedding,

    persist_directory="./chroma_store"

)

vectorstore.persist()

Step 6: Define Your Prompt and Invoke the Chain

from langchain.chains import RetrievalQA

from langchain.llms import Ollama

llm = Ollama(model="mistral")  # or llama3

qa_chain = RetrievalQA.from_chain_type(

    llm=llm,

    retriever=vectorstore.as_retriever()

)

query = "What services do we offer on our website?"

response = qa_chain.run(query)

print(response)

 Conclusion

This pipeline enables you to turn your website into a custom AI knowledge source without fine-tuning any models. With just a few tools—LangChain, Ollama, and ChromaDB—you can create intelligent assistants that understand and reason over your own content.