Introduction to AutoLLM.

Ibrahim Zaman
2 min readNov 2, 2023

--

As we all know there are many ways to utilize LLM technology to chat with our data, most popular approach is Langchain, low-code Flowise or Langflow etc. They are great but there are little complicated.

AutoLLM created by SafeVideo.ai, just made easy to build a chatbot on private data. Just add a .env file which contains OpenAI key, then write 5 lines of code and you’re done. we will look deep into it later.

Crazy part is we can also use local LLMs that are deployed by Ollama and 100+ other third-party LLMs support built in.

Ibrahim Zaman (@abrahimzaman360)

AutoLLM also support multiple vector database support by default LanceDB is used to store embeddings of our docs and easy way to inference through API using FastAPI.

Let’s create a simple chatbot using AutoLLM with Quick Start.

  1. Use Colab or create new conda/pipenv environment, use python 3.8 or above version.
  2. Install AutoLLM package using pip using following:
pip install autollm
  1. Open a folder in VS Code or any of editor of your choice.
  2. Create a folder in working directory like data and copy the data with which you want to chat.
  3. Create a python file like main.py and add following lines:# Import things from AutoLLM Package
    from autollm import AutoQueryEngine, read_files_as_documents

    # Import data:
    documents = read_files_as_documents(input_dir="./data")

    # Create Query Engine:
    query_engine = AutoQueryEngine.from_default(documents=documents)

    # Ask your questions and store response:
    response = query_engine.query(
    "Who are tom and jerry in this story?"
    )

    # Show output:
    print(response.response)

Let’s test Chatbot by running python file and here is output:

Tom and Jerry are one of most popular cartoon, Tom is a cat and Jerry is a crazy mouse.

Great output, let’s create inference to Chatbot using FastAPI:

# Import things from AutoLLM Package
from autollm import AutoQueryEngine, read_files_as_documents, AutoFastAPI
import uvicorn

# Import data:
documents = read_files_as_documents(input_dir="./data")

# Create Query Engine:
query_engine = AutoQueryEngine.from_defaults(documents=documents)

# Create API inference:
app = AutoFastAPI.from_query_engine(query_engine)

# Running the inference engine:
uvicorn.run(app, host="0.0.0.0", port=8000)

# Output:
# INFO: Started server process [12345]
# INFO: Waiting for application startup.
# INFO: Application startup complete.
# INFO: Uvicorn running on http://http://0.0.0.0:8000/

You can now connect your web app or test API using postman by sending request on localhost:8000 or 0.0.0.0:8000

That’s it (:

To learn more, here are some links to AutoLLM:

  1. Supported Vector Databases
  2. Supported LLMs
  3. Quick Start

If you need any assistance, you can ping me on any of links found in my Link in Bio, Ibrahim Zaman.

--

--