Introducing Llama Packs: Pre-packaged Module Center

Llama Packs - a community-driven pre-packaged module center that allows you to quickly launch your Large Language Model (LLM) applications. Importing these modules can be used for a wide range of use cases, from building a Streamlit application to setting up an advanced retrieval system based on Weaviate, to resume parsers that can perform structured data extraction. Equally important, you can customize these modules according to your preferences.

Overview

Llama Packs can be described in two ways:

• On one hand, they are pre-packaged modules that can be initialized with parameters and can be used out of the box to achieve a given use case (whether it's a complete RAG (Reusable, Additive, Composable) workflow, application templates, etc.). You can also import sub-modules (such as LLMs, query engines) for direct use.

• On the other hand, Llama Packs are also templates that you can view, modify, and use.

They can be downloaded with a single line of code using the llama_index Python library or the Command Line Interface (CLI):

Command Line Interface (CLI):

llamaindex-cli download-llamapack <pack_name> --download-dir <pack_directory>dir <pack_directory>

Python

from llama_index.llama_pack import download_llama_pack
# download and install dependencies
VoyageQueryEnginePack = download_llama_pack(
  "<pack_name>", "<pack_directory>"
)

Llama Packs can span different levels of abstraction - some are complete pre-packaged templates (complete Streamlit / Gradio applications), while others combine smaller modules (such as our SubQuestionQueryEngine with Weaviate).

Example Demo

The best way to showcase the features of Llama Packs is to demonstrate an example. We will walk through a simple Llama Pack that provides users with a RAG pipeline setup with Voyage AI embedding.

First, we download and initialize the Pack with a set of documents:

from llama_index.llama_pack import download_llama_pack
# download pack
VoyageQueryEnginePack = download_llama_pack("VoyageQueryEnginePack", "./voyage_pack")
# initialize pack (assume documents is defined)
voyage_pack = VoyageQueryEnginePack(documents)

Each Llama Pack implements a get_modules() function that allows you to inspect/use the modules.

modules = voyage_pack.get_modules()
display(modules)
# get LLM, vector index
llm = modules["llm"]
vector_index = modules["index"]

Llama Packs can be run out of the box. By calling run, we execute the RAG pipeline and get a response. In this setup, you don't need to worry about the internals.

# this will run the full pack
response = voyage_pack.run("What did the author do growing up?", similarity_top_k=2)
print(str(response))

The author spent his time outside of school mainly writing and programming. He wrote short stories and attempted to write programs on an IBM 1401. Later, he started programming on a TRS-80, creating simple games and a word processor. He also painted still lives while studying at the Accademia.

The second important thing is that you have full access to the code of Llama Packs. This allows you to customize Llama Packs, remove code, or use it as a reference to build your own applications. Let's take a look at the downloaded pack in voyage_pack/base.py and replace OpenAI LLM with Anthropic:

from llama_index.llms import Anthropic
...
class VoyageQueryEnginePack(BaseLlamaPack):
    def __init__(self, documents: List[Document]) -> None:
        llm = Anthropic()
        embed_model = VoyageEmbedding(
            model_name="voyage-01", voyage_api_key=os.environ["VOYAGE_API_KEY"]
        )
        service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model)
        self.llm = llm
        self.index = VectorStoreIndex.from_documents(
            documents, service_context=service_context
        )
    def get_modules(self) -> Dict[str, Any]:
        """Get modules."""
        return {"llm": self.llm, "index": self.index}
    def run(self, query_str: str, **kwargs: Any) -> Any:
        """Run the pipeline."""
        query_engine = self.index.as_query_engine(**kwargs)
        return query_engine.query(query_str)

You can directly import the module and run it again:

from voyage_pack.base import VoyageQueryEnginePack
voyage_pack = VoyageQueryEnginePack(documents)
response = voyage_pack.run("What did the author do during his time in RISD?")
print(str(response))