Redis Support in Unstructured Platform: Supercharging Your RAG Pipeline

Retrieval-Augmented Generation (RAG) applications depend on efficient data ingestion, transformation, and storage to deliver relevant responses quickly. However, managing the preprocessing pipeline and integrating a high-performance vector database can be complex and resource-intensive.

To simplify this process, Unstructured Platform now offers seamless Redis Cloud integration. By combining Unstructured’s advanced data preprocessing capabilities with Redis Cloud’s in-memory performance, data teams can build RAG applications that are both scalable and lightning-fast. This integration enables you to process unstructured data and store it efficiently in Redis Cloud, unlocking high-speed vector search and retrieval capabilities.

Why Redis?

Redis is a great choice for RAG applications because of its speed, scalability, and ability to be used for many real-time use cases. It provides several key features that enhance vector search and retrieval:

• Vector Indexing: Stores and organizes vector embeddings efficiently, enabling rapid similarity searches.

• Semantic Caching: Improves response times and reduces processing costs by storing frequently accessed results, such as commonly asked questions in a knowledge retrieval system.

• LLM Context Management: Maintains conversation history for AI-driven interactions, ensuring continuity and better contextual understanding in chatbot or virtual assistant applications.

• High Performance: With its in-memory architecture, Redis delivers low-latency, high-throughput operations, making it an excellent choice for large-scale AI and RAG workloads.

• Broad integrations: Redis works with the top GenAI tools and frameworks including Unstructured so you can focus on building apps instead of maintaining infrastructure.

Unstructured Platform + Redis

With the Unstructured Platform’s Redis Cloud integration, data teams can:

• Process unstructured data from 60+ file formats using Unstructured’s robust parsing tools

• Convert documents into structured, AI-ready chunks with intelligent partitioning

• Store document content and metadata directly in Redis Cloud for fast retrieval

• Utilize Redis Cloud’s vector search capabilities for efficient similarity matching

• Scale seamlessly with high-throughput, batch-processing support

By leveraging Redis Cloud’s real-time processing capabilities alongside Unstructured’s powerful data transformation tools, engineers can build high-performance RAG applications that deliver faster, more accurate responses. Whether handling live data streams or extensive document repositories, this integration optimizes retrieval speed and efficiency, reducing complexity in AI-driven workflows.

Setting Up the Integration

Here's how to configure the Redis Cloud integration in Unstructured Platform:

Using the Platform UI

1. Navigate to the Connectors section in the sidebar and click Destinations

2. Click "New" and provide a unique name for your connector

3. Choose "Redis" as the Provider and fill out the following configuration details:

• Connection URI, or hostname/port/username/password

• Database index (typically 0-15)

• SSL settings (if applicable)

• Batch size for uploads

Screenshots of a sample Redis Cloud destination connector configuration:

Using the Platform API

For those who prefer programmatic setup, here's how to create a Redis Cloud destination connector using our API:

curl --request 'POST' \
--location "$UNSTRUCTURED_API_URL/destinations" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY" \
--header 'content-type: application/json' \
--data '{
    "name": "<name>",
    "type": "redis",
    "config": {
        "database": <database>,
        "ssl": <true|false>,
        "batch_size": <batch-size>,
        # For URI authentication:
        "uri": "<uri>"
        # For password authentication:
        "host": "<host>",
        "port": <port>,
        "username": "<username>",
        "password": "<password>

Get started!

If you're already an Unstructured Platform user, the Redis Cloud integration is available in your dashboard today! New users can sign up for the Unstructured Platform here with a 14-day free trial to start.

Expert access

Need a tailored setup for your specific use case? Our engineering team is available to help optimize your implementation. Book a consultation session to discuss your requirements here.