Introducing Unstructured Platform API for Programmatic Data Transformation

Following the launch of the Unstructured Platform, an enterprise-ready data transformation solution, we’re excited to introduce the Unstructured Platform API – a powerful way to interact with Unstructured Platform programmatically.

No code UI is a great way to build data transformation workflows, but certain scenarios could benefit from a programmatic approach. Here are some of the use cases that Unstructured Platform API unlocks: 

• Seamlessly integrate data transformation into your CI/CD pipelines

• Ensure that your data transformation logic is consistent across development, staging, and production environments

• Document and version control your data transformation workflows

What is the Platform API?

The Platform API is a REST-enabled interface that allows you to leverage Unstructured's data transformation capabilities through a headless, push-style experience. It's designed to give developers a straightforward method to ingest, process, and upload data across various sources and destinations.

At its core, the Platform API revolves around four primary concepts:

• Source Connectors: These are your data entry points. They handle ingesting files or data from various data source locations into the Unstructured Platform.

• Destination Connectors: Once data is processed, these connectors seamlessly transfer the RAG-ready data to your desired destination.

• Workflows: Define how your data will be processed. The current API release supports the three pre-configured (“Build it with Me”) workflows: Basic, Advanced, and Platinum – with the platinum workflow leveraging Claude 3.5 Sonnet by default for advanced transformations. Custom workflows will be supported in future API updates.

• Jobs: These are the execution units that run your specified workflow at a specific time.

With the Platform API you can create source and destination connectors, set workflows from pre-configured options (basic/advanced/platinum), trigger jobs, and get the outputs into the destination.  

Getting Started

Your existing Unstructured Serverless API keys will work with the Platform API, however, the API Endpoint is different: https://platform.unstructuredapp.io/api/v1.

If you’re new to the Unstructured Platform, check out the documentation to learn how you can obtain your API key. 

Integrating the Platform API is straightforward, here are some examples:

Creating a source connector

curl --request 'POST' --location \
"$UNSTRUCTURED_API_URL/sources" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY" \
--header 'content-type: application/json' \
--data \
'{
    # Specify the settings for the connector here.
}'

Creating a destination connector

curl --request 'POST' --location \
"$UNSTRUCTURED_API_URL/destinations" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY" \
--header 'content-type: application/json' \
--data \
'{
    # Specify the settings for the connector here.
}'

Creating a workflow

curl --request 'POST' --location \
"$UNSTRUCTURED_API_URL/workflows" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY" \
--header 'content-type: application/json' \
--data \
'{
    # Specify the settings for the workflow here.
}'

Executing a workflow

curl --request 'POST' --location \
"$UNSTRUCTURED_API_URL/workflows/<workflow-id>/run" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY"

You can find the full API documentation here. For more examples, refer to the Platform API section of the Platform documentation. 

FAQ

Q: Does this mean the Unstructured Serverless API is going away?

A: No, this is an additional offering. All existing Serverless API functionality remains intact.

Q: What are the current limitations?

The current version doesn't support custom workflows, we are going to add this functionality in Q1 2025. We are taking an iterative approach and will be expanding the Platform API’s functionality gradually.

Q: Can I use my Serverless API key with the Platform API?

Yes, you can! However, make sure to use the correct API endpoint - https://platform.unstructuredapp.io/api/v1