Programmatic document extraction

In this section you will learn how to process your first document with the natif.ai API using API key authentication.

Hint

You will need an API key secret token to successfully execute the code examples from this section. See: Create an API key

The final script at a glance

PythonShell

processing.py

import requests
import urllib.parse

API_KEY_SECRET = ""  # TODO: Insert or load your API-key secret here
WORKFLOW = "invoice_extraction"
FILE_PATH = "./example.pdf"
PROCESSING_URI = "https://api.natif.ai/processing/{workflow}?{params}"


def process_via_natif_api(file_path, workflow, include="extractions"):
    # Encapsulates HTTP calls to the natif.ai processing API
    headers = {"Accept": "application/json", "Authorization": "ApiKey " + API_KEY_SECRET}
    params = {"include": include}
    url = PROCESSING_URI.format(workflow=workflow, params=urllib.parse.urlencode(params))
    with open(file_path, 'rb') as file:
        response = requests.post(url, headers=headers, files={"file": file})
        if not response.ok:
            raise Exception(response.text)
        return response.json()


if __name__ == '__main__':
    # Submit a file for invoice extraction
    result = process_via_natif_api(FILE_PATH, WORKFLOW)

    # Print one of the extracted values alongside its extraction confidence
    print("Vendor name: ", result.get("extractions").get("vendor", {}).get("name", {}).get("value", None), "\n")
    print("Confidence:  ", result.get("extractions").get("vendor", {}).get("name", {}).get("confidence", None), "\n")

curl -X 'POST' \
  'https://api.natif.ai/processing/invoice_extraction?include=ocr&include=extractions&wait_for=120' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'Authorization: ApiKey <API_KEY_SECRET>' \
  -F 'file=@FILE_PATH;type=image/png' \
  -F 'parameters={"language": "de"}'

This code is self-contained and can be run as is, given that you correctly specify API_KEY_SECRET and FILE_PATH before you run it.

Step by step

Configure API key authentication

Each request to the natif.ai API needs to provide an authentication header. For applications API key authentication can be provided via an Authorization: ApiKey header.

PythonShell

process.py

import requests
import urllib.parse

API_KEY_SECRET = ""  # TODO: Insert or load your API-key secret here
WORKFLOW = "invoice_extraction"
FILE_PATH = "./example.pdf"
PROCESSING_URI = "https://api.natif.ai/processing/{workflow}?{params}"


def process_via_natif_api(file_path, workflow, include="extractions"):
    # Encapsulates HTTP calls to the natif.ai processing API
    headers = {"Accept": "application/json", "Authorization": "ApiKey " + API_KEY_SECRET}
    params = {"include": include}
    url = PROCESSING_URI.format(workflow=workflow, params=urllib.parse.urlencode(params))
    with open(file_path, 'rb') as file:
        response = requests.post(url, headers=headers, files={"file": file})
        if not response.ok:
            raise Exception(response.text)
        return response.json()


if __name__ == '__main__':
    # Submit a file for invoice extraction
    result = process_via_natif_api(FILE_PATH, WORKFLOW)

    # Print one of the extracted values alongside its extraction confidence
    print("Vendor name: ", result.get("extractions").get("vendor", {}).get("name", {}).get("value", None), "\n")
    print("Confidence:  ", result.get("extractions").get("vendor", {}).get("name", {}).get("confidence", None), "\n")

curl -X 'POST' \
  'https://api.natif.ai/processing/invoice_extraction?include=ocr&include=extractions&wait_for=120' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'Authorization: ApiKey <API_KEY_SECRET>' \
  -F 'file=@FILE_PATH;type=image/png' \
  -F 'parameters={"language": "de"}'

Specify the desired workflow identifier

The unique identifier for each available workflow is described in their respective documentation sections.

This code example uses invoice_extraction. The identifier for the desired workflow is passed as an endpoint path parameter, i.e. as a part of the endpoint URL.

PythonShell

process.py

import requests
import urllib.parse

API_KEY_SECRET = ""  # TODO: Insert or load your API-key secret here
WORKFLOW = "invoice_extraction"
FILE_PATH = "./example.pdf"
PROCESSING_URI = "https://api.natif.ai/processing/{workflow}?{params}"


def process_via_natif_api(file_path, workflow, include="extractions"):
    # Encapsulates HTTP calls to the natif.ai processing API
    headers = {"Accept": "application/json", "Authorization": "ApiKey " + API_KEY_SECRET}
    params = {"include": include}
    url = PROCESSING_URI.format(workflow=workflow, params=urllib.parse.urlencode(params))
    with open(file_path, 'rb') as file:
        response = requests.post(url, headers=headers, files={"file": file})
        if not response.ok:
            raise Exception(response.text)
        return response.json()


if __name__ == '__main__':
    # Submit a file for invoice extraction
    result = process_via_natif_api(FILE_PATH, WORKFLOW)

    # Print one of the extracted values alongside its extraction confidence
    print("Vendor name: ", result.get("extractions").get("vendor", {}).get("name", {}).get("value", None), "\n")
    print("Confidence:  ", result.get("extractions").get("vendor", {}).get("name", {}).get("confidence", None), "\n")

curl -X 'POST' \
  'https://api.natif.ai/processing/invoice_extraction?include=ocr&include=extractions&wait_for=120' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'Authorization: ApiKey <API_KEY_SECRET>' \
  -F 'file=@FILE_PATH;type=image/png' \
  -F 'parameters={"language": "de"}'

Upload the actual file

The actual file that should be processed is transmitted to the API endpoint via a multipart/form file upload. Most requests libraries know how to transmit files on their own.

PythonShell

process.py

import requests
import urllib.parse

API_KEY_SECRET = ""  # TODO: Insert or load your API-key secret here
WORKFLOW = "invoice_extraction"
FILE_PATH = "./example.pdf"
PROCESSING_URI = "https://api.natif.ai/processing/{workflow}?{params}"


def process_via_natif_api(file_path, workflow, include="extractions"):
    # Encapsulates HTTP calls to the natif.ai processing API
    headers = {"Accept": "application/json", "Authorization": "ApiKey " + API_KEY_SECRET}
    params = {"include": include}
    url = PROCESSING_URI.format(workflow=workflow, params=urllib.parse.urlencode(params))
    with open(file_path, 'rb') as file:
        response = requests.post(url, headers=headers, files={"file": file})
        if not response.ok:
            raise Exception(response.text)
        return response.json()


if __name__ == '__main__':
    # Submit a file for invoice extraction
    result = process_via_natif_api(FILE_PATH, WORKFLOW)

    # Print one of the extracted values alongside its extraction confidence
    print("Vendor name: ", result.get("extractions").get("vendor", {}).get("name", {}).get("value", None), "\n")
    print("Confidence:  ", result.get("extractions").get("vendor", {}).get("name", {}).get("confidence", None), "\n")

curl -X 'POST' \
  'https://api.natif.ai/processing/invoice_extraction?include=ocr&include=extractions&wait_for=120' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'Authorization: ApiKey <API_KEY_SECRET>' \
  -F 'file=@FILE_PATH;type=image/png' \
  -F 'parameters={"language": "de"}'

Submit the HTTP request

… and double check, if the request has succeeded. On the other hand, what should go wrong?

If the request succeeded, extract the JSON response body.

Hint

This code uses the processing API in a "synchronous way" via long-polling. Without further parametrization the server will wait for the document extraction to finish and upon success immediately return the result. For large documents or complex extractions this can take a few seconds. If instead you prefer to push documents for processing without waiting and fetch the results later, you may want to read into this section.

PythonShell

process.py

import requests
import urllib.parse

API_KEY_SECRET = ""  # TODO: Insert or load your API-key secret here
WORKFLOW = "invoice_extraction"
FILE_PATH = "./example.pdf"
PROCESSING_URI = "https://api.natif.ai/processing/{workflow}?{params}"


def process_via_natif_api(file_path, workflow, include="extractions"):
    # Encapsulates HTTP calls to the natif.ai processing API
    headers = {"Accept": "application/json", "Authorization": "ApiKey " + API_KEY_SECRET}
    params = {"include": include}
    url = PROCESSING_URI.format(workflow=workflow, params=urllib.parse.urlencode(params))
    with open(file_path, 'rb') as file:
        response = requests.post(url, headers=headers, files={"file": file})
        if not response.ok:
            raise Exception(response.text)
        return response.json()


if __name__ == '__main__':
    # Submit a file for invoice extraction
    result = process_via_natif_api(FILE_PATH, WORKFLOW)

    # Print one of the extracted values alongside its extraction confidence
    print("Vendor name: ", result.get("extractions").get("vendor", {}).get("name", {}).get("value", None), "\n")
    print("Confidence:  ", result.get("extractions").get("vendor", {}).get("name", {}).get("confidence", None), "\n")

curl -X 'POST' \
  'https://api.natif.ai/processing/invoice_extraction?include=ocr&include=extractions&wait_for=120' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'Authorization: ApiKey <API_KEY_SECRET>' \
  -F 'file=@FILE_PATH;type=image/png' \
  -F 'parameters={"language": "de"}'

Print parts of the extracted information

The API response contains a JSON object with the processing results. A possible result of invoice_extraction could look like this:

Api response JSON

{
  "processing_id": "f5be58b6-712f-4a42-ab43-03c3a448e498",
  "ocr": {
    "pages": [
      {
        "width": 1818,
        "height": 2572,
        "bboxes": [
          { "id": 1, "x1": 210, "y1": 634, "x2": 312, "y2": 659, "text": "66666", "text_entropy": 0 },
          { "id": 2, "x1": 211, "y1": 517, "x2": 282, "y2": 542, "text": "Herr", "text_entropy": 0.1378173828125 },
          { "id": 3, "x1": 211, "y1": 556, "x2": 289, "y2": 581, "text": "John", "text_entropy": 0.038055419921875 }
          ...
        ],
        "fulltext": "natif.ai GmbH - Campus A1 1 - ..."
      }
    ]
  },
  "extractions": {
    "schema_version": 2,
    "document_type": "invoice",
    "customer": {
      "name": {
        "validation_problem": false,
        "note": "",
        "confidence": 0.9968333333333333,
        "bbox_refs": [...],
        "value": "Herr John Doe"
      }
    }
  }
}

The exact structure and contents of the extractions section within this json object depends on the workflow.

In the code example we are interested in the name of the vendor that sent us the processed invoice and print it to the console alongside the AI extraction confidence.

Python

process.py

import requests
import urllib.parse

API_KEY_SECRET = ""  # TODO: Insert or load your API-key secret here
WORKFLOW = "invoice_extraction"
FILE_PATH = "./example.pdf"
PROCESSING_URI = "https://api.natif.ai/processing/{workflow}?{params}"


def process_via_natif_api(file_path, workflow, include="extractions"):
    # Encapsulates HTTP calls to the natif.ai processing API
    headers = {"Accept": "application/json", "Authorization": "ApiKey " + API_KEY_SECRET}
    params = {"include": include}
    url = PROCESSING_URI.format(workflow=workflow, params=urllib.parse.urlencode(params))
    with open(file_path, 'rb') as file:
        response = requests.post(url, headers=headers, files={"file": file})
        if not response.ok:
            raise Exception(response.text)
        return response.json()


if __name__ == '__main__':
    # Submit a file for invoice extraction
    result = process_via_natif_api(FILE_PATH, WORKFLOW)

    # Print one of the extracted values alongside its extraction confidence
    print("Vendor name: ", result.get("extractions").get("vendor", {}).get("name", {}).get("value", None), "\n")
    print("Confidence:  ", result.get("extractions").get("vendor", {}).get("name", {}).get("confidence", None), "\n")

Ready to integrate

If you want to learn further details about the /processing endpoint, like:

• Advanced configuration options
• Advanced response-code handling
• Details about the response body format
• Specifics about the available workflows

… you should check out the detailed endpoint documentation.