🌲 PageIndex Document Processing
PageIndex generates a hierarchical “table of contents” tree that preserves the original document’s logical flow and organizational structure. This LLM-optimized index enables precise navigation and is ready for reasoning-based RAG. See our cookbook for a practical example.
Currently accepts PDF files only (more formats coming soon).
Submit a Document
Index a PDF and get back a doc_id used by every other operation.
Parameters
| Name | Type | Required | Description | Default |
|---|---|---|---|---|
| file_path | string | yes | Local path to the PDF file. | - |
| mode | string | no | Processing mode. Local uses Flash indexing. Cloud modes are passed through (e.g. "mcp"). | None |
| metadata | dict | no | Your own JSON-serializable tags. Returned in get_tree / get_ocr responses and list_documents entries. | None |
| wait | boolean | no | Return only once the document is ready. Cloud: polls until "completed". Local: indexing is already synchronous. | False |
| folder_id | string | no | Cloud-only — folder (workspace) ID. | None |
| beta_headers | list | no | Cloud-only — beta feature headers. | None |
Example Request
result = client.submit_document("./2023-annual-report.pdf")
doc_id = result["doc_id"]Example Response
{
"doc_id": "pi-abc123def456",
"name": "2023-annual-report.pdf"
}name is the stored document name. A name already in use gains a numeric suffix (name_1 … name_99) and a UserWarning is emitted.
Local mode: Flash indexing
In local mode, submit_document indexes the document in that same call and stores it under storage_path. It defaults to Flash indexing: the structure is extracted from the PDF’s own layout (no LLM), then refined for retrieval by a deterministic merge and an LLM expansion pass. Node summaries, the expansion pass, and the document description use the index model. It takes seconds.
doc_id = client.submit_document("./report.pdf")["doc_id"] # flash indexingIndexing runs at roughly $0.001 per page with index_model="gpt-5.6-luna", and documents from 9 to 1,098 pages complete in about 13 seconds to 4.5 minutes. See the benchmark repo for full numbers.
Cloud mode: asynchronous processing
Cloud uploads process asynchronously. Either block with wait=True, or submit many documents and poll afterwards.
# Block until ready
doc_id = client.submit_document("./report.pdf", wait=True)["doc_id"]
# Or poll yourself
doc_id = client.submit_document("./report.pdf")["doc_id"]
if client.get_document(doc_id)["status"] == "completed":
print("Document processing completed")You can organize cloud documents into Folders for better workspace management.
Get the Document Structure
The structure view: titles, page indexes, and node summaries, with no node text — the cheapest way to show an agent or a user what is in a document.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| doc_id | string | yes | Document ID |
Example Request
structure = client.get_document_structure(doc_id)Example Response
[
{
"title": "Financial Stability",
"node_id": "0006",
"page_index": 21,
"prefix_summary": "The Federal Reserve maintains financial stability through comprehensive monitoring...",
"nodes": [
{
"title": "Monitoring Financial Vulnerabilities",
"node_id": "0007",
"page_index": 22,
"summary": "The Federal Reserve's monitoring focuses on identifying and assessing potential risks..."
},
{
"title": "Domestic and International Cooperation and Coordination",
"node_id": "0008",
"page_index": 28,
"summary": "In 2023, the Federal Reserve collaborated internationally with central banks..."
}
]
}
]Leaf nodes carry summary; nodes with children carry prefix_summary.
See more example documents and generated tree structures .
Get the Full Tree
Check processing status and get the tree, optionally with node summaries and node text.
Parameters
| Name | Type | Required | Description | Default |
|---|---|---|---|---|
| doc_id | string | yes | Document ID | - |
| node_summary | boolean | no | Include a summary for each node. | False |
| include_text | boolean | no | Include node text. Pass False for a structure-only view (saves tokens). | True |
Example Request
tree_result = client.get_tree(doc_id)
if tree_result.get("status") == "completed":
print("PageIndex Tree Structure:", tree_result.get("result"))Example Response (Processing)
{
"doc_id": "pi-abc123def456",
"status": "processing"
}Example Response (Completed)
{
"doc_id": "pi-abc123def456",
"status": "completed",
"retrieval_ready": true,
"result": [
{
"title": "Financial Stability",
"node_id": "0006",
"page_index": 21,
"text": "The Federal Reserve maintains financial stability through comprehensive monitoring and regulatory oversight...",
"nodes": [
{
"title": "Monitoring Financial Vulnerabilities",
"node_id": "0007",
"page_index": 22,
"text": "The Federal Reserve's monitoring focuses on identifying and assessing potential risks..."
}
]
}
]
}Local documents are always "completed" by the time submit_document returns.
Check retrieval readiness
if client.is_retrieval_ready(doc_id):
print("Ready to chat")API errors (including a missing document) are reported as False; transport errors propagate.
Get Page Content
Read the text of specific pages.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| doc_id | string | yes | Document ID |
| pages | string | yes | Page specifier — "5-7", "3,8", or "12" |
Example Request
pages = client.get_page_content(doc_id, "5-7")
for page in pages:
print(page["page_index"], page["markdown"][:200])The full extraction is also available through get_ocr(), which returns page-based, node-based, or concatenated markdown.
In local mode this is the text extracted from the PDF during indexing — no OCR model runs locally, so scanned or image-only PDFs have no local text. Use PageIndex Cloud for production OCR and image understanding.
Get Document Metadata
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| doc_id | string | yes | Document ID |
Returns
| Field | Type | Description |
|---|---|---|
| id | string | Document ID |
| name | string | Document filename |
| description | string | Document description (if available) |
| status | string | "queued", "processing", "completed", or "failed" |
| createdAt | string | Creation timestamp, UTC with no timezone marker |
| pageNum | int | Number of pages |
| folderId | string | Folder the document belongs to (always None in local mode) |
Example
meta = client.get_document("pi-abc123def456")
print(f"{meta['name']}: {meta['status']}, {meta['pageNum']} pages")To display createdAt in the user’s timezone:
from datetime import datetime, timezone
datetime.fromisoformat(meta["createdAt"]).replace(tzinfo=timezone.utc).astimezone()List Documents
Paginated list of indexed documents, newest first.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| limit | int | Maximum documents to return (1–100) | 50 |
| offset | int | Number of documents to skip | 0 |
| folder_id | string | Cloud-only — filter by folder | None |
Returns: an object with documents (array of document metadata), total, limit, and offset.
Example
result = client.list_documents(limit=10)
print(f"Total: {result['total']}")
for doc in result["documents"]:
print(f"- {doc['name']} ({doc['status']})")
# Next page
result = client.list_documents(limit=10, offset=10)Delete a Document
Permanently delete a document and all its associated data.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| doc_id | string | yes | Document ID |
Example Request
client.delete_document(doc_id)Example Response
{ "message": "Document deleted successfully." }Folders (Workspaces)
Folders (workspaces) let you organize your PageIndex documents into groups. You can create nested folder hierarchies, assign documents to folders during upload, and filter documents by folder.
This feature is currently available for Max plan users.
Folders are PageIndex Cloud only. create_folder(), list_folders(), and the folder_id parameters raise PageIndexAPIError in local mode. See Client Configuration.
Create a Folder
Create a new folder for the authenticated user.
Parameters:
| Name | Type | Required | Description | Default |
|---|---|---|---|---|
| name | string | yes | Folder name | - |
| description | string | no | Folder description | None |
| parent_folder_id | string | no | Parent folder ID for nesting | None |
Example
result = client.create_folder("Research Papers", description="2024 research collection")
print(result["folder"]["id"])With Parent Folder (nested)
result = client.create_folder("Q1 Reports", parent_folder_id="my-folder-id")Example Response
{
"folder": {
"id": "my-folder-id",
"name": "Research Papers",
"description": "2024 research collection",
"parent_folder_id": null,
"created_at": "2024-06-15 10:30:00",
"file_count": 0,
"children_count": 0
}
}List Folders
List folders for the authenticated user, optionally filtered by parent.
Parameters:
| Name | Type | Required | Description | Default |
|---|---|---|---|---|
| parent_folder_id | string | no | "root" for root-level only, a folder ID for subfolders, or omit for all | None |
Example
# List all folders
result = client.list_folders()
# List only root-level folders
result = client.list_folders(parent_folder_id="root")
# List subfolders of a specific folder
result = client.list_folders(parent_folder_id="my-folder-id")Example Response
{
"folders": [
{
"id": "my-folder-id",
"name": "Research Papers",
"description": "2024 research collection",
"parent_folder_id": null,
"created_at": "2024-06-15T10:30:00",
"updated_at": "2024-06-15T10:30:00",
"file_count": 5,
"children_count": 2
}
],
"total": 1
}Using Folders with Documents
Once you have folders, you can assign documents to them during upload and filter documents by folder.
Upload a document to a specific folder
result = client.submit_document("./report.pdf", folder_id="my-folder-id")List documents in a specific folder
# List documents in a specific folder
docs = client.list_documents(folder_id="my-folder-id")
# List all documents (no folder filter)
docs = client.list_documents()
# List documents not assigned to any folder
docs = client.list_documents(folder_id="root")
# With pagination
docs = client.list_documents(folder_id="my-folder-id", limit=10, offset=20)Get document metadata (includes folderId)
meta = client.get_document("pi-abc123def456")
print(meta["folderId"])