Projects and Collections

Create and manage projects and collections using object-oriented or path-based navigation

This tutorial explains how to create and manage projects and collections, the fundamental organizational units in the Nexus platform. You'll learn both object-oriented and path-based navigation styles.

Introduction

The Miura API provides two complementary navigation styles:

Object-Oriented Navigation: Navigate through objects (Project → Collection → Item)
Path-Based Navigation: Navigate using hierarchical paths (e.g., "project/collection/item")

Both styles are equally supported and can be used interchangeably. Choose the style that best fits your use case.

Projects and Collections are the fundamental organizational units in the Nexus platform:

Projects: Top-level containers that organize your work
Collections: Data containers within projects that hold your files and folders

Async vs Sync APIs

Important: The Miura API has two interfaces:

Async API (miura.aio): Primary interface, uses async/await syntax
- Import: from miura.aio import AsyncNexus
- Usage: async with AsyncNexus() as nexus: ...
- Methods: await nexus.get_project(...)
Sync API (miura): Compatibility layer, synchronous calls
- Import: from miura import Nexus
- Usage: with Nexus() as nexus: ... (automatic cleanup)
- Methods: nexus.get_project(...) (no await)

Both navigation styles work with both APIs, but:

Object-oriented navigation works with both Async and Sync APIs
Path-based navigation (nexus.get()) is primarily available in the Sync API

The examples below clearly label which API is being used to avoid confusion.

Working with Projects

Creating Projects

Object-Oriented Style:

python

import asyncio
from miura.aio import AsyncNexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def main():
    async with AsyncNexus() as nexus:

    # Create a new project
    project = await nexus.create_project("my-project")
    logger.info(f"Created project: {project.name} ({project.uuid})")

asyncio.run(main())

Path-Based Style (Sync API):

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:

# Create a new project
project = nexus.create_project("my-project")
logger.info(f"Created project: {project.name} ({project.uuid})")
# Nexus automatically closes when exiting the with block

Listing Projects

Object-Oriented Style:

python

async with AsyncNexus() as nexus:

# List all projects
projects = await nexus.list_projects()
logger.info(f"Found {len(projects)} project(s)")
for project in projects:
    logger.info(f"  - {project.name} ({project.uuid})")

# Iterate projects (for large datasets)
async for project in nexus.iter_projects():
    logger.info(f"Project: {project.name}")

Path-Based Style:

python

with Nexus() as nexus:

# List all projects
projects = nexus.list_projects()
logger.info(f"Found {len(projects)} project(s)")
for project in projects:
    logger.info(f"  - {project.name} ({project.uuid})")

# Iterate projects
for project in nexus.iter_projects():
    logger.info(f"Project: {project.name}")
# Nexus automatically closes when exiting the with block

Getting a Specific Project

Object-Oriented Style:

python

async with AsyncNexus() as nexus:

# Get a specific project by name
project = await nexus.get_project("my-project")
logger.info(f"Retrieved project: {project.name}")

Path-Based Style:

python

with Nexus() as nexus:

# Get a specific project using path
project = nexus.get("my-project")
logger.info(f"Retrieved project: {project.name}")
# Nexus automatically closes when exiting the with block

Working with Collections

Creating Collections

Object-Oriented Style:

python

async with AsyncNexus() as nexus:

# Get project first
project = await nexus.get_project("my-project")

# Create collection with schema
schema = [
    {
        "pattern": ".*\\.vtk$",
        "min_occurrence": 1,
        "max_occurrence": None
    }
]

python

    collection = await project.create_collection(
        collection_name="my-collection",
        schema=schema,
        metadata={"description": "VTK files collection"}
    )
    logger.info(f"Created collection: {collection.name} ({collection.uuid})")

Path-Based Style:

python

with Nexus() as nexus:

# Get project using path
project = nexus.get("my-project")

# Create collection (same as object-oriented)
schema = [
    {
        "pattern": ".*\\.vtk$",
        "min_occurrence": 1,
        "max_occurrence": None
    }
]

python

    collection = project.create_collection(
        name="my-collection",
        schema=schema,
        metadata={"description": "VTK files collection"}
    )
    logger.info(f"Created collection: {collection.name} ({collection.uuid})")

# Nexus automatically closes when exiting the with block

Listing Collections

Object-Oriented Style:

python

async with AsyncNexus() as nexus:
    project = await nexus.get_project("my-project")

# List collections
collections = await project.list_collections()
logger.info(f"Found {len(collections)} collection(s)")
for collection_info in collections:
    logger.info(f"  - {collection_info.name} ({collection_info.uuid})")


# Iterate collections
async for collection_info in project.iter_collections():
    logger.info(f"Collection: {collection_info.name}")

Path-Based Style:

python

with Nexus() as nexus:
    project = nexus.get("my-project")

# List collections (same as object-oriented)
collections = project.list_collections()
logger.info(f"Found {len(collections)} collection(s)")
for collection_info in collections:
    logger.info(f"  - {collection_info.name} ({collection_info.uuid})")


# Iterate collections
for collection_info in project.iter_collections():
    logger.info(f"Collection: {collection_info.name}")
# Nexus automatically closes when exiting the with block

Getting a Specific Collection

Object-Oriented Style:

python

async with AsyncNexus() as nexus:
    project = await nexus.get_project("my-project")
    collection = await project.get_collection("my-collection")
    logger.info(f"Retrieved collection: {collection.name}")

Path-Based Style:

python

with Nexus() as nexus:

# Get collection directly using path
collection = nexus.get("my-project/my-collection")
logger.info(f"Retrieved collection: {collection.name}")
# Nexus automatically closes when exiting the with block

Navigation Styles

Object-Oriented Navigation

Object-oriented navigation follows a natural hierarchy: you get a project, then get a collection from that project, then get a collection item from that collection.

Async API Example:

python

import asyncio
from miura.aio import AsyncNexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def main():
    async with AsyncNexus() as nexus:

    # List all projects
    projects = await nexus.list_projects()
    for project in projects:
        logger.info(f"Project: {project.name} ({project.uuid})")
    

    # Get a specific project
    project = await nexus.get_project("my-project")
    logger.info(f"Retrieved project: {project.name}")

asyncio.run(main())

Sync API Example:

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:

# List all projects
projects = nexus.list_projects()
for project in projects:
    logger.info(f"Project: {project.name} ({project.uuid})")


# Get a specific project
project = nexus.get_project("my-project")
logger.info(f"Retrieved project: {project.name}")
# Nexus automatically closes when exiting the with block

Getting Collections

Async API:

From a project object (async)

project = await nexus.get_project("my-project")

Get a specific collection

collection = await project.get_collection("my-collection") logger.info(f"Retrieved collection: {collection.name}")

Sync API:

List collections in the project

collections = project.list_collections() for collection_info in collections: logger.info(f"Collection: {collection_info.name} ({collection_info.uuid})")

Getting Items

Async API:

List items in the collection

response = await collection.list_items(path="/", page=1, page_size=10) items = response.get("items", ) for item in items: logger.info(f"Item: {item.name}")

From a collection object (sync)

collection = project.get_collection("my-collection")

Get a specific item

item = collection.get_item("/data/file.txt") if item: logger.info(f"Retrieved item: {item.name}")

Path-Based Navigation

Path-based navigation uses hierarchical paths to navigate directly to resources in a single call.

Note: Path-based navigation (nexus.get()) is available in the Sync API (miura). The Async API primarily uses object-oriented navigation, though you can combine both styles.

Sync API (Path-Based Navigation):

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:

# Get a project
project = nexus.get("my-project")
logger.info(f"Project: {project.name}")


# Get a collection
collection = nexus.get("my-project/my-collection")
logger.info(f"Collection: {collection.name}")


# Get an item (file)
item = nexus.get("my-project/my-collection/data/file.txt")
logger.info(f"Item: {item.name}")
logger.info(f"  URI: {item.item_uri}")


# Get an item (folder)
folder = nexus.get("my-project/my-collection/models/")
logger.info(f"Folder: {folder.name}")
# Nexus automatically closes when exiting the with block

Note: While path-based navigation is primarily a sync API feature, you can still use object-oriented navigation in the sync API:

Sync API with object-oriented navigation

with Nexus() as nexus: project = nexus.get_project("my-project") collection = project.get_collection("my-collection") item = collection.get_item("/data/file.txt") # Nexus automatically closes when exiting the with block

Path Format

Paths follow this format:

Project: "project-name"
Collection: "project-name/collection-name"
Item: "project-name/collection-name/item-path"

Item paths within collections start with /:

File: "project/collection/data/file.txt"
Folder: "project/collection/models/"

When to Use Each Style

Use Object-Oriented Navigation When:

Working with multiple resources from the same parent
Get project once, then work with multiple collections

project = await nexus.get_project("my-project") collection1 = await project.get_collection("collection-1") collection2 = await project.get_collection("collection-2")
Iterating through resources
project = await nexus.get_project("my-project") async for collection in project.iter_collections(): # Process each collection items = await collection.list_items()
You need the object for multiple operations
collection = await project.get_collection("my-collection")
Use the same collection object for multiple operations

items = await collection.list_items() await collection.upload(datasource) await collection.download(path="/", local_path="./downloads/")
Async code (object-oriented is the primary async API style)

Use Path-Based Navigation When:

You know the exact path
Direct navigation to a known resource

item = nexus.get("my-project/my-collection/data/file.txt")
Quick access to a specific resource
One call to get exactly what you need

collection = nexus.get("my-project/my-collection")
Synchronous code (path-based is convenient in sync contexts)
Scripting and automation
Easy to construct paths programmatically

for run_id in range(1, 10): item = nexus.get(f"project/collection/run_/data.txt")

Combining Both Styles

You can combine both navigation styles in the same code:

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:

# Use path-based to get collection quickly
collection = nexus.get("my-project/my-collection")


# Then use object-oriented for collection operations
items = collection.list_items()
for item in items:
    logger.info(f"Item: {item.name}")


# Or use path-based to get a specific item
item = nexus.get("my-project/my-collection/data/file.txt")
if item:
    # Use object-oriented for item operations
    result = item.download("./downloads/", confirm=False)
# Nexus automatically closes when exiting the with block

Complete Examples

Example 1: Object-Oriented Workflow

python

import asyncio
from datetime import datetime
from miura.aio import AsyncNexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def object_oriented_workflow():
    """Complete workflow using object-oriented navigation."""
    async with AsyncNexus() as nexus:

    # List projects
    logger.info("=== Listing Projects ===")
    projects = await nexus.list_projects()
    logger.info(f"Found {len(projects)} project(s)")
    

    # Create project
    logger.info("=== Creating Project ===")
    project_name = f"demo-project-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
    project = await nexus.create_project(project_name)
    logger.info(f"Created: {project.name}")
    

    # Get project
    logger.info("=== Getting Project ===")
    project = await nexus.get_project(project_name)
    logger.info(f"Retrieved: {project.name}")
    

    # Create collection
    logger.info("=== Creating Collection ===")
    collection_name = f"demo-collection-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
    schema = [{"pattern": ".*\\.txt$", "min_occurrence": 1}]
    collection = await project.create_collection(
        collection_name=collection_name,
        schema=schema
    )
    logger.info(f"Created: {collection.name}")
    

    # Get collection
    logger.info("=== Getting Collection ===")
    collection = await project.get_collection(collection_name)
    logger.info(f"Retrieved: {collection.name}")
    

    # List collections
    logger.info("=== Listing Collections ===")
    collections = await project.list_collections()
    logger.info(f"Found {len(collections)} collection(s)")

asyncio.run(object_oriented_workflow())

Example 2: Path-Based Workflow

python

from datetime import datetime
from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

def path_based_workflow():
    """Complete workflow using path-based navigation."""
    with Nexus() as nexus:

# List projects
    logger.info("=== Listing Projects ===")
    projects = nexus.list_projects()
    logger.info(f"Found {len(projects)} project(s)")
    

    # Create project
    logger.info("=== Creating Project ===")
    project_name = f"demo-project-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
    project = nexus.create_project(project_name)
    logger.info(f"Created: {project.name}")
    

    # Get project using path
    logger.info("=== Getting Project ===")
    project = nexus.get(project_name)
    logger.info(f"Retrieved: {project.name}")
    

    # Create collection
    logger.info("=== Creating Collection ===")
    collection_name = f"demo-collection-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
    schema = [{"pattern": ".*\\.txt$", "min_occurrence": 1}]
    collection = project.create_collection(
        name=collection_name,
        schema=schema
    )
    logger.info(f"Created: {collection.name}")
    

    # Get collection using path
    logger.info("=== Getting Collection ===")
    collection = nexus.get(f"{project_name}/{collection_name}")
    logger.info(f"Retrieved: {collection.name}")
    

    # List collections
    logger.info("=== Listing Collections ===")
    collections = project.list_collections()
    logger.info(f"Found {len(collections)} collection(s)")
# Nexus automatically closes when exiting the with block

path_based_workflow()

Example 3: Mixed Style Workflow

python

import asyncio
from miura.aio import AsyncNexus
from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def mixed_workflow():
    """Workflow combining both navigation styles."""

# Use async for bulk operations
async with AsyncNexus() as nexus:
    # Object-oriented: List and iterate projects
    async for project_info in nexus.iter_projects():
        logger.info(f"Processing project: {project_info.name}")
        

        # Object-oriented: Get project for operations
        project = await nexus.get_project(project_info.name)
        

        # Object-oriented: Iterate collections
        async for collection_info in project.iter_collections():
            logger.info(f"  Processing collection: {collection_info.name}")


# Use sync for quick lookups
nexus_sync = Nexus()
try:
    # Path-based: Quick access to specific resources
    collection = nexus_sync.get("my-project/my-collection")
    logger.info(f"Quick access to: {collection.name}")
finally:
    nexus_sync.close()

asyncio.run(mixed_workflow())

Best Practices

1. Choose Based on Context

Async code: Prefer object-oriented (it's the primary async API)
Sync code: Either style works; path-based can be more convenient
Bulk operations: Prefer object-oriented with iterators
Quick lookups: Path-based is often more concise

2. Be Consistent Within a Function

While you can mix styles, being consistent within a function improves readability:

Also good: Consistent path-based (in sync context)

def quick_lookup(): item = nexus.get("project/collection/file.txt") return item

3. Use Iterators for Large Datasets

Both styles support iterators for efficient processing:

Object-oriented iterator

async for project in nexus.iter_projects(): async for collection in project.iter_collections(): async for item in collection.iter_items(): # Process item

4. Handle Missing Resources

Both styles raise NotFoundError when resources don't exist:

python

from miura.api.exceptions import NotFoundError

try:
    project = await nexus.get_project("nonexistent")
except NotFoundError:
    logger.warning("Project not found")

# Create it or handle the error

5. Use Descriptive Names

Choose meaningful names for projects and collections:

python


# Good: Descriptive names
project = await nexus.create_project("cfd-simulation-2024")
collection = await project.create_collection(collection_name="turbulence-results")

API Reference - Complete API documentation
Quick Start Guide - Get started with the Nexus API

Getting Started

Nexus Quick Start - Get up and running with the Miura Nexus API in minutes

Schemas and Schema Generation

Understanding schemas as quality gates and automatically generating them

Projects and Collections

From a project object (async)

Get a specific collection

List collections in the project

List items in the collection

From a collection object (sync)

Get a specific item

Sync API with object-oriented navigation

Get project once, then work with multiple collections

Use the same collection object for multiple operations

Direct navigation to a known resource

One call to get exactly what you need

Easy to construct paths programmatically

Also good: Consistent path-based (in sync context)

Object-oriented iterator