Projects and Collections

Create and manage projects and collections using object-oriented or path-based navigation

This tutorial explains how to create and manage projects and collections, the fundamental organizational units in the Nexus platform. You'll learn both object-oriented and path-based navigation styles.

Introduction

The Miura API provides two complementary navigation styles:

Object-Oriented Navigation: Navigate through objects (Project → Collection → Item)
Path-Based Navigation: Navigate using hierarchical paths (e.g., "project/collection/item")

Both styles are equally supported and can be used interchangeably. Choose the style that best fits your use case.

Projects and Collections are the fundamental organizational units in the Nexus platform:

Projects: Top-level containers that organize your work
Collections: Data containers within projects that hold your files and folders

Async vs Sync APIs

Important: The Miura API has two interfaces:

Async API (miura.api): Primary interface, uses async/await syntax
- Import: from miura.api import AsyncNexus
- Usage: async with AsyncNexus() as nexus: ...
- Methods: await nexus.get_project(...)
Sync API (miura): Compatibility layer, synchronous calls
- Import: from miura import Nexus
- Usage: with Nexus() as nexus: ... (automatic cleanup)
- Methods: nexus.get_project(...) (no await)

Both navigation styles work with both APIs, but:

Object-oriented navigation works with both Async and Sync APIs
Path-based navigation (nexus.get()) is primarily available in the Sync API

The examples below clearly label which API is being used to avoid confusion.

Working with Projects

Creating Projects

Object-Oriented Style:

python

import asyncio
from miura.api import AsyncNexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def main():
    async with AsyncNexus() as nexus:
        # Create a new project
        project = await nexus.create_project("my-project")
        logger.info(f"Created project: {project.name} ({project.uuid})")

asyncio.run(main())

Path-Based Style (Sync API):

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:
    # Create a new project
    project = nexus.create_project("my-project")
    logger.info(f"Created project: {project.name} ({project.uuid})")
    # Nexus automatically closes when exiting the with block

Listing Projects

Object-Oriented Style:

python

async with AsyncNexus() as nexus:
    # List all projects
    projects = await nexus.list_projects()
    logger.info(f"Found {len(projects)} project(s)")
    for project in projects:
        logger.info(f"  - {project.name} ({project.uuid})")
    
    # Iterate projects (for large datasets)
    async for project in nexus.iter_projects():
        logger.info(f"Project: {project.name}")

Path-Based Style:

python

with Nexus() as nexus:
    # List all projects
    projects = nexus.list_projects()
    logger.info(f"Found {len(projects)} project(s)")
    for project in projects:
        logger.info(f"  - {project.name} ({project.uuid})")
    
    # Iterate projects
    for project in nexus.iter_projects():
        logger.info(f"Project: {project.name}")
    # Nexus automatically closes when exiting the with block

Getting a Specific Project

Object-Oriented Style:

python

async with AsyncNexus() as nexus:
    # Get a specific project by name
    project = await nexus.get_project("my-project")
    logger.info(f"Retrieved project: {project.name}")

Path-Based Style:

python

with Nexus() as nexus:
    # Get a specific project using path
    project = nexus.get("my-project")
    logger.info(f"Retrieved project: {project.name}")
    # Nexus automatically closes when exiting the with block

Get-or-create (convenience methods)

When you need a project or collection to exist but don't care whether it was already there or just created, use the convenience methods get_or_create_project (on the Nexus client) and get_or_create_collection (on a project). They return the existing resource if found, or create it if missing—handy for scripts and workflows that assume a given project/collection exists.

Async:

python

async with AsyncNexus() as nexus:
    project = await nexus.get_or_create_project("my-project")
    schema = [{"pattern": ".*\\.vtk$", "min_occurrence": 1, "max_occurrence": None}]
    collection = await project.get_or_create_collection(
        collection_name="my-collection",
        schema=schema,
    )

Sync:

python

with Nexus() as nexus:
    project = nexus.get_or_create_project("my-project")
    schema = [{"pattern": ".*\\.vtk$", "min_occurrence": 1, "max_occurrence": None}]
    collection = project.get_or_create_collection(name="my-collection", schema=schema)

Working with Collections

Creating Collections

Object-Oriented Style:

python

async with AsyncNexus() as nexus:
    # Get project first
    project = await nexus.get_project("my-project")
    
    # Create collection with schema
    schema = [
        {
            "pattern": ".*\\.vtk$",
            "min_occurrence": 1,
            "max_occurrence": None
        }
    ]
    
    collection = await project.create_collection(
        collection_name="my-collection",
        schema=schema,
        metadata={"description": "VTK files collection"}
    )
    logger.info(f"Created collection: {collection.name} ({collection.uuid})")

Path-Based Style:

python

with Nexus() as nexus:
    # Get project using path
    project = nexus.get("my-project")
    
    # Create collection (same as object-oriented)
    schema = [
        {
            "pattern": ".*\\.vtk$",
            "min_occurrence": 1,
            "max_occurrence": None
        }
    ]
    
    collection = project.create_collection(
        name="my-collection",
        schema=schema,
        metadata={"description": "VTK files collection"}
    )
    logger.info(f"Created collection: {collection.name} ({collection.uuid})")
    # Nexus automatically closes when exiting the with block

Listing Collections

Object-Oriented Style:

python

async with AsyncNexus() as nexus:
    project = await nexus.get_project("my-project")
    
    # List collections
    collections = await project.list_collections()
    logger.info(f"Found {len(collections)} collection(s)")
    for collection_info in collections:
        logger.info(f"  - {collection_info.name} ({collection_info.uuid})")
    
    # Iterate collections
    async for collection_info in project.iter_collections():
        logger.info(f"Collection: {collection_info.name}")

Path-Based Style:

python

with Nexus() as nexus:
    project = nexus.get("my-project")
    
    # List collections (same as object-oriented)
    collections = project.list_collections()
    logger.info(f"Found {len(collections)} collection(s)")
    for collection_info in collections:
        logger.info(f"  - {collection_info.name} ({collection_info.uuid})")
    
    # Iterate collections
    for collection_info in project.iter_collections():
        logger.info(f"Collection: {collection_info.name}")
    # Nexus automatically closes when exiting the with block

Getting a Specific Collection

Object-Oriented Style:

python

async with AsyncNexus() as nexus:
    project = await nexus.get_project("my-project")
    collection = await project.get_collection("my-collection")
    logger.info(f"Retrieved collection: {collection.name}")

Path-Based Style:

python

with Nexus() as nexus:
    # Get collection directly using path
    collection = nexus.get("my-project/my-collection")
    logger.info(f"Retrieved collection: {collection.name}")
    # Nexus automatically closes when exiting the with block

Navigation Styles

Object-Oriented Navigation

Object-oriented navigation follows a natural hierarchy: you get a project, then get a collection from that project, then get a collection item from that collection.

Async API Example:

python

import asyncio
from miura.api import AsyncNexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def main():
    async with AsyncNexus() as nexus:
        # List all projects
        projects = await nexus.list_projects()
        for project in projects:
            logger.info(f"Project: {project.name} ({project.uuid})")
        
        # Get a specific project
        project = await nexus.get_project("my-project")
        logger.info(f"Retrieved project: {project.name}")

asyncio.run(main())

Sync API Example:

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:
    # List all projects
    projects = nexus.list_projects()
    for project in projects:
        logger.info(f"Project: {project.name} ({project.uuid})")
    
    # Get a specific project
    project = nexus.get_project("my-project")
    logger.info(f"Retrieved project: {project.name}")
    # Nexus automatically closes when exiting the with block

Getting Collections

Async API:

python

# From a project object (async)
project = await nexus.get_project("my-project")

# Get a specific collection
collection = await project.get_collection("my-collection")
logger.info(f"Retrieved collection: {collection.name}")

Sync API:

python

# List collections in the project
collections = project.list_collections()
for collection_info in collections:
    logger.info(f"Collection: {collection_info.name} ({collection_info.uuid})")

Getting Items

Async API:

python

# List items in the collection
items = await collection.list_items(path="/")
for item in items:
    logger.info(f"Item: {item.name}")

Sync API:

python

# From a collection object (sync)
collection = project.get_collection("my-collection")

# Get a specific item
item = collection.get_item("/data/file.txt")
if item:
    logger.info(f"Retrieved item: {item.name}")

Path-Based Navigation

Path-based navigation uses hierarchical paths to navigate directly to resources in a single call.

Note: Path-based navigation (nexus.get()) is available in the Sync API (miura). The Async API primarily uses object-oriented navigation, though you can combine both styles.

Sync API (Path-Based Navigation):

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:
    # Get a project
    project = nexus.get("my-project")
    logger.info(f"Project: {project.name}")
    
    # Get a collection
    collection = nexus.get("my-project/my-collection")
    logger.info(f"Collection: {collection.name}")
    
    # Get an item (file)
    item = nexus.get("my-project/my-collection/data/file.txt")
    logger.info(f"Item: {item.name}")
    logger.info(f"  URI: {item.item_uri}")
    
    # Get an item (folder)
    folder = nexus.get("my-project/my-collection/models/")
    logger.info(f"Folder: {folder.name}")
    # Nexus automatically closes when exiting the with block

Note: While path-based navigation is primarily a sync API feature, you can still use object-oriented navigation in the sync API:

python

# Sync API with object-oriented navigation
with Nexus() as nexus:
    project = nexus.get_project("my-project")
    collection = project.get_collection("my-collection")
    item = collection.get_item("/data/file.txt")
    # Nexus automatically closes when exiting the with block

Path Format

Paths follow this format:

Project: "project-name"
Collection: "project-name/collection-name"
Item: "project-name/collection-name/item-path"

Item paths within collections start with /:

File: "project/collection/data/file.txt"
Folder: "project/collection/models/"

When to Use Each Style

Use Object-Oriented Navigation When:

Working with multiple resources from the same parent

python

# Get project once, then work with multiple collections
project = await nexus.get_project("my-project")
collection1 = await project.get_collection("collection-1")
collection2 = await project.get_collection("collection-2")

Iterating through resources

python

project = await nexus.get_project("my-project")
async for collection in project.iter_collections():
    # Process each collection
    items = await collection.list_items()

You need the object for multiple operations

python

collection = await project.get_collection("my-collection")
# Use the same collection object for multiple operations
items = await collection.list_items()
await collection.upload(datasource)
await collection.download(path="/", local_path="./downloads/")

Async code (object-oriented is the primary async API style)

Use Path-Based Navigation When:

You know the exact path

python

# Direct navigation to a known resource
item = nexus.get("my-project/my-collection/data/file.txt")

Quick access to a specific resource

python

# One call to get exactly what you need
collection = nexus.get("my-project/my-collection")

Synchronous code (path-based is convenient in sync contexts)

Scripting and automation

python

# Easy to construct paths programmatically
for run_id in range(1, 10):
    item = nexus.get(f"project/collection/run_{run_id:03d}/data.txt")

Combining Both Styles

You can combine both navigation styles in the same code:

python

from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

with Nexus() as nexus:
    # Use path-based to get collection quickly
    collection = nexus.get("my-project/my-collection")
    
    # Then use object-oriented for collection operations
    items = collection.list_items()
    for item in items:
        logger.info(f"Item: {item.name}")
    
    # Or use path-based to get a specific item
    item = nexus.get("my-project/my-collection/data/file.txt")
    if item:
        # Use object-oriented for item operations
        result = item.download("./downloads/", confirm=False)
    # Nexus automatically closes when exiting the with block

Complete Examples

Example 1: Object-Oriented Workflow

python

import asyncio
from datetime import datetime
from miura.api import AsyncNexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def object_oriented_workflow():
    """Complete workflow using object-oriented navigation."""
    async with AsyncNexus() as nexus:
        # List projects
        logger.info("=== Listing Projects ===")
        projects = await nexus.list_projects()
        logger.info(f"Found {len(projects)} project(s)")
        
        # Create project
        logger.info("=== Creating Project ===")
        project_name = f"demo-project-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
        project = await nexus.create_project(project_name)
        logger.info(f"Created: {project.name}")
        
        # Get project
        logger.info("=== Getting Project ===")
        project = await nexus.get_project(project_name)
        logger.info(f"Retrieved: {project.name}")
        
        # Create collection
        logger.info("=== Creating Collection ===")
        collection_name = f"demo-collection-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
        schema = [{"pattern": ".*\\.txt$", "min_occurrence": 1}]
        collection = await project.create_collection(
            collection_name=collection_name,
            schema=schema
        )
        logger.info(f"Created: {collection.name}")
        
        # Get collection
        logger.info("=== Getting Collection ===")
        collection = await project.get_collection(collection_name)
        logger.info(f"Retrieved: {collection.name}")
        
        # List collections
        logger.info("=== Listing Collections ===")
        collections = await project.list_collections()
        logger.info(f"Found {len(collections)} collection(s)")

asyncio.run(object_oriented_workflow())

Example 2: Path-Based Workflow

python

from datetime import datetime
from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

def path_based_workflow():
    """Complete workflow using path-based navigation."""
    with Nexus() as nexus:
        # List projects
        logger.info("=== Listing Projects ===")
        projects = nexus.list_projects()
        logger.info(f"Found {len(projects)} project(s)")
        
        # Create project
        logger.info("=== Creating Project ===")
        project_name = f"demo-project-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
        project = nexus.create_project(project_name)
        logger.info(f"Created: {project.name}")
        
        # Get project using path
        logger.info("=== Getting Project ===")
        project = nexus.get(project_name)
        logger.info(f"Retrieved: {project.name}")
        
        # Create collection
        logger.info("=== Creating Collection ===")
        collection_name = f"demo-collection-{datetime.now().strftime('%Y%m%d-%H%M%S')}"
        schema = [{"pattern": ".*\\.txt$", "min_occurrence": 1}]
        collection = project.create_collection(
            name=collection_name,
            schema=schema
        )
        logger.info(f"Created: {collection.name}")
        
        # Get collection using path
        logger.info("=== Getting Collection ===")
        collection = nexus.get(f"{project_name}/{collection_name}")
        logger.info(f"Retrieved: {collection.name}")
        
        # List collections
        logger.info("=== Listing Collections ===")
        collections = project.list_collections()
        logger.info(f"Found {len(collections)} collection(s)")
    # Nexus automatically closes when exiting the with block

path_based_workflow()

Example 3: Mixed Style Workflow

python

import asyncio
from miura.api import AsyncNexus
from miura import Nexus
from miura.logging import get_logger

logger = get_logger(__name__)

async def mixed_workflow():
    """Workflow combining both navigation styles."""
    # Use async for bulk operations
    async with AsyncNexus() as nexus:
        # Object-oriented: List and iterate projects
        async for project_info in nexus.iter_projects():
            logger.info(f"Processing project: {project_info.name}")
            

            # Object-oriented: Get project for operations
            project = await nexus.get_project(project_info.name)
            

            # Object-oriented: Iterate collections
            async for collection_info in project.iter_collections():
                logger.info(f"  Processing collection: {collection_info.name}")
    

    # Use sync for quick lookups
    nexus_sync = Nexus()
    try:
        # Path-based: Quick access to specific resources
        collection = nexus_sync.get("my-project/my-collection")
        logger.info(f"Quick access to: {collection.name}")
    finally:
        nexus_sync.close()

asyncio.run(mixed_workflow())

Best Practices

1. Choose Based on Context

Async code: Prefer object-oriented (it's the primary async API)
Sync code: Either style works; path-based can be more convenient
Bulk operations: Prefer object-oriented with iterators
Quick lookups: Path-based is often more concise

2. Be Consistent Within a Function

While you can mix styles, being consistent within a function improves readability:

python

# Also good: Consistent path-based (in sync context)
def quick_lookup():
    item = nexus.get("project/collection/file.txt")
    return item

3. Use Iterators for Large Datasets

Both styles support iterators for efficient processing:

python

# Object-oriented iterator
async for project in nexus.iter_projects():
    async for collection in project.iter_collections():
        async for item in collection.iter_items():
            # Process item

4. Handle Missing Resources

Both styles raise NotFoundError when resources don't exist:

python

from miura.sdk import NotFoundError

try:
    project = await nexus.get_project("nonexistent")
except NotFoundError:
    logger.warning("Project not found")
    # Create it or handle the error

5. Use Descriptive Names

Choose meaningful names for projects and collections:

python

# Good: Descriptive names
project = await nexus.create_project("cfd-simulation-2024")
collection = await project.create_collection(collection_name="turbulence-results")

API Reference - Complete API documentation
Quick Start Guide - Get started with the Nexus API

Getting Started

Nexus Quick Start - Get up and running with the Miura Nexus API in minutes

Schemas

Understanding schemas as quality gates for data ingestion