Quick Start Guide

The Bailo python client enables intuitive interaction with the Bailo service, from within a python environment. This example notebook will run through the following concepts:

Searching for existing models on Bailo.
Creating a new model and populating its model card.
Managing collaborators for a model.
Creating releases and uploading files.
Downloading files from a release.

Prerequisites:

Python 3.10 or higher (including a notebook environment for this demo).
A local or remote Bailo service (see https://github.com/gchq/Bailo).

Introduction

The Bailo python client is split into two sub-packages: core and helper.

Core: For direct interactions with the service endpoints.
Helper: For more intuitive interactions with the service, using classes (e.g. Model) to handle operations.

In order to create helper classes, you will first need to instantiate a Client() object from the core. By default, this object will not support any authentication. However, Bailo also supports PKI authentication, which you can use from Python by passing a PkiAgent() or TokenAgent() object into the Client() object when you instantiate it.

[ ]:

# Necessary import statements
from bailo import Model, Client

# Instantiating the PkiAgent(), if using.
from bailo import PkiAgent

agent = PkiAgent(cert="", key="", auth="")
# Instantiating the TokenAgent(), if using.
from bailo import TokenAgent

agent = TokenAgent(access_key="", secret_key="")

# Instantiating the Bailo client
client = Client("http://127.0.0.1:8080", agent)  # <- INSERT BAILO URL (if not hosting locally)

Searching for models

The Marketplace is Bailo’s home page for browsing models. Using the Python client, you can search for models programmatically with Model.search(). This returns a list of Model() objects matching your parameters.

You can filter by task, libraries, search text, and more. In the example below, we first list all available models, then demonstrate filtering by a search term.

[ ]:

# List all available models
models = Model.search(client=client)
print(f"Found {len(models)} model(s)")

# Search with a filter
filtered_models = Model.search(client=client, search="ResNet")
print(f"Found {len(filtered_models)} model(s) matching 'ResNet'")

Creating a new model

In this section, we will create a new model using the Model.create() classmethod. On the Bailo service, a model must consist of at least name and description parameters upon creation. Other attributes like model cards, files, or releases are added later on. Below, we use the Client() object created before when instantiating the new Model() object.

NOTE: This creates the model on your Bailo service too! The model_id is assigned by the backend, and we will use this later to retrieve the model.

[ ]:

model = Model.create(client=client, name="Demo Model", description="A quick-start demo model for Bailo.")

model_id = model.model_id
print(f"Created model {model_id}")

You may make changes to these attributes and then call the update() method to relay the changes to the service, as below:

model.name = "New Name"
model.update()

Selecting a schema and filling in the model card

When creating a model card, first we need to generate an empty one using the card_from_schema() method. In this instance, we will use minimal-general-v10. You can manage custom schemas using the Schema() helper class, but this is out of scope for this demo.

[ ]:

from bailo.core.enums import MinimalSchema

model.card_from_schema(schema_id=MinimalSchema.MODEL)

print(f"Model card version is {model.model_card_version}")

If successful, the above will have created a new model card, and the model_card_version attribute should be set to 1.

Next, we can populate the model card using the update_model_card() method. This can be used any time you want to make changes, and the backend will create a new model card version each time.

NOTE: Your model card must match the schema, otherwise a BailoException will be raised.

[ ]:

new_card = {
    "overview": {
        "tags": ["quickstart", "demo"],
        "modelSummary": "A quick-start demo model for Bailo.",
    }
}

model.update_model_card(model_card=new_card)

print(f"Model card version is {model.model_card_version}")

Managing collaborators

Collaborators control who can access and contribute to your model. Each collaborator is defined using a CollaboratorEntry, which links an entity (e.g. a user) to one or more roles. The available roles are:

Owner: Full control over the model.
Contributor: Can edit the model card and upload files.
Consumer: Can download files and images (after access approval).

After setting collaborators on the model, call model.update() to push the changes to the service.

[ ]:

from bailo.core.enums import CollaboratorEntry, Role

model.collaborators = [
    CollaboratorEntry(entity="user:user", roles=[Role.OWNER]),
]
model.update()

print(f"Collaborators updated for model {model.model_id}")

Creating a release

On the Bailo service, different versions of the same model are managed using releases. In this section we will create a release and then upload a file to it.

Release() is a separate helper class in itself, but we can use our Model() object to create and retrieve releases. Running the below code will create a new release of the model, and return an instantiated Release() object which we will use to upload files with.

NOTE: If the model does not have a populated model card schema then a BailoException will be thrown and the release will fail.

[ ]:

release = model.create_release(version="1.0.0", notes="Initial release of the demo model.")

print(f"Created release {release.version} with notes {release.notes!r}")

Uploading files to a release

To upload files for a release, we can use the release upload() method which takes a file path. In this demo, we use the existing demo_file.txt in the notebooks directory.

NOTE: The upload() method optionally takes the data parameter of BytesIO type to allow for other integrations, such as with S3. If the path parameter is a directory then upload() will automatically create a zip of the directory, allowing for multiple files to be uploaded as an archive.

Alternatively, you can upload files outside of a release context using client.simple_upload(), which takes a model_id, file name, and a BytesIO buffer directly.

[ ]:

release.upload(path="demo_file.txt")

print(f"Uploaded file to release {release.version}")

Uploading large files with multipart upload

For large files, the client supports multipart upload which splits the file into chunks and uploads them individually. This is more resilient for large transfers and is handled through three steps:

Start the upload — the server returns a fileId, uploadId, and the byte ranges (chunks) for each part.
Upload each part — send each chunk of data and collect the returned ETag for each part.
Finish the upload — send all the part ETags to complete the file.

These methods are available directly on the Client object. Below is a complete example using in-memory data.

[ ]:

from io import BytesIO

# Create some sample data (in practice, this would be a large file read in chunks)
file_data = b"x" * 2048  # 2 KB of sample data
file_size = len(file_data)

# Step 1: Start the multipart upload
start_response = client.start_multipart_upload(
    model_id=model_id,
    name="large_model_weights.bin",
    size=file_size,
)

file_id = start_response["fileId"]
upload_id = start_response["uploadId"]
chunks = start_response["chunks"]

print(f"Started multipart upload: fileId={file_id}, {len(chunks)} chunk(s)")

# Step 2: Upload each part using the byte ranges from the start response
parts = []
for i, chunk in enumerate(chunks):
    chunk_data = file_data[chunk["startByte"] : chunk["endByte"]]
    part_response = client.upload_multipart_part(
        model_id=model_id,
        file_id=file_id,
        upload_id=upload_id,
        part_number=i + 1,
        data=chunk_data,
    )
    parts.append({"ETag": part_response["ETag"], "PartNumber": i + 1})
    print(f"  Uploaded part {i + 1}/{len(chunks)}")

# Step 3: Finish the upload
finish_response = client.finish_multipart_upload(
    model_id=model_id,
    file_id=file_id,
    upload_id=upload_id,
    parts=parts,
)

print(f"Multipart upload complete: {finish_response['file']['name']}")

Downloading files from a release

You can download specific files from a release using the download() method, or download all files using the download_all() method.

NOTE: For download(), the filename parameter refers to the filename on Bailo, and path is the local destination. For download_all(), you can provide include and exclude lists with glob patterns for filtering, e.g. include=["*.txt", "*.json"].

[ ]:

release.download_all(path="downloads")

print("Downloaded all release files to 'downloads' directory")

Retrieving an existing model

In this section, we will retrieve our previous model using the Model.from_id() classmethod. This will create your Model() object as before, but using existing information retrieved from the service.

[ ]:

model = Model.from_id(client=client, model_id=model_id)

print(f"Model description: {model.description}")

What’s next?

For more detailed examples and advanced features, see the following notebooks:

models_and_releases_demo_pytorch.ipynb - In-depth model management with PyTorch integration.
datacards_demo.ipynb - Managing datacards.
schemas_demo.ipynb - Creating and managing custom schemas.
access_requests_demo.ipynb - Managing access requests.
experiment_tracking_demo.ipynb - Experiment tracking with MLFlow integration.