extension-object-storage

Object Storage

Object storage extension for Caffeine AI.

Overview

This skill adds off-chain file/object storage with on-chain references. The MixinObjectStorage mixin provides infrastructure for file operations; you track uploaded files in your own data structures using Storage.ExternalBlob.

Required Setup Checklist

All four steps are mandatory. Skipping any one causes 403 Forbidden: Invalid payload at upload time.

1. mops dependency — add caffeineai-object-storage to mops.toml under [dependencies].
2. Mixin invocation — include MixinObjectStorage() in main.mo (imported from "mo:caffeineai-object-storage/Mixin").
3. Storage.ExternalBlob types — every data field that represents a file MUST use Storage.ExternalBlob, never Text.
4. Frontend npm package — @caffeineai/object-storage installed and ExternalBlob.fromBytes() used at the call site.

CRITICAL: The frontend package (@caffeineai/object-storage) does NOT work without the backend mops package (caffeineai-object-storage). Installing only the npm package and not the mops package causes silent upload failures (403 from the storage gateway). You MUST install both together.

Backend

File content is stored off-chain. The backend manages references to external files using the Storage.ExternalBlob type from mo:caffeineai-object-storage/Storage. The frontend handles the actual upload/download; the backend only stores the reference.

CRITICAL: ANY data field that represents a file, image, photo, document, or media MUST use Storage.ExternalBlob as its type -- NEVER Text. Using Text breaks the upload/download proxy. Method parameters that accept file uploads MUST also use Storage.ExternalBlob, not Text.

Correct:

blob : Storage.ExternalBlob

Wrong:

blobId : Text
imageUrl : Text
fileRef : Text

Module API

The only type you use from mo:caffeineai-object-storage/Storage is ExternalBlob (which is Blob). All other functions in Storage.mo are internal infrastructure used by MixinObjectStorage -- do not call them directly.

Setup in main.mo

include MixinObjectStorage() MUST be placed in main.mo, not in a custom mixin file. Your own file-tracking logic goes in a separate mixin.

import MixinObjectStorage "mo:caffeineai-object-storage/Mixin";
import Storage "mo:caffeineai-object-storage/Storage";

actor {
  include MixinObjectStorage();

   // Track file references
  type Data = {
        id: Text;
        blob: Storage.ExternalBlob;
        name: Text;
        // other metadata
    };
};

Wrong: Do NOT Implement Storage Methods Yourself

NEVER create your own implementation of _immutableObjectStorageCreateCertificate or any other _immutableObjectStorage* method. These are platform-reserved method names provided exclusively by the MixinObjectStorage mixin from the mops package. Hand-written implementations produce wrong return types and cause 403 Forbidden: Invalid payload at upload time.

Wrong — inline stub in main.mo:

// WRONG: Do not write this yourself
public shared func _immutableObjectStorageCreateCertificate(fileHash : Text) : async Blob {
  CertifiedData.set(Blob.fromArray(hashBytes));
  Blob.fromArray([])
};

Wrong — custom mixin file mimicking the platform shape:

// WRONG: Do not create src/backend/mixins/object-storage-api.mo
import ObjectStorageMixin "mixins/object-storage-api";
include ObjectStorageMixin();

The correct import path is ALWAYS "mo:caffeineai-object-storage/Mixin" — a mops package, never a relative path. Any relative import like "mixins/object-storage-api" or "./ObjectStorage" is wrong.

The correct signature produced by the platform mixin is:

_immutableObjectStorageCreateCertificate : (blobHash : Text) -> async record { method : Text; blob_hash : Text }

Any other return type (Blob, (), Text, etc.) will fail gateway validation.

Frontend

Backend Blob fields are represented as ExternalBlob on the frontend.

import { ExternalBlob } from "@caffeineai/object-storage";
import type { FileRecord } from "@caffeineai/object-storage";

ExternalBlob API

class ExternalBlob {
  getBytes(): Promise<Uint8Array<ArrayBuffer>>;
  getDirectURL(): string;
  static fromURL(url: string): ExternalBlob;
  static fromBytes(blob: Uint8Array<ArrayBuffer>): ExternalBlob;
  withUploadProgress(onProgress: (percentage: number) => void): ExternalBlob;
}

Uploading Files

Convert the browser File object to ExternalBlob and pass the original filename alongside:

const handleUpload = async (file: File) => {
  const bytes = new Uint8Array(await file.arrayBuffer());
  const blob = ExternalBlob.fromBytes(bytes).withUploadProgress((pct) => {
    setProgress(pct);
  });

  await actor.uploadFile(file.name, blob);
};

Always send file.name so the backend stores the original filename.

Displaying Files

Use getDirectURL() for inline display (images, videos). This returns an opaque proxy URL -- it has no file extension, so never inspect the URL to determine file type.

<img src={record.blob.getDirectURL()} alt={record.filename} />

File Type Detection

CRITICAL: Never detect file types by inspecting the URL from getDirectURL(). These are opaque proxy URLs with no extension. Instead use the filename field from the backend record:

const isImage = (filename: string) =>
  /\.(jpg|jpeg|png|gif|webp|svg|bmp|ico)$/i.test(filename);

// Conditional rendering
{isImage(record.filename) ? (
  <img src={record.blob.getDirectURL()} alt={record.filename} />
) : (
  <div>{record.filename}</div>
)}

If the backend also returns a mimeType field, prefer that:

const isImage = (mimeType?: string) => mimeType?.startsWith("image/");

Downloading Files

For downloads with the original filename, use getBytes() to create a downloadable link:

const handleDownload = async (record: FileRecord) => {
  const bytes = await record.blob.getBytes();
  const blob = new Blob([bytes]);
  const url = URL.createObjectURL(blob);
  const a = document.createElement("a");
  a.href = url;
  a.download = record.filename;
  document.body.appendChild(a);
  a.click();
  document.body.removeChild(a);
  URL.revokeObjectURL(url);
};

Use getDirectURL() for inline display, getBytes() for save-as downloads.

Summary

Use case	Method	Notes
Display image/video	blob.getDirectURL()	Streaming, cached
Download with filename	blob.getBytes()	Wrap in Blob + anchor
Upload from browser	ExternalBlob.fromBytes(bytes)	Pair with .withUploadProgress()
Detect file type	filename or mimeType field	NEVER inspect the URL

Verifying the Setup

Confirm the backend has the mops dependency installed. Check src/backend/mops.toml:

[dependencies]
caffeineai-object-storage = "0.1.2"

If caffeineai-object-storage is missing from [dependencies], object storage will not work regardless of what the frontend does. Add it, run mops install, and rebuild.

Troubleshooting

Error	Cause	Fix
403 Forbidden: Invalid payload on PUT /v1/blob-tree/	Backend canister missing _immutableObjectStorageCreateCertificate or returning wrong type	Install caffeineai-object-storage in mops.toml, add include MixinObjectStorage() in main.mo, redeploy
403 Forbidden: Invalid payload (all files)	@caffeineai/object-storage npm installed but caffeineai-object-storage mops NOT installed	Add the mops dependency and rebuild backend
Method exists but still 403	Hand-written stub returns wrong type (e.g. Blob or () instead of record { method; blob_hash })	Remove the custom implementation, use the platform mixin instead
Forbidden: Owner does not have an account with the cashier	Cashier registration issue (unrelated to this skill)	Redeploy the backend canister to trigger self-healing registration