Read the Same Report After Your Dagster Worker Exits

A Dagster asset builds a report and the worker exits. Upload the closed JSON file, and a new process checks the same run id.

A Dagster asset finishes a report. Then the worker exits. The file was on that worker's disk, and the disk is gone.

The local run record was only for that process, and it is gone too. Building the report again means running the work a second time. The short id the asset returned is not the file.

The asset uploads the closed JSON file to mountable.io and returns only that id. mountable.io is a persistent filesystem for AI agents. The filesystem id stays in the worker's environment, so the asset does not return that id or a path.

Diagram of a Dagster writer that uploads a closed JSON report to a mountable.io filesystem under reports/alice, returns an id, and exits. A new process receives that id and the run id, reads the file, and checks the run id, asset key, row count and checksum. — The writer asset uploads the report and exits. A new process checks the same file.

The run closes the report, then returns an id

One local run materializes two assets. The first builds two synthetic rows in memory. The second closes a JSON report that records the Dagster run id, the asset key, the row count and the checksum of those rows. It uploads that file and returns the report id.

from mountable_sdk import Mountable
 
fs = Mountable().filesystem(filesystem_id, "rw")
fs.write(f"reports/alice/{report_id}.json", report_bytes, if_none_match=True)

if_none_match=True creates the file only when it is not already there. The example generates those two rows and no customer data.

A new process checks the run

Stop the writer and start a new process. It does not open the Dagster instance. It receives the report id and the run id. It opens a read-only handle and checks the stored run id, asset key, row count and checksum.

uv run python report.py read --report-id "$ID" --run-id "$RUN_ID"

The command prints match, the path, and those same fields.

A second create leaves the first file

Write the same id again. The new upload stops, and the first bytes stay where they are. The reader still prints the original checksum.

A failed upload fails the asset

If the write fails, the asset fails. The command exits without a report_id= line. There is no id to hand to the next process.

Delete the file when you are done

uv run python report.py cleanup --report-id "$ID"

This deletes that one file. A file that is already gone is left gone.

Storage is $0.20 per GiB-month at the normal usage prices. A short report like this one is under a kilobyte. Delete it when you no longer need it.

For one filesystem and a hard monthly spend cap per end user, follow the capped workspace guide. This recipe uses one existing filesystem.

Try it

The Dagster guide: set up the example, write a report, read it from a new process, and clean up.
The example code: the assets, the offline tests, and the two-process check.
Python and TypeScript SDKs: read and write files from your backend.

mountable.io maintains this example. It is not part of Dagster.