I see. But I am still trying to understand one bit.

If I am understanding this correctly, you have got two buckets. One you use as a staging one to do all the preparation and another one where your final changes are pushed. The latter is the one that is always used by the pipeline. Correct?

If so, my question is how is the final .jsonl file pushed to the main GCS Bucket? For the purposes of the post, are you pushing that one manually?