Update your Glue script

In some cases you may need to update the Glue Python script in your AWS S3 Parquet data lake integration. Below we describe how this is done.

Update the Glue Python script

  1. Download the latest Glue script
  2. Go to your Glue jobs overview page and click your job
  3. In the Scripts tab, replace the existing script with the latest version and click ‘Save’
  4. Upload the Glue script in your S3 input bucket for revision tracking

Note on revision numbers

If you update your Glue script to a newer revision vs. when you originally deployed your data lake, the revision number will no longer match the number in your input bucket root. This is, however, not in itself an issue.

The revision number matching is mainly relevant during the initial deployment as the CloudFormation stacks rely on specific revision numbers. If your stack is already successfully deployed, you can freely change the Glue script subsequently without changing the rest of the stack.

Note, however, that we recommend also adding the matching version of the Glue script in your S3 input bucket - this is required if you e.g. wish to deploy data aggregation via CloudFormation.