Azure Data Factory:How to delete blobs/files older than n days ?

This task utilized managed service of Azure named as Azure Data Factory.

ADF template can be downloaded from below.

Delete files older than n days

.zip

Download ZIP • 27KB

Above ADF template can be imported and can be utilized to delete file under a container or a folder with a wildcard prefix.

1. Go to data factory and add a data factory.

2. Go to Author & Monitor (If not opening please try to open in Mozilla and Chrome or update your browser)

3. Import this template

4. This pipeline take Source and logging path (you can disable logging)

5. Go to manage in left menu.

6. Check the Linked Service RemoveOld and add your source bucket test the connection.

1. Go back to Author select your pipeline

2. Select the file path in Binary1 in my case I have a storage account name testbucket in that there is a container named foo.

Eg - testBucket(storage account) / foo (container) / file1, file2, file3, assets

I want to delete files which are under foo container.

1. In wildcard file name add the desired name

2. In end time box you can see -1 this implies last modified date from the current time you can click and make it n older days like 30 days older -1 to -30.

3. Checking logging setting and enter the desired path.

4. Debug and Trigger the pipeline to test it from the monitor panel.

You can also check the logs and name of deleted files will be there.

** Adding Trigger to pipelines will be added in next version of document

Lifecycle management rules can also be used if you want to delete whole blobs/folder in a particular storage account can't be used for individual files.

Hope this helps :)