Skip to main content
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Calling all Data Engineers! Fabric Data Engineer (Exam DP-700) live sessions are back! Starting October 16th. Sign up.

Reply
HamidBee
Valued Contributor

How to Set Dynamic File Names in a Copy Activity for Lakehouse in Microsoft Fabric?

Hi everyone,

 

I'm setting up a data pipeline in Microsoft Fabric, where I need to copy an Excel file into a Lakehouse folder every month. The file name should be dynamic to reflect the current year and month (e.g., MonthlyReport_202310.xlsx for October 2023).

I've tried using expressions like:

 

concat('MonthlyReport_', formatDateTime(utcnow(), 'yyyyMM'), '.xlsx')

 

in the File name field, but itโ€™s creating a literal folder with the expression text instead of evaluating it dynamically. I also tried defining the concatenated file name in a pipeline parameter and referencing it in the destination path, but Iโ€™m still running into issues.

 

Has anyone successfully configured a dynamic file name in a Copy Activity for Lakehouse? Any tips on how to make this work properly would be greatly appreciated.

 

Thanks for any help.

1 ACCEPTED SOLUTION
Anonymous
Not applicable

Hi @HamidBee ,

 

There is an alternative:

 

New and changed files can be copied from one lakehouse to another in an incremental manner. It uses โ€œ Filter by Last Modified โ€ to determine which files to copy.

 

After completing the steps here, Data Factory scans all files in the source store, applies file filters that are filtered by โ€œ Filter by Last Modified โ€, and then copies the new files and files that have been updated since the last time to the target store.

 

For more information, please refer to:

Incrementally copy new and changed files based on the last modified date - Microsoft Fabric | Micros...

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

View solution in original post

3 REPLIES 3
Anonymous
Not applicable

Hi @HamidBee ,

 

There is an alternative:

 

New and changed files can be copied from one lakehouse to another in an incremental manner. It uses โ€œ Filter by Last Modified โ€ to determine which files to copy.

 

After completing the steps here, Data Factory scans all files in the source store, applies file filters that are filtered by โ€œ Filter by Last Modified โ€, and then copies the new files and files that have been updated since the last time to the target store.

 

For more information, please refer to:

Incrementally copy new and changed files based on the last modified date - Microsoft Fabric | Micros...

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Anonymous
Not applicable

Hi @HamidBee ,

 

Is my follow-up just to ask if the problem has been solved?

 

If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?

 

Thank you very much for your cooperation!

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Jeanxyz
Valued Contributor

I think you are close, but instead of use the concate() directly in the copy activity, you need to create a variable called 'source file name', then add a Set Variable activity where you can define the variable using Add dynamic content, and input @concat('Monthly Report', utcNow('yyyyMMM'),'.xlsx') .

 

see my learning notes attached. 

https://sites.google.com/d/19xjSh8teKmcvNz7VGnz2bMXu8bUNlW0C/p/1pMnHF6aTxMTOQUn3VvilruNBv6CoiE80/edi...

Helpful resources

Announcements
Users online (27)