Processing JSON Data with Azure Functions and Storing in Microsoft Fabric Lakehouse

[VinceHe]

Hello everyone,

I’m receiving JSON data via a POST request and would like to process it using an Azure Function, which is triggered via an HTTP request, so that I can store the data in my Lakehouse in Microsoft Fabric.

• Which approach makes the most sense here?
  Should the Azure Function read the body of the POST request and pass it to a pipeline in Fabric, which would then process the JSON structure and append it to an existing table?
• Or, should the Azure Function store the JSON as a file in the Lakehouse, and then trigger a pipeline once a day to process the newly added JSON files?

I think both methods could work, but I'm not sure how to create the Azure Function to receive the POST request via the HTTP trigger and hand the data off to Fabric.

Does anyone have tips on how to approach this?

[v-nmadadi-msft · Accepted solution]

Hi @VinceHe,
Thanks for reaching out to the Microsoft fabric community forum.

To determine the best approach, it's essential to consider the pipeline's objective. If the pipeline involves transforming the JSON structure, the first approach would be the preferred choice.

With the second approach, the pipeline is triggered once and processes all uploaded JSON structures in a batch. In contrast, the first approach enables real-time updates as data is received.

Please choose the approach that best aligns with your requirements, considering the points mentioned above.

Attached below is the Functions activity in pipeline reference document:
 Functions activity - Microsoft Fabric | Microsoft Learn
Hoping this document will help you to build any of the approach you will decide upon.

If you find this post helpful, please mark it as an "Accept as Solution" and consider giving a KUDOS.
Thanks and Regards

[v-nmadadi-msft]

Hi @VinceHe,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.