Write data into synapse dedicated sql pool from fabric notebook

[SriThiru]

How to write data into synapse dedicated sql pool from fabric notebook. 

I want to save the spark dataframe from Fabric Notebook into Azure Synapse dedicated sql pool.

Is it doable, if yes, can anyone please share the code, along with any configuration settings if needed at the dedicated sql pool level.

[mabdollahi]

Hi @SriThiru ,

I see your question is remained as unanswered for a long time.

Yes, you can definitely write data from a Spark DataFrame in a Fabric Notebook to an Azure Synapse dedicated SQL pool.

You’ll just need to use the .write method with the jdbc format. Here's a simple example to get you started:

df.write \
  .format("jdbc") \
  .option("url", "jdbc:sqlserver://<your-server-name>.sql.azuresynapse.net:1433;database=<your-database-name>") \
  .option("dbtable", "<your-schema>.<your-table-name>") \
  .option("user", "<your-username>") \
  .option("password", "<your-password>") \
  .option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver") \
  .mode("append") \
  .save()

A few quick tips:

• Make sure your dedicated SQL pool allows Azure services to connect (you can set this in the firewall settings).

• The table you're writing to should already exist in Synapse with the right schema.

• If you're using managed identity instead of SQL auth, you’ll need to handle authentication a bit differently (happy to help with that if needed).

Regards,

Mehrdad Abdollahi