Write df to DataWarehouse tables

[Anonymous]

can someone show me how to write a dataframe df to a table in a Warehouse in Fabric?

I try below code, but having issue... I suppose there is no username or password involved here:

from pyspark.sql import SparkSession

# Initialize Spark Session

spark = SparkSession.builder.appName("FabricDW").getOrCreate()

# Sample DataFrame

data = [("Alice", 30, "Engineer"),

        ("Bob", 35, "Analyst"),

        ("Charlie", 40, "Manager")]

columns = ["Name", "Age", "JobTitle"]

df = spark.createDataFrame(data, columns)

# JDBC Connection using Interactive Authentication

dw_url = "jdbc:sqlserver://testconnectionstring.datawarehouse.fabric.microsoft.com:1433;" \

         "database=WideWorldImportersDW;encrypt=true"

dw_properties = {

    "driver": "com.microsoft.sqlserver.jdbc.SQLServerDriver"

}

# Write DataFrame to Data Warehouse Table

df.write.jdbc(url=dw_url, table="dbo.Employee", mode="overwrite", properties=dw_properties)

print("✅ Data written successfully to Fabric Data Warehouse!")

Py4JJavaError: An error occurred while calling o6216.jdbc. : com.microsoft.sqlserver.jdbc.SQLServerException: Could not login because the authentication failed. ClientConnectionId:6797891d-d522-4841-9792-5c16025f39b1 at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:259)

[nilendraFabric · Accepted solution]

hello @Anonymous 

Prepare the JDBC URL with the correct endpoint, database name, encryption, and authentication

dw_url = (“jdbc:sqlserver://<your_server_identifier>.datawarehouse.fabric.microsoft.com:1433;”
“DatabaseName=WideWorldImportersDW;”
“encrypt=true;”
“Authentication=ActiveDirectoryPassword”)

Provide authentication details in the connection properties
dw_properties = {
“driver”: “com.microsoft.sqlserver.jdbc.SQLServerDriver”,
“user”: “your_username@yourdomain.com”,“password”: “your_password” 
}

Write the DataFrame to the target table in the Data Warehouse
df.write.jdbc(url=dw_url, table=“dbo.Employee”, mode=“overwrite”, properties=dw_properties)

[v-hashadapu]

Hi @Anonymous , Thank you for reaching out to the Microsoft Community Forum.

Please let us know if your issue is solved. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.

specail thanks to @nilendraFabric for your prompt reply

[v-hashadapu]

Hi @Anonymous , Thank you for reaching out to the Microsoft Community Forum.

Please let us know if your issue is solved. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.

[v-hashadapu]

Hi @tanthiamhuat88 , Thank you for reaching out to the Microsoft Community Forum.

Please let us know if your issue is solved. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.