Skip to main content
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Calling all Data Engineers! Fabric Data Engineer (Exam DP-700) live sessions are back! Starting October 16th. Sign up.

Reply
prathijp
New Contributor III

Parquet file to delta table using Notebooks: Issue with Datetime column

I have parquet file in lakehouse with the below columns. When I try to save the file as lakhouse table I am getting error with the date field. Please suggest if any solution available? Also date column is not available in sql endpoint once saved.

dataref = "/Files/Bronze/CT_Value.parquet"

datadf = spark.read.parquet(dataref)
datadf.write.mode("overwrite").format("delta").save("Tables/CT_Value")
 
Error: "Columns of the specified data types are not supported for (ColumnName: '[date] TIMESTAMP_NTZ')."
 
 
File is as below

prathijp_0-1748120610135.png

 

1 ACCEPTED SOLUTION
Ilgar_Zarbali
Contributor III

The error happens because the TIMESTAMP_NTZ type (timestamp without time zone) doesnโ€™t work with Lakehouse tables in Fabric when using Delta format.

To fix it, change the date column to a supported type like DATE or normal TIMESTAMP before saving.

If the column type isnโ€™t supported, it wonโ€™t show up in SQL after saving โ€” but converting it will fix that and make it visible.

View solution in original post

2 REPLIES 2
Ilgar_Zarbali
Contributor III

The error happens because the TIMESTAMP_NTZ type (timestamp without time zone) doesnโ€™t work with Lakehouse tables in Fabric when using Delta format.

To fix it, change the date column to a supported type like DATE or normal TIMESTAMP before saving.

If the column type isnโ€™t supported, it wonโ€™t show up in SQL after saving โ€” but converting it will fix that and make it visible.

v-prasare
Honored Contributor II

@prathijp, As we havenโ€™t heard back from you, we wanted to kindly follow up to check if the solution provided for your issue worked? or let us know if you need any further assistance here?

 

@Ilgar_Zarbali, Thanks for your prompt response.

 

 

Thanks,

Prashanth Are

MS Fabric community support

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

Helpful resources

Announcements
Users online (27)