Creating a table template for a lakehouse

[pmscorca]

Hi,

I'm trying to create a reusable table template to create an empty table on a lakehouse using a Spark notebook.

I'm trying this pyspark code:

from pyspark.sql.types import StructType, StructField, StringType, IntegerType, DateType, TimestampType
#from pyspark.sql.types import VarcharType, CharType
from datetime import datetime

#table template
schema = StructType([
    StructField("code", StringType(), True),  # string field
    StructField("description", StringType(), True),  # string field
    StructField("revenue", IntegerType(), True),  # integer field
    StructField("insert_date", DateType(), True),  # date field
    StructField("insert_datetime", TimestampType(), True)  # datetime field
])

If possible, I'd like to use also the VarcharType() and the CharType, specifying a dimension. It should be useful specifying a dimension also for the IntegerType(). I've accomplish some proofs and it seems that the VarcharType() and the CharType() with a specified dimensione aren't supported.

Any suggests to me to complete this template? Thanks

[frithjof_v · Accepted solution]

This article may be relevant for this topic: https://blog.gbrueckl.at/2024/01/using-varchar-in-microsoft-fabric-lakehouses-and-sql-endpoints/

[pmscorca]

Hi, thanks your suggest is good and interesting.

I'd like to use pyspark to solve this issue.