Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Calling all Data Engineers! Fabric Data Engineer (Exam DP-700) live sessions are back! Starting October 16th. Sign up.

Reply
CoulterJames2
New Contributor III

Shortcut to Azure SQL

I am working on a Fabric project for a customer. We need to bring data into OneLake from an existing Azure SQL database which contains a data warehouse made up of dimension and fact tables. We cannot use mirroring because the tables are reloaded in full during each ETL run using TRUNCATE statements (see known limitations here:https://learn.microsoft.com/en-us/fabric/database/mirrored-database/azure-sql-database-limitations)

 

I thought it would be possible to create shortcuts instead. But there is no option to create a shortcut to an Azure SQL Database. The only options available are ADLSGen2, OneLake, Dataverse, Amazon S3 or Google Cloud. I cannot believe that Azure SQL is not an option here. Is this on the roadmap at all? If not, it needs to be.

 

I am now faced with the prospect of having to create a Copy Data pipeline to duplicate the data I need from this source in my Fabric Onelake.

1 ACCEPTED SOLUTION
AndyDDC
Valued Contributor

Hi @CoulterJames2 there's currently nothing on the roadmap for shortcuts into Azure SQL DB unfortunately. There's nothing logged on the Fabric ideas site so you could add an idea https://ideas.fabric.microsoft.com

 

https://learn.microsoft.com/en-gb/fabric/release-plan/onelake

 

Whats the volume of data in the Azure SQL DB? You should get pretty good throughput loading data into onelake from Azure SQL.  You could look at using a Notebook to import and write data to a lakehouse instead of data pipelines (you'll need to see what the CU impact is with pipelines vs notebook)

 

https://learn.microsoft.com/en-gb/training/modules/ingest-data-with-spark-fabric-notebooks/

 

-----------------------------------------------------------

If my reply has been helpful please consider providing 

kudos and marking as the solution

-----------------------------------------------------------

View solution in original post

3 REPLIES 3
AndyDDC
Valued Contributor

Hi @CoulterJames2 there's currently nothing on the roadmap for shortcuts into Azure SQL DB unfortunately. There's nothing logged on the Fabric ideas site so you could add an idea https://ideas.fabric.microsoft.com

 

https://learn.microsoft.com/en-gb/fabric/release-plan/onelake

 

Whats the volume of data in the Azure SQL DB? You should get pretty good throughput loading data into onelake from Azure SQL.  You could look at using a Notebook to import and write data to a lakehouse instead of data pipelines (you'll need to see what the CU impact is with pipelines vs notebook)

 

https://learn.microsoft.com/en-gb/training/modules/ingest-data-with-spark-fabric-notebooks/

 

-----------------------------------------------------------

If my reply has been helpful please consider providing 

kudos and marking as the solution

-----------------------------------------------------------

CoulterJames2
New Contributor III

I've logged an feature request - I can't believe this isn't an option.

Well, shortcuts are API calls really into supported storage services. If you could create a shortcut to an azure sql db, what are you shortcutting to? Database? Table? What runs the compute required to retrieve data?  Shortcutting to rdbms is more complex than storage services IMHO

Helpful resources

Announcements
Top Solution Authors
Users online (2,585)