InvalidRequest when adding "Private Link Service" as a "Managed Private Endpoint"

[Digidank]

I get a simple error saying "InvalidRequest" when i try to add my Private Link Service as a Managed Private Endpoint in the Workspace Setttings Network Security page.  I can add others fine like a PE for my Azure SQL server.  It just seems to be an issue with "Private Link Service" specifically.  Is this possibly a bug with adding PLS as a Managed Private Endpoint?

[Anonymous]

Hi @Digidank,

What type of capacity are you worked? Trial or fabric capacity? Which region are you host to configure the private link service? Please share some more detail information about these.

BTW, the document mentioned some of regions not support PLS feature, perhaps you can take a look at the following link to know more about these:

Overview of managed private endpoints for Microsoft Fabric - Microsoft Fabric | Microsoft Learn

Regards,

Xiaoxin Sheng

[Digidank]

I read that doc before and according to it I thought it should work.  I'm in US East2 and tried F2 and F4 capacity.  The PLS and capacity are both in US East2.

According to this it says Private Link Service is an option for Managed Private Endpoint https://learn.microsoft.com/en-us/fabric/security/security-managed-private-endpoints-create#supporte....

I can create managed pe's for Azure SQL fine.  Appears to just be a Private Link Service that gets the error.

I have a PLS configured to a load balancer that routes sql traffic to an onprem sql server through a VM.  The PLS works fine in a VM and onprem servers.  If this won't work do you suggest anything else I could try to access my onprem sql server in spark?  I don't want to use a Pipeline.  I already have that working fine with the onprem data gateway.  I'm wanting to do this in Spark specifically.  Creating a PE to fabric is not an option at the moment due to some of the limitations it defined here: https://learn.microsoft.com/en-us/fabric/security/security-private-links-overview#other-consideratio....

[Anonymous]

HI @Digidank,

I check the document of PLS limitations part and find it include one scenario for fabric users:

For Fabric users: On-premises data gateways aren't supported and fail to register when Private Link is enabled. To run the gateway configurator successfully, Private Link must be disabled. Learn more about this scenario. VNet data gateways will work. For more information, see these considerations.

If that is the case, it means the private link service will conflict with on-premises data gateway usages.(based on your description, you already use this on-premise data gateway mapping an on premise source to the azure SQL)

Regards,

Xiaoxin Sheng

[Digidank]

I read that doc and understand the limitations of "Private Link" into Fabric.  The limitation you mentioned along with others is the reason we cannot enable the Private Link into Fabric and it must remain Public.  But, that is not a PLS (Private Link Service).

I think there is some confusion between "Private Link" for Fabric and using a "Private Link Service" as a datasource for a "Managed Private Endpoint".  

I am not trying to enable "Private Link" for INBOUND access to Fabric privately.  That is documented here: https://learn.microsoft.com/en-us/fabric/security/security-private-links-overview and what the tenant admin setting for "Private Link" is for.   I am referring to OUTBOUND using the actual Azure Resource/Service called "Private Link Service" as a datasource for a "Managed Private Endpoint".  

I can create a "Managed Private Endpoint" perfectly fine for a private Azure SQL server and already am using it in a spark job.  It works great.  It is solely adding the specific azure resource called "Private Link Service" that i have a problem doing.  “Private Link Service” is not the same thing as "Private Link" in Fabric settings as they are 2 completely different things.  "Private Link Service" is really just a service that leverages a load balancer and private link capabilities for private access similar to how Azure SQL, Cosmos, etc do.

[Digidank]

Yeah, I read that in the documentation as well.  That was why i was saying we cannot use it. 

However, I think there is some confusion between "Private Link" for Fabric and using a "Private Link Service" as a datasource for a "Managed Private Endpoint".  

I am not trying to enable "Private Link" for INBOUND access to Fabric privately.  That is documented here: https://learn.microsoft.com/en-us/fabric/security/security-private-links-overview and what the tenant admin setting for "Private Link" is for.   I am referring to OUTBOUND using the Azure resource called "Private Link Service" as a datasource for a "Managed Private Endpoint" documented here: https://learn.microsoft.com/en-us/fabric/security/security-managed-private-endpoints-create.  

I can create "Managed Private Endpoint" perfectly fine for Azure SQL and already am using it in a spark job.  It works great.  It is solely adding the specific azure resource called "Private Link Service".  "Private Link Service" is not the same thing as "Private Link" in Fabric settings as they are 2 completely different things.

[msundman78]

I'm having the exact same problem.

I've created a PLS in region Sweden, but get InvalidRequest when trying to add it as Managed Private endpoint in Fabric. I've tried both with my Trial capacity in region North Europe, and with a WS connected to a F2 capacity in region Sweden.

Also tried changing the Visibility on the PLS to "Anyone with your alias". But still same error no matter.

[Digidank]

Sounds like we are both struggling with the same thing. My ongoing support ticket with PowerBI/Fabric support keeps saying that "Private Link" in admin tenant settings needs to be enabled, which is for enabling private access into Fabric. But they also keep referring me to documentation about creating a PL into Fabric, which is not what I am trying to do.  So, I really do not think that is the issue.  Seems like support cannot differentiate PL for Fabric vs a PLS.

Are you able to try enabling "Private Link" in your Tenant Admin settings?  Our PowerBI admins will not enable it because some of the limitations defined here: https://learn.microsoft.com/en-us/fabric/security/security-private-links-overview#other-consideratio...

I am going to try on my personal Azure account later today when i get a chance.  I just want to rule that out.

[msundman78]

I've tried now to enable Private Link under Tenant Admin settings, waited 30 min, but still get the same error.

[Digidank]

Thanks for trying!  I too tried on my personal tenant with the exact same result.  It cannot be the setting.  I have forwarded this thread along with my personal tenant requestId to see if they can troubleshoot further.  I will be sure to update you if i get anywhere. 

[Digidank · Accepted solution]

This is extremely frustrating.  Here is the support response:

Hello Josh,

I hope you are having an amazing day.

We have shared the issue with all collected details in our SME channel. They responded that currently, private endpoints targeting Private Link Service are not supported. Additionally, we do not support creating managed private endpoints with FQDNs at this time and there is no ETA. The product team will add this limitation to public documents once they have an ETA.

I am pleased to inform you that there is no issue from your side; however, the actual feature is not supported currently. We have already created an incident with the product team, and they are working on updating the feature.

I apologize for any inconvenience this may cause. Rest assured, you will be able to access this feature soon.

Please feel free to reach out to me if you have any questions, concerns, or additional information that you would like to share. I'll be more than happy to assist you.

Thank you for your understanding.

[Digidank]

Most recent update:

Hello Josh,

Hope you are doing well!

The Product team has informed us that currently, there is no estimated time of arrival (ETA) for updating of the Document. There's a possibility we might receive an update on the ETA by the end of this month, but this is a provisional internal timeline and not guaranteed. Once the ETA is established, the public documentation will be updated accordingly.

Please let me know if you have any other concerns. Based on your response we will proceed with the case.

[msundman78]

Thanks for the update, Josh.

Do they mean that they might get back with an ETA when documentation is updated to clearly say PLS is *not* supported, or do they mean they might get back with an ETA when it *will be* supported?

I've also opened a support ticket, which first came back with the same suggestion to enable Private Link ect. It's now been escalated to "next level", but I assume they will get back with same info as you got, but hopefully it can help raise the priority on the issue if more people report it as a major showstopper. 

[Digidank]

From my understanding this is entirely about an ETA updating the documentation and nothing to do with an ETA to actaully solve it.  I am responding asking for that clarification though.

I hope you get further than I have!  Let me know if you get any better news please.

[msundman78]

After a long discussion with support I finally got the exact same answer as you, that it is currently not supported, and there is no ETA when it will be available.

So, I can just conclude that Fabric is not production ready yet for customers that needs connectivity from Spark notebooks to on-prem services/systems, which is a pity.

Our current work-around is to use local python scripts executing on the on-prem data-gateway server that produces CSV files in a local folder that can then copied to OneLake via a Fabric Pipeline, and later processed by Fabric Spark Notebooks.

[Digidank]

Yeah it is very unfortunate.  I have been using a small compute cluster in Databricks to write as a delta table to an ADLSv2 storage account.  Then created a shortcut in Fabric to read it.  Was my fastest solution so far.  It was my first time working with Databricks and it was extremely simple to get running with onprem access.  Hopefully the Fabric team gets this sorted out soon.

[dbeavon3]

@Digidank 

Do you have a ticket open?  DId they update the docs to say that PLS is not supported?

The docs are pretty poor.  The only relevant thing I can find is like so:

https://learn.microsoft.com/en-us/fabric/security/security-managed-private-endpoints-overview#limita...

Creating a managed private endpoint with a fully qualified domain name (FQDN) is not supported.

These limitations and considerations might affect your use cases and workflows. Take them into account

This doesn't specifically talk about PLS, but the FQDN part is often a show-stopper for PLS.

[dbeavon3]

I found a full list of supported connections via MPE, and private-link service is not one of them:

https://learn.microsoft.com/en-us/fabric/security/security-managed-private-endpoints-create#supporte...

I can say from my past experience (in Synapse) that it took far longer then I thought it would to add support for MPE's to private link service.  I think we were talking to Microsoft for almost 18 months before they finally added MPE's to PLS.  The additional complexity (compared to other types of data sources) is very minimal, eg. the ability to specify FQDN's.    I think the folks who are the biggest bottlenecks are the ones writing the U/I interactions.  Perhaps we should be asking for a REST API to manage our MPE's and that might allow us to get to the finish line  a lot sooner, without placing a dependency on any of the front-end web developers.

[dbeavon3]

Not sure if anyone else has tried to create a PLS private endpoint by going around the back door.  But it doesn't work, from what I've tried.   Image:

I've been able to create PE requests for blob and other resources, but it never succeeds when I try to create one for PLS, with a blank group ID and a FQDN.  I think we are at the mercy of whatever team is sitting on this request. 
IMO, It seems like a pretty basic requirement.  Private link service is a pretty fundamental component in azure nowadays, and was intended to be used for precisely this sort of scenario.

Again, it took the Synapse team about a year to create the U/I that allows this PLS connectivity to work, and I am guessing the Fabric team will take the same a similar amount of time.  It would be nice if they could give a "back-door" approach that circumvents the U/I, as a temporary solution.  For some reason U/I teams in the cloud seem to be extremely slow at what they do. Back-door options include a REST api, or powershell command or similar.

[dbeavon3]

Wahted to re-iterate that a pipeline activity "copy activity" might be the best way to work around the missing PLS.  Ie. you have to go thru a round-about approach that sends the data from a service thru an ADLS storage account (temp files) before it is accessible from Fabric.

This unfortunately can cost more money, and has more moving parts (on premise gateway).

More details on reddit:
https://www.reddit.com/r/MicrosoftFabric/comments/1er2z51/invalidrequest_when_adding_a_private_link_...

[dbeavon3]

I was able to get in touch with a Mindtree engineer about this today (12/17/2024):

I have an update for you from my PG, they informed me that as of now there is no ETA for this deployment as currently there is a deployment freeze. And they have provided below platforms where we can check for the latest update directly from the product team.

And I also checked for workaround but they informed me that there is not workaround and PG team is working on it to bring it as soon as possible.

https://blog.fabric.microsoft.com/en-us/blog/
https://blog.fabric.microsoft.com/en-us/blog/category/roadmap

As a side,  I looked at the roadmap and it goes to the end of Q1 2025.  It is reasonable to assume they are NOT committed to fixing the PLS private endpoints before the end of Q1.  Unfortunately we are already planning to get workloads out of Synapse before then... so we must stumble our way thru this in one way or another.  I'm guessing we will have to retrieve this service-hosted data by way of data-factory-pipelines or some other wierdness.  I guess this is par for the course, when building a solution inside of a SaaS. 

[Anonymous]

Hi @Digidank,

Thanks for sharing these detail information here. I think they will help other user to clarify this scenario.

Regards,

Xiaoxin Sheng

[mikecmpbll]

I also assumed that this would work, and I see no way of connecting Fabric to our VNet integrated Azure MySQL without it.

I've created an Ideas submission here, please upvote it! https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=a2c6eb27-727a-ef11-a4e6-000d3a7b101f

[dbeavon3]

Thank you so much for posting.  I was probably about to waste a month of time with support as well.  Any chance you have a case identifier, like an SR number or ICM number?  I may want to follow-up but don't necessarily want to start from scratch.

[msundman78]

I can also confirm that just as you @Digidank , I have no problem creating Private Endpoints in Fabric to other Azure services. Just successfully added one to a storage account without problem.

I've also tried to create a spark pool in Synapse in the same Azure tenant as my PLS, and from Synapse, I can successfully connect to my PLS (even manually, using the exact same Resource ID as I'm trying to use from Fabric.

All my attempts from Fabric returns HTTP error 400 (InvalidRequest, pbi.error), regardless if I misstype the servicename or enter the correct servicename.

[Anonymous]

HI @Digidank ,

Sorry it seems like I confused the Azure private link service and private links. For your scenario, perhaps you can open a ticket on support page to contact the Dev team to ensure if managed Private Endpoint supports azure private link service.

Microsoft Fabric Support and Status | Microsoft Fabric

Regards,

Xiaoxin Sheng

[Digidank]

No worries @Anonymous!  I have an open ticket i've been going back and forth on.  Will be doing a teams/screensharing call today with support.  I will be sure to keep updating this as i get more info.

[kthr]

I have the same issue. When entering the details in Fabric->Workspace settings->Network Security it passes all input validation. But when I click create, I get "InvalidRequest". I know the target resource is working because I can create a private link from Azure and that works just fine - the request is sent and can be approved just fine. But that doesn't help me connect Fabric to the managed private endpoint. 

[msundman78]

Are you also trying to create a PE linked to a "Private Link Service", or what backend service type are you trying to connect to?

[kthr]

Sorry for the delay. Yea, the resource type is an Azure "Private Link Service" that I'm trying to connect to with Fabric's private endpoint.