Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Calling all Data Engineers! Fabric Data Engineer (Exam DP-700) live sessions are back! Starting October 16th. Sign up.

Reply
BenLyon
New Contributor II

Get data from a API with a copy data

Hello 

 

I need to get data from this api :
https://ads.atmosphere.copernicus.eu

to my lakehouse. I need either a file or a table.
I tried so many things !

If someone has an idea ?

 

Thank 

1 ACCEPTED SOLUTION
spencer_sa
Contributor III

Update:

This code works and successfully downloads a zip file into a lakehouse.

url='https://ads.atmosphere.copernicus.eu/api'
key = 'Your API key'
import cdsapi
dataset = "cams-gridded-solar-radiation"
request = {
    "variable": ["direct_normal_irradiation"],
    "sky_type": ["clear"],
    "version": ["4.6"],
    "year": ["2023"],
    "month": ["01"]
}
client = cdsapi.Client(url = url, key = key)
a = client.retrieve(dataset, request,'/lakehouse/default/Files/Data/file.zip')

 

If this helps, please consider Accepting as a Solution to help other people to find it.

View solution in original post

3 REPLIES 3
spencer_sa
Contributor III

Having looked at the API spec and the cdsapi python library source code...
... using a python notebook in the first instance.
https://ads.atmosphere.copernicus.eu/how-to-api

https://github.com/ecmwf/cdsapi/blob/master/cdsapi/api.py

Replace the line in the example code that looks like;

client = cdsapi.Client()

with the following so it doesn't need to read from the config file;

 

 

client = cdsapi.Client(url='https://ads.atmosphere.copernicus.eu/api', key='<API key>')

 

 



spencer_sa
Contributor III

Update:

This code works and successfully downloads a zip file into a lakehouse.

url='https://ads.atmosphere.copernicus.eu/api'
key = 'Your API key'
import cdsapi
dataset = "cams-gridded-solar-radiation"
request = {
    "variable": ["direct_normal_irradiation"],
    "sky_type": ["clear"],
    "version": ["4.6"],
    "year": ["2023"],
    "month": ["01"]
}
client = cdsapi.Client(url = url, key = key)
a = client.retrieve(dataset, request,'/lakehouse/default/Files/Data/file.zip')

 

If this helps, please consider Accepting as a Solution to help other people to find it.

nitinmehta2879
New Contributor III

Try using copy data activity under Data Pipeline. Select Data store type as external, connection as Rest API, connection type=Rest. In destination select lakehouse and then table or file and provide name of table/file.

Helpful resources

Announcements
Users online (27)