Using Power BI APIs to get tenant overview

[FabScone75]

Hello, 

I'd appreciate advice on my process and whether I am going the right way here. 

The project is to get an overview of our tenant, who is doing what, which workspaces we have, what reports are being built and by whom etc. To that end, we have created a process to query the Power BI Rest API on a daily basis. This we do via a pipleline with a copy data activity, that creates a json file inside a folder structure of year/month/day (eg raw_api_groups/2025/09/28/20250928.json). This is my bronze layer.  My thought was to keep a 90 day history of these json files. 

For silver layer, I have a Dataflow Gen2 that opens the raw_api_groups and brings in all the json files. It combines these together, and then deduplicates based on the primary key (eg on workspaces, this would be workspace_id). I add in the source file (eg 20250928.json) as well as the ingestion_date as a column and then output this to replace the current dim_workspace table. 

Is this a good process? Any suggestions in terms of improvements? 

[tayloramy]

Hi @FabScone75 ,

Below is an approach I’ve used in real tenants for “who’s doing what / where” and how I’d tweak your Bronze > Silver flow to make it more reliable and easier to report on later.

1. Collect the right signals
   - Use the Power BI Admin “scanner” APIs for workspace/artifact inventory and schema (Metadata scanning (Scanner API)). Prefer GetModifiedWorkspaces for incremental scans.
   - Use Admin “activity events” for user actions (Get Activity Events) - this is the authoritative “who did what” source.
   - Resolve users/licensing with Microsoft Graph (optional), e.g. Users.
2. Keep raw forever (cheap) and normalize early
   - Keep your daily JSON in a date-path forever or compact to Parquet/Delta weekly.
   - In Silver, don’t throw away change history. Maintain an SCD Type 2 history table and a dim_workspace_current view/table.
3. Use incremental + watermarking
   - Stamp ingestion_date from the file path and use Dataflow incremental refresh.
   - For scanner APIs, use GetModifiedWorkspaces with your last successful scan time.
4. Harden the pipeline
   - Service principal auth; enable read-only admin APIs (Tenant settings).
   - Implement retries/backoff for 429/5xx; store API run metadata.
5. Model for analytics
   - Gold star schema: FactActivity plus DimUsers, DimWorkspaces, DimDatasets, DimReports for easy slicing.

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.

[v-sdhruv]

Hi @FabScone75 ,

I hope the information provided above assists you in resolving the issue. If you have any additional questions, please feel free to reach out.

Thank You

[v-sdhruv]

Hi @FabScone75 ,

I hope the information provided above assists you in resolving the issue. If you have any additional questions, please feel free to reach out.

Thank You

[BhaveshPatel]

Hi @FabScone75 

You are not using the right approach for Audit Log. Here I can see mess everywhere. 

First thing first

1. Use Power BI Dataflow Gen 2 to land the data in SQL database. 

2. Use Microsoft Purview for the unified audit log ( Before it was called Power BI Audit Log, now they are called Fabric Unified Audit Log in Microsoft Purview. You might need to contact your Fabric Admin for that. All activities are recorded "Who is doing what and which workspace are assigned..etc" in the Audit Log.

[v-sdhruv]

Hi @FabScone75 ,
Since we didnt hear back, we would be closing this thread.
If you need any assistance, feel free to reach out by creating a new post.

Thank you for using Microsoft Community Forum