Refresh Settings & Failure Management for Large Semantic Models in Microsoft Fabric (2026 Edition)

-

 Large semantic models—those exceeding 1 GB—require special handling in Microsoft Fabric and Power BI Premium due to their refresh cost, memory requirements, and impact on shared capacity. Refreshing these models safely is critical to operational stability, data reliability, and user experience.

This blog post provides a complete guide to refresh settings, refresh orchestration, and failure recovery for large models, supported by diagrams, best‑practice configurations, Fabric‑aligned techniques, and citations to Microsoft Learn and Fabric resources.

🔷 1. Why Large Models Need Special Refresh Handling

Large semantic models:

  • Consume significantly more memory during refresh (often 2.5× the model size).
  • Cause capacity evictions or query delays if unmanaged.
  • Require incremental refresh and partition-aware refresh to remain stable.
  • Require scale-out replicas for uninterrupted user experience during refresh.

Microsoft confirms that Import mode semantic models are the only ones that require refresh, and refresh operations depend heavily on model size and storage mode.

1

🔷 2. Fabric Refresh Architecture for Large Models

Fabric unifies OneLake, lakehouses, warehouses, pipelines, and semantic models into a single refresh ecosystem. The diagram below shows how refresh flows through components:

📊 Diagram 1 — Large Semantic Model Refresh Architecture

large_model_refresh_architecture.png

Key Components

OneLake Storage

Stores parquet/delta data for lakehouse/warehouse ingestion.
Fabric’s unified storage reduces the need for repeated movement.

2

Semantic Model (Large)

Configured with:

  • Large semantic model format enabled
  • Incremental Refresh
  • Optional Hybrid tables

Scale‑Out Replicas

Premium capacities support read‑only replicas; queries hit replicas while the write replica performs refresh.
Microsoft Learn highlights that scale‑out prevents refresh from hurting user performance. [CodeInterpreter | PowerPoint]

XMLA Endpoint (Read/Write)

Allows direct partition manipulation using TMSL/TOM.
Enables targeted partition refreshes not possible in the UI. [learn.microsoft.com]

Report Consumers

Always served by read-only replicas (scale-out) during refresh windows.

🔷 3. Required Refresh Settings for Large Models (Premium/Fabric)

✔ Enable Large Semantic Model Format

Required for any dataset exceeding 1 GB.
This makes partitions and metadata compatible with Fabric’s advanced refresh engine.
[learn.microsoft.com]

✔ Configure Incremental Refresh (Mandatory)

Incremental Refresh reduces memory and CPU cost by refreshing only recent data.
Only the refresh window partitions are processed; historical partitions remain untouched.

1

Recommended settings:

  • Store period: 5–7 years
  • Refresh period: 1–7 days
  • Detect Data Changes: Enabled (if source has LastModified)

✔ Enable Query Scale‑Out

Scale-out creates read-only replicas so users don’t experience slowdowns during refresh.
It supports auto-replicas using maxReadOnlyReplicas = -1.
[CodeInterpreter | PowerPoint]

Recommended:

maxReadOnlyReplicas = -1
autoSyncReadOnlyReplicas = true

✔ Allow XMLA Endpoint (Read/Write)

Allows:

  • Partition viewing/editing
  • Running TMSL incremental refresh commands
  • Programmatic control of refresh
  • Metadata-only deployments

Microsoft Learn confirms XMLA is the enterprise gateway to refresh automation.
[learn.microsoft.com]

🔷 4. How Refresh Occurs (Deep Dive)

📊 Diagram 2 — Refresh & Failure Management Flow

large_model_failure_flow.png

Step 1 — Scheduled Refresh

Triggered via Power BI Service scheduling or Fabric pipeline.

Step 2 — Credential/Gateway Checks

Many refresh failures stem from expired credentials.
Microsoft Learn stresses credential validity as a critical refresh dependency.

1

Step 3 — Partition Processing

  • Historical partitions untouched
  • Refresh partitions re-imported
  • XMLA can override partitions selectively

Step 4 — Replica Sync

Read-only replicas are synced automatically (unless manual sync configured).
[CodeInterpreter | PowerPoint]

Step 5 — Failure Handling

If any step fails, refresh aborts and an error is logged.

🔷 5. Common Failure Causes (Based on Microsoft Documentation)

1. Credential / Authentication failure

Most frequent cause of refresh errors.

1

2. Insufficient memory during refresh

Large models require up to 2.5× model size to refresh.
Scale-out helps by isolating read/write loads.

3. Query folding broken

Causes full refresh behavior → enormous cost.

4. Schema drift

Columns added/removed upstream in OneLake cause refresh to break.

5. Timeout from slow data sources

Mitigated by Dataflow IR or Warehouse ingestion.

🔷 6. Failure Recovery in Microsoft Fabric

✔ Pipeline Activities with Retries

Fabric Data Factory pipelines support activity-level retries (e.g., 2–3 attempts, 30–60s).
Useful for temporary outages or intermittent locks.
[learn.microsoft.com]

✔ XMLA to Refresh Only a Failed Partition

Instead of re-running the entire model, use TMSL:

JSON
{
"refresh": {
"type": "full",
"objects": [
{ "database": "ModelName",
"table": "FactTable",
"partition": "2026-03-10" }
]
}
}

✔ Manual or Scripted Replica Sync (advanced)

Only required if autoSyncReadOnlyReplicas = false.
[CodeInterpreter | PowerPoint]

🔷 7. Recommended Best Practices (2026 Fabric Edition)

Architectural

  • Prefer Direct Lake where feasible (no refresh needed).
  • Use Dataflows Gen2 for staged incremental extraction.
  • Use Warehouse MERGE for stable upsert logic.

Operational

  • Schedule refreshes at off-peak times (evening or early morning).
  • Monitor capacity metrics weekly.
  • Enable refresh notifications for BI Ops.
  • Use XMLA for partition-level targeting.

Governance

  • Review refresh history logs
  • Document Store/Refresh policies
  • Enable workspace-level monitoring in Fabric Admin

🔷 8. References

Comments

Post a Comment

Popular posts from this blog

Migrating from Power BI Premium to Microsoft Fabric: A Step-by-Step Guide

-
Image
Why This Matters? Microsoft has announced the retirement of Power BI Premium per capacity SKUs (P-SKUs) in favor of the Microsoft Fabric F-SKU capacity model as part of its unified analytics platform strategy. Fabric includes all Power BI capabilities plus additional workloads like data engineering, data warehousing, Synapse-style analytics, governance, and AI. If you currently run on Power BI Premium capacity and want to continue without disruption, you need to transition your content to Microsoft Fabric capacity before your next renewal or end of subscription. 📅 Key Licensing and Transition Details Before we dive into steps, here’s the high-level context you need: Purchase of Fabric capacity (F-SKUs) is done via Azure; P-SKUs are no longer purchasable after February 1, 2025. You can continue using your existing P-SKU until your current agreement expires . After the P-SKU ends, Microsoft gives 30 days of free Fabric capacity equal to your previous SKU size to tran...
Read more

Incremental Refresh in Dataflows Gen2 (2026 Edition)

-
Image
  Incremental Refresh (IR) in Power BI Dataflows Gen2 is a powerful, often misunderstood capability—especially now that Microsoft Fabric positions OneLake as the unified storage layer for ingestion, transformation, and analytics. This post explains how Dataflow Incremental Refresh works, how it differs from dataset IR, when to use it, and how to design it safely for enterprise‑grade workloads. 🔷 1. What Dataflow Incremental Refresh Is (and Isn’t) Dataflow IR performs incremental ingestion at the ETL layer. It controls what data is extracted from source systems and loaded into OneLake , not how semantic models partition their imported data. Key characteristics: ✔ Dataflow IR manages data ingestion , not semantic model partitions Dataflows apply Store/Refresh windows during data extraction and materialization. Unlike dataset IR, they do not create semantic model partitions. 1 ✔ Query folding is mandatory If folding breaks, IR is ignored and the entire dataflow refreshes fully. M...
Read more