This guide helps you choose the right storage and compute settings for Microsoft Fabric workspaces, balancing cost and performance for your data workloads. You’ll learn how to configure capacity, manage storage, and optimize compute for your team’s needs.
What You’ll Need
- Access to a Microsoft Fabric tenant with admin permissions (or at least capacity admin rights).
- A basic understanding of your data workloads (e.g., data engineering, data science, real-time analytics).
- Familiarity with the Fabric portal and capacity management settings.
Step-by-Step
- Assess your workload types. Identify whether your team primarily runs data pipelines (Data Factory), uses notebooks (Data Engineering/Science), or builds reports (Power BI). Each workload consumes compute differently.
- Choose a capacity SKU. In the Fabric admin portal, navigate to Capacity settings and select a capacity tier (e.g., F2, F4, F8, F16, F32, F64, F128, F256, F512, F1024). Higher SKUs provide more compute and memory. Start with a smaller SKU and scale up as needed.
- Enable autoscale (if available). For capacities with autoscale, set a maximum SKU size. This allows Fabric to automatically increase compute during peak loads and decrease it during idle times, helping manage costs.
- Configure storage settings. In the workspace settings, go to Storage and decide whether to use the default Fabric storage (OneLake) or connect an external Azure Data Lake Storage (ADLS) Gen2 account. For most scenarios, OneLake is recommended for simplicity and integration.
- Set up data residency (optional). If your organization requires data to reside in a specific geographic region, choose the appropriate region when creating the capacity. This affects both storage and compute location.
- Assign workspaces to the capacity. In the workspace settings, under License mode, select Fabric capacity and choose the capacity you created. This ensures the workspace uses the capacity’s compute and storage.
- Monitor usage and performance. Use the Fabric Capacity Metrics app to track compute consumption (CU) and storage usage. This helps identify bottlenecks and underutilized resources.
- Adjust compute settings per workload. For specific items like Spark jobs or SQL endpoints, you can set compute concurrency limits and session timeouts. For example, in a Spark notebook, set the session size (e.g., Small, Medium, Large) based on data size.
- Review and optimize regularly. Reassess your capacity SKU monthly. If you consistently use less than 50% of your capacity, consider downsizing; if you hit throttling, scale up.
Common Mistakes
- Over-provisioning compute. Choosing a large SKU (e.g., F64) for a small team with light workloads wastes money. Start small and scale.
- Ignoring autoscale. Not enabling autoscale can lead to either throttling during spikes or paying for idle capacity.
- Mixing capacities. Assigning all workspaces to a single capacity can cause resource contention. Group workspaces with similar performance needs.
- Forgetting about storage limits. Each capacity includes a certain amount of storage (e.g., F2 includes 2 TB). Exceeding it incurs extra costs, so monitor storage usage.
- Not using OneLake. Connecting external ADLS adds complexity and may reduce performance due to network latency. Use OneLake unless you have a specific reason.
Tips
- Use the Fabric Capacity Metrics app to see which items consume the most compute. This helps you optimize queries and pipelines.
- Leverage autoscale with a maximum SKU that is one or two tiers above your base. This gives headroom for occasional bursts without permanent cost.
- For development workspaces, use a smaller capacity (e.g., F2) and a separate production capacity (e.g., F32) to keep costs down.
- Consider using Spark pools with autoscale for data engineering jobs to adjust executor count based on workload.
- Set up budget alerts in Azure Cost Management to get notified when costs exceed a threshold.
Troubleshooting
- Throttling errors (429) occur. This means your capacity is overloaded. Reduce concurrency, optimize queries, or scale up the SKU.
- Workspace not appearing in capacity assignment. Ensure you have the correct permissions (Workspace Admin) and that the capacity is active. Refresh the page.
- Data not appearing in OneLake. Check if the workspace is assigned to a capacity. Without a capacity, OneLake is not available.
- Autoscale not working. Verify that autoscale is enabled in capacity settings and that your capacity SKU supports it (F2 and above).
- Slow performance on Power BI reports. Increase the capacity SKU or optimize the data model (e.g., reduce cardinality, use aggregations).
Frequently Asked Questions
What is the difference between storage and compute in Fabric?
Storage refers to where your data resides (OneLake or ADLS), while compute is the processing power (CPU/memory) used to run queries, pipelines, and notebooks. Both are billed under the same capacity SKU.
Can I change the capacity SKU later?
Yes, you can upgrade or downgrade your capacity SKU at any time from the admin portal. Changes take effect immediately, but billing is pro-rated.
How much storage does a Fabric capacity include?
Each SKU includes a base storage amount (e.g., F2 includes 2 TB, F64 includes 64 TB). Additional storage is billed per GB per month.
What is autoscale and how does it work?
Autoscale automatically adjusts the capacity SKU within a range you specify (e.g., F8 to F32) based on demand. It prevents throttling and reduces cost during idle periods.
Can I use my own Azure Data Lake Storage with Fabric?
Yes, you can attach an ADLS Gen2 account to a workspace, but it must be in the same region. This is useful for existing data lakes, but OneLake is simpler for most cases.
How do I monitor compute usage?
Use the Fabric Capacity Metrics app in the admin portal. It shows Compute Unit (CU) consumption per item and helps identify bottlenecks.
Related
General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.
More in this bed
- Add Transit Pass to Apple Wallet: Step-by-Step
- Safely Remove Hardware Icon Missing? Fix It
- How to Fix Excel Document Not Saved Error
- Mouse Scroll Speed Too Fast or Slow in Windows 11
- Disable Automatic Driver Installation Windows 11
- Samsung Galaxy Notifications Delayed or Missing? Data Saver Fixes