Introduction to Snowflake Architecture

Snowflake’s architecture is fundamentally different from traditional data warehouses. It uses a hybrid architecture that combines the benefits of both shared-disk and shared-nothing architectures.

Three-Layer Architecture

Snowflake’s architecture consists of three distinct layers:

1. Database Storage Layer
2. Query Processing Layer (Virtual Warehouses)
3. Cloud Services Layer

Snowflake Three-Layer Architecture

Visual representation of how Snowflake separates storage, compute, and services into independent, scalable layers

1. Database Storage Layer

The storage layer is where all your data lives. Snowflake automatically manages:

• Compression: Data is automatically compressed
• Organisation: Micro-partitions and metadata
• Optimisation: Columnar storage format

Micro-Partition Structure

Illustration of how Snowflake automatically organises table data into immutable micro-partitions with columnar storage

Key Concepts

Micro-Partitions:

• Automatically created and sized by Snowflake
• Typically 50-500 MB compressed
• Immutable (never updated in place)
• Columnar format for efficient scanning

Metadata:

• Min/max values per column
• Number of distinct values
• Additional optimisation statistics
• Enables automatic query pruning

2. Query Processing Layer (Virtual Warehouses)

Virtual Warehouses are independent compute clusters that execute queries.

Virtual Warehouse Characteristics

• Elastic: Scale up/down instantly
• Isolated: No resource contention between warehouses
• MPP Clusters: Massively Parallel Processing
• Auto-suspend/resume: Cost optimisation

Virtual Warehouse Scaling

Visualisation of how virtual warehouses can scale up (larger size) or scale out (more clusters) independently

1-- Create a virtual warehouse
2CREATE WAREHOUSE ANALYTICS_WH
3WITH WAREHOUSE_SIZE = 'LARGE'
4AUTO_SUSPEND = 300
5AUTO_RESUME = TRUE
6INITIALLY_SUSPENDED = TRUE;
7
8-- Resize warehouse on-the-fly
9ALTER WAREHOUSE ANALYTICS_WH SET WAREHOUSE_SIZE = 'X-LARGE';
10
11-- Start the warehouse
12ALTER WAREHOUSE ANALYTICS_WH RESUME;
13
14-- Suspend the warehouse
15ALTER WAREHOUSE ANALYTICS_WH SUSPEND;

Warehouse Sizes

3. Cloud Services Layer

The brain of Snowflake - coordinates all activities across the platform:

• Authentication: User login and security
• Infrastructure Management: Optimisation and monitoring
• Metadata Management: Query parsing and optimisation
• Query Parsing & Optimisation: Execution planning
• Access Control: Role-based permissions

Complete Architecture Stack

End-to-end view showing how Cloud Services orchestrate compute and storage layers

Key Services

┌─────────────────────────────────────────┐
│       Cloud Services Layer              │
├─────────────────────────────────────────┤
│  • Authentication & Access Control      │
│  • Query Compilation & Optimisation     │
│  • Transaction Management               │
│  • Metadata Management                  │
│  • Security & Encryption                │
└─────────────────────────────────────────┘
              ↓
┌─────────────────────────────────────────┐
│    Virtual Warehouses (Compute)         │
│  [WH1]  [WH2]  [WH3]  ...  [WHn]        │
└─────────────────────────────────────────┘
              ↓
┌─────────────────────────────────────────┐
│    Database Storage (Data)              │
│  Tables | Stages | Micro-partitions     │
└─────────────────────────────────────────┘

Data Sharing Architecture

Snowflake enables secure data sharing without data copying:

• Live Access: Consumers query provider’s data directly
• Zero-Copy: No data movement required
• Real-Time: Always up-to-date data
• Granular Control: Share specific objects only

Data Sharing Architecture

How data providers share live data with consumers without physical copying, using metadata and access grants

1-- Create a share
2CREATE SHARE SALES_SHARE;
3
4-- Grant usage on database
5GRANT USAGE ON DATABASE SALES_DB TO SHARE SALES_SHARE;
6
7-- Grant usage on schema
8GRANT USAGE ON SCHEMA SALES_DB.PUBLIC TO SHARE SALES_SHARE;
9
10-- Grant select on specific table
11GRANT SELECT ON TABLE SALES_DB.PUBLIC.ORDERS TO SHARE SALES_SHARE;
12
13-- Add account to share
14ALTER SHARE SALES_SHARE ADD ACCOUNTS = xy12345;

Multi-Cluster Warehouses

For high concurrency scenarios, Snowflake offers Multi-cluster Warehouses:

• Auto-scaling: Add clusters based on load
• Maximized Mode: All clusters run simultaneously
• Auto-scale Mode: Clusters scale based on demand
• Concurrent Query Handling: Distributes queries across clusters

1CREATE WAREHOUSE PRODUCTION_WH
2WITH WAREHOUSE_SIZE = 'LARGE'
3MIN_CLUSTER_COUNT = 1
4MAX_CLUSTER_COUNT = 5
5SCALING_POLICY = 'STANDARD'
6AUTO_SUSPEND = 300
7AUTO_RESUME = TRUE;

Key Architecture Benefits

1. Separation of Storage and Compute

   • Scale independently
   • Pay for what you use
   • Multiple warehouses access same data

2. Automatic Optimisation

   • No indexes to manage
   • No partitioning required
   • Automatic statistics

3. Data Sharing Without Copying

   • Secure and controlled
   • Real-time updates
   • No ETL required

4. Zero Maintenance

   • No infrastructure management
   • Automatic updates
   • Built-in disaster recovery

Architecture Benefits Summary

Visual summary comparing traditional data warehouses with Snowflake's modern architecture advantages

Practice Questions

Additional Resources

Official Snowflake Documentation

• Architecture Overview
• Virtual Warehouses
• Micro-partitions & Data Clustering

Recommended Reading

• Snowflake Architecture Whitepaper
• Best Practices Guide for COF-C02
• Performance Optimisation Techniques

Next Steps

Now that you understand Snowflake’s architecture, continue to:

• Virtual Warehouses Deep Dive
• Databases, Schemas & Tables
• Performance Optimisation