# Core Concepts

- [Warehouses](/core-concepts/warehouses.md): Snowflake virtual warehouses configured for DataPancake scanning operations, with specific requirements for Snowpark-Optimized warehouses and concurrency settings.
- [Data Sources](/core-concepts/data-sources.md): Introduction to data sources and navigation hub for all data source documentation.
- [Adding Data Sources](/core-concepts/data-sources/adding-data-sources.md)
- [Baseline Scan Settings](/core-concepts/data-sources/baseline-scan-settings.md)
- [Basic Configuration Settings](/core-concepts/data-sources/basic-configuration-settings.md)
- [Data Source Types](/core-concepts/data-sources/data-source-types.md)
- [Dynamic Table Settings](/core-concepts/data-sources/dynamic-table-settings.md)
- [Materialization Settings](/core-concepts/data-sources/materialization-settings.md)
- [Product Tiers & Features](/core-concepts/data-sources/product-tiers-and-features.md)
- [Schema Filters](/core-concepts/data-sources/schema-filters.md)
- [Schema Transformations](/core-concepts/data-sources/schema-transformations.md)
- [Secure View Settings](/core-concepts/data-sources/secure-view-settings.md)
- [Source Object Settings](/core-concepts/data-sources/source-object-settings.md)
- [Scan Configurations](/core-concepts/scan-configurations.md): Introduction to scan configurations - learn and control how DataPancake scans your data.
- [Basic Configuration Settings](/core-concepts/scan-configurations/basic-configuration-settings.md): Core settings for naming, attribute discovery method, status, and auto code generation.
- [Scheduling Settings](/core-concepts/scan-configurations/scheduling-settings.md): Configure automated scanning with cron schedules and time zones.
- [Data Source & Warehouse Settings](/core-concepts/scan-configurations/data-source-and-warehouse-settings.md): Configure record limits, WHERE clauses, and warehouse selection for scan operations.
- [Vertical Scale Settings](/core-concepts/scan-configurations/vertical-scale-settings.md): Control thread count, procedure calls, and memory management for optimal scan performance.
- [Source Stream Settings](/core-concepts/scan-configurations/source-stream-settings.md): Configure incremental scanning with timestamp tracking for efficient delta updates.
- [Common Patterns & Best Practices](/core-concepts/scan-configurations/common-patterns-and-best-practices.md): Pre-configured patterns and optimization recommendations for common scan scenarios.
- [Attribute Metadata](/core-concepts/attribute-metadata.md)
- [Attribute Discovery Process](/core-concepts/attribute-metadata/attribute-discovery-process.md): How DataPancake discovers attributes during scanning, including polymorphic detection and recursive parsing of stringified JSON.
- [Attribute Metadata Types](/core-concepts/attribute-metadata/attribute-metadata-types.md): Overview of the three attribute types and where they come from: Discovered (from scans), Schema (from samples), and Virtual (user-created).
- [Attribute Metadata Details](/core-concepts/attribute-metadata/attribute-metadata-details.md): Complete reference for attribute metadata, covering discovered source schema fields and configurable extended metadata that controls SQL transformation, security policies, and code generation.
- [Source Schema](/core-concepts/attribute-metadata/attribute-metadata-details/source-schema.md): Read-only metadata discovered during scanning, including attribute paths, nesting levels, data types, structure information, and sample values representing what DataPancake found in your source data.
- [Destination Schema](/core-concepts/attribute-metadata/attribute-metadata-details/destination-schema.md): Configuration for mapping source attributes to Snowflake data types, including precision, scale, datetime formats, primary key identification, and data quality indicators for Dynamic Tables.
- [Transformation Security](/core-concepts/attribute-metadata/attribute-metadata-details/transformation-security.md): Configuration for attribute transformations during materialization, semantic layer expressions, column naming, null handling, schema consolidation, and security policies including row-level access and
- [Arrays](/core-concepts/attribute-metadata/attribute-metadata-details/arrays.md): Configuration for array attributes in code generation, including table creation, relationship types and joins, custom aliases for tables and views, and security policies for array tables.
- [Foreign Keys](/core-concepts/attribute-metadata/attribute-metadata-details/foreign-keys.md): Configuration for foreign key relationships between array tables and parent tables, including parent attribute selection, column aliases, and row-level security policy integration.
- [Polymorphic Versions](/core-concepts/attribute-metadata/polymorphic-versions.md): How DataPancake proactively creates all 7 polymorphic versions for every attribute upfront, then activates only the versions that match discovered data types.
- [DataPancake Metadata Fields](/core-concepts/attribute-metadata/datapancake-metadata-fields.md): Reference for all extended metadata fields that control how attributes are transformed into SQL, including data types, transformations, and security policies.
- [Arrays & Nested Structures](/core-concepts/attribute-metadata/arrays-and-nested-structures.md): How DataPancake handles object arrays, primitive arrays, and nested structures, including foreign key relationships and table generation.
- [Attribute Lifecycle](/core-concepts/attribute-metadata/attribute-lifecycle.md): How attributes are created, versioned, and managed over time, including schema evolution and version activation/deactivation.
- [Attributes in Code Generation](/core-concepts/attribute-metadata/attributes-in-code-generation.md): How active polymorphic versions become SQL columns in Dynamic Tables, including transformation application and array table generation.
- [Best Practices](/core-concepts/attribute-metadata/best-practices.md): Configuration recommendations for attributes, including data type selection, transformation expressions, and performance optimization.
- [Integration & API](/core-concepts/attribute-metadata/integration-and-api.md): Programmatic access to attributes via system views and stored procedures for automation and external system integration.
- [Scan Processing](/core-concepts/scan-processing.md)
- [Views](/core-concepts/views.md): System views for monitoring, querying, and integrating with DataPancake metadata and operations.
- [Schema Summary Views](/core-concepts/views/schema-summary-views.md): Views for data source overview, scan history, configurations, and billing information.
- [Schema Analysis Views](/core-concepts/views/schema-analysis-views.md): Views for analyzing attributes, polymorphic versions, and schema evolution.
- [Security & Governance Views](/core-concepts/views/security-and-governance-views.md): Views for security policy configuration, data dictionary metadata, and transformation settings.
- [Integration & Code Generation Views](/core-concepts/views/integration-and-code-generation-views.md): Views for semantic layer configuration, foreign key relationships, and generated SQL code.
- [Monitoring & Diagnostic Views](/core-concepts/views/monitoring-and-diagnostic-views.md): Views for integration process tracking and access control.
- [Attribute Consolidation](/core-concepts/attribute-consolidation.md): Configure how attributes extract values from source data when schema transformations have consolidated paths during discovery. Attribute consolidation bridges the gap between consolidated discovery pa