# Core Concepts - [Warehouses](https://docs.datapancake.com/core-concepts/warehouses.md): Snowflake virtual warehouses configured for DataPancake scanning operations, with specific requirements for Snowpark-Optimized warehouses and concurrency settings. - [Data Sources](https://docs.datapancake.com/core-concepts/data-sources.md): Introduction to data sources and navigation hub for all data source documentation. - [Adding Data Sources](https://docs.datapancake.com/core-concepts/data-sources/adding-data-sources.md) - [Baseline Scan Settings](https://docs.datapancake.com/core-concepts/data-sources/baseline-scan-settings.md) - [Basic Configuration Settings](https://docs.datapancake.com/core-concepts/data-sources/basic-configuration-settings.md) - [Data Source Types](https://docs.datapancake.com/core-concepts/data-sources/data-source-types.md) - [Dynamic Table Settings](https://docs.datapancake.com/core-concepts/data-sources/dynamic-table-settings.md) - [Materialization Settings](https://docs.datapancake.com/core-concepts/data-sources/materialization-settings.md) - [Product Tiers & Features](https://docs.datapancake.com/core-concepts/data-sources/product-tiers-and-features.md) - [Schema Filters](https://docs.datapancake.com/core-concepts/data-sources/schema-filters.md) - [Schema Transformations](https://docs.datapancake.com/core-concepts/data-sources/schema-transformations.md) - [Secure View Settings](https://docs.datapancake.com/core-concepts/data-sources/secure-view-settings.md) - [Source Object Settings](https://docs.datapancake.com/core-concepts/data-sources/source-object-settings.md) - [Scan Configurations](https://docs.datapancake.com/core-concepts/scan-configurations.md): Introduction to scan configurations - learn and control how DataPancake scans your data. - [Basic Configuration Settings](https://docs.datapancake.com/core-concepts/scan-configurations/basic-configuration-settings.md): Core settings for naming, attribute discovery method, status, and auto code generation. - [Scheduling Settings](https://docs.datapancake.com/core-concepts/scan-configurations/scheduling-settings.md): Configure automated scanning with cron schedules and time zones. - [Data Source & Warehouse Settings](https://docs.datapancake.com/core-concepts/scan-configurations/data-source-and-warehouse-settings.md): Configure record limits, WHERE clauses, and warehouse selection for scan operations. - [Vertical Scale Settings](https://docs.datapancake.com/core-concepts/scan-configurations/vertical-scale-settings.md): Control thread count, procedure calls, and memory management for optimal scan performance. - [Source Stream Settings](https://docs.datapancake.com/core-concepts/scan-configurations/source-stream-settings.md): Configure incremental scanning with timestamp tracking for efficient delta updates. - [Common Patterns & Best Practices](https://docs.datapancake.com/core-concepts/scan-configurations/common-patterns-and-best-practices.md): Pre-configured patterns and optimization recommendations for common scan scenarios. - [Attribute Metadata](https://docs.datapancake.com/core-concepts/attribute-metadata.md) - [Attribute Discovery Process](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-discovery-process.md): How DataPancake discovers attributes during scanning, including polymorphic detection and recursive parsing of stringified JSON. - [Attribute Metadata Types](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-types.md): Overview of the three attribute types and where they come from: Discovered (from scans), Schema (from samples), and Virtual (user-created). - [Attribute Metadata Details](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-details.md): Complete reference for attribute metadata, covering discovered source schema fields and configurable extended metadata that controls SQL transformation, security policies, and code generation. - [Source Schema](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-details/source-schema.md): Read-only metadata discovered during scanning, including attribute paths, nesting levels, data types, structure information, and sample values representing what DataPancake found in your source data. - [Destination Schema](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-details/destination-schema.md): Configuration for mapping source attributes to Snowflake data types, including precision, scale, datetime formats, primary key identification, and data quality indicators for Dynamic Tables. - [Transformation Security](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-details/transformation-security.md): Configuration for attribute transformations during materialization, semantic layer expressions, column naming, null handling, schema consolidation, and security policies including row-level access and - [Arrays](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-details/arrays.md): Configuration for array attributes in code generation, including table creation, relationship types and joins, custom aliases for tables and views, and security policies for array tables. - [Foreign Keys](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-details/foreign-keys.md): Configuration for foreign key relationships between array tables and parent tables, including parent attribute selection, column aliases, and row-level security policy integration. - [Polymorphic Versions](https://docs.datapancake.com/core-concepts/attribute-metadata/polymorphic-versions.md): How DataPancake proactively creates all 7 polymorphic versions for every attribute upfront, then activates only the versions that match discovered data types. - [DataPancake Metadata Fields](https://docs.datapancake.com/core-concepts/attribute-metadata/datapancake-metadata-fields.md): Reference for all extended metadata fields that control how attributes are transformed into SQL, including data types, transformations, and security policies. - [Arrays & Nested Structures](https://docs.datapancake.com/core-concepts/attribute-metadata/arrays-and-nested-structures.md): How DataPancake handles object arrays, primitive arrays, and nested structures, including foreign key relationships and table generation. - [Attribute Lifecycle](https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-lifecycle.md): How attributes are created, versioned, and managed over time, including schema evolution and version activation/deactivation. - [Attributes in Code Generation](https://docs.datapancake.com/core-concepts/attribute-metadata/attributes-in-code-generation.md): How active polymorphic versions become SQL columns in Dynamic Tables, including transformation application and array table generation. - [Best Practices](https://docs.datapancake.com/core-concepts/attribute-metadata/best-practices.md): Configuration recommendations for attributes, including data type selection, transformation expressions, and performance optimization. - [Integration & API](https://docs.datapancake.com/core-concepts/attribute-metadata/integration-and-api.md): Programmatic access to attributes via system views and stored procedures for automation and external system integration. - [Foreign Keys](https://docs.datapancake.com/core-concepts/attribute-metadata/foreign-keys.md) - [SQL Expressions](https://docs.datapancake.com/core-concepts/attribute-metadata/sql-expressions.md) - [Scan Processing](https://docs.datapancake.com/core-concepts/scan-processing.md) - [Views](https://docs.datapancake.com/core-concepts/views.md): System views for monitoring, querying, and integrating with DataPancake metadata and operations. - [Schema Summary Views](https://docs.datapancake.com/core-concepts/views/schema-summary-views.md): Views for data source overview, scan history, configurations, and billing information. - [Schema Analysis Views](https://docs.datapancake.com/core-concepts/views/schema-analysis-views.md): Views for analyzing attributes, polymorphic versions, and schema evolution. - [Security & Governance Views](https://docs.datapancake.com/core-concepts/views/security-and-governance-views.md): Views for security policy configuration, data dictionary metadata, and transformation settings. - [Integration & Code Generation Views](https://docs.datapancake.com/core-concepts/views/integration-and-code-generation-views.md): Views for semantic layer configuration, foreign key relationships, and generated SQL code. - [Monitoring & Diagnostic Views](https://docs.datapancake.com/core-concepts/views/monitoring-and-diagnostic-views.md): Views for integration process tracking and access control. - [Attribute Consolidation](https://docs.datapancake.com/core-concepts/attribute-consolidation.md): Configure how attributes extract values from source data when schema transformations have consolidated paths during discovery. Attribute consolidation bridges the gap between consolidated discovery pa --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.datapancake.com/core-concepts.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.