Destination Schema

Configuration for mapping source attributes to Snowflake data types, including precision, scale, datetime formats, primary key identification, and data quality indicators for Dynamic Tables.

Overview

Destination Schema metadata controls how source attributes map to Snowflake data types and identifies key characteristics for Dynamic Tables. Configure target schema here.

---

Snowflake Data Type Configuration

Data Platform Data Type

Data Platform Data Type (DATA_PLATFORM_DATA_TYPE)

• Snowflake data type for this attribute

• Examples: VARCHAR, VARCHAR(n), NUMBER, NUMBER(p,s), TIMESTAMP_NTZ, DATE, BOOLEAN, VARIANT

• Editable - Override inferred type

• DataPancake infers from source data; override when inference is incorrect or you need specific types/precision

---

Numeric Type Configuration

Data Platform Data Type Precision (DATA_PLATFORM_DATA_TYPE_PRECISION)

• For numeric types, precision (total number of digits)

• Editable - Applies to DECIMAL, DEC, NUMERIC, NUMBER

• Example: For NUMBER(10,2), precision is 10

Data Platform Data Type Scale (DATA_PLATFORM_DATA_TYPE_SCALE)

• For numeric types, scale (number of decimal places)

• Editable - Applies to DECIMAL, DEC, NUMERIC, NUMBER

• Example: For NUMBER(10,2), scale is 2

Note: Set based on known data ranges; avoid over-sizing (wastes storage).

---

DateTime Configuration

Data Platform Data Type DateTime Format (DATA_PLATFORM_DATA_TYPE_DATETIMEFORMAT)

• Comma-separated list of valid Snowflake datetime formats

• Used for accurate datetime parsing

• Editable - Users can specify formats if inference is incorrect

• Example: 'MM/DD/YYYY', 'YYYY-MM-DD', 'MM-DD-YYYY HH24:MI:SS'

• See Snowflake datetime formats documentation

Use DateTime Format (USE_DATETIME_FORMAT)

• Boolean for using datetime formats in Dynamic Table SQL generation

• Editable

• FALSE: Uses try_to_{data_type} with AUTO detection

• TRUE: Uses DATA_PLATFORM_DATA_TYPE_DATETIMEFORMAT for parsing

When to use formats: Consistent formats, need precise control, AUTO unreliable.

When to use AUTO: Inconsistent/unknown formats, want Snowflake auto-detection.

---

Key Identification

Is Primary Key (IS_PRIMARY_KEY)

• Boolean indicating if attribute is part of primary key

• Editable - Multiple attributes can be marked (composite keys)

• Used for metadata table generation and relationship tracking

Has All Unique Values (HAS_ALL_UNIQUE_VALUES)

• Boolean indicating if all values in this version are unique

• Editable - Mark based on data analysis

• Useful for identifying potential primary keys

---

Data Quality Indicators

Is Enum (IS_ENUM)

• Boolean indicating if version contains only enum values

• Editable - Mark for categorical data (e.g., status codes)

• Useful for validation and documentation

Has All Null Values (HAS_ALL_NULL_VALUES)

• Boolean indicating if all values are null

• Read-only - Detected during scanning

• Useful for identifying unused attributes

---

Reference Fields (Read-only)

Shown for reference in Destination Schema tab:

• ATTRIBUTE_PATH - Attribute identification

• POLYMORPHIC_ATTRIBUTE_NAME - Version name

• SOURCE_DATA_TYPE - Source type reference

• SAMPLE_VALUE - Data content reference

• ATTRIBUTE_FOREIGN_KEY_COLUMNS - Configured in Foreign Keys section

---

Data Type Selection

String types:

• VARCHAR - Variable length without constraints

• VARCHAR(n) - Known maximum length

• VARIANT - Truly variable type, preserve original structure

Numeric types:

• Set precision/scale based on known data ranges

• Balance flexibility vs. storage efficiency

• Financial data: Use appropriate scale (typically 2-4 decimal places)

DateTime types:

• DATE - Date only

• TIME - Time only

• TIMESTAMP_NTZ - Timestamp without timezone

• TIMESTAMP_LTZ - Timestamp with local timezone

• TIMESTAMP_TZ - Timestamp with timezone

See Snowflake data types documentation for details.

---

Common Scenarios

Correcting incorrect type inference:

1. Change DATA_PLATFORM_DATA_TYPE to correct type

2. Set DATA_PLATFORM_DATA_TYPE_PRECISION and DATA_PLATFORM_DATA_TYPE_SCALE if numeric

3. Verify with sample data

Configuring datetime formats:

1. Check SAMPLE_VALUE for actual format

2. Set DATA_PLATFORM_DATA_TYPE_DATETIMEFORMAT with correct format(s)

3. Set USE_DATETIME_FORMAT = TRUE

4. Test with code generation

Setting primary keys:

1. Identify attributes that uniquely identify records

2. Set IS_PRIMARY_KEY = TRUE (multiple for composite keys)

3. Verify uniqueness with HAS_ALL_UNIQUE_VALUES

Optimizing numeric types:

1. Analyze actual data ranges

2. Set DATA_PLATFORM_DATA_TYPE_PRECISION to minimum needed

3. Set DATA_PLATFORM_DATA_TYPE_SCALE appropriately