# DataPancake Metadata Fields Complete reference for all attribute metadata fields. Metadata is stored in: * `core.datasource_attribute` - Core attribute information * `core.datasource_attribute_polymorphic_version` - Version-specific metadata **Note:** See [attribute-metadata-details](/core-concepts/attribute-metadata/attribute-metadata-details.md) for field organization by UI tabs (Source Schema, Destination Schema, Transformation/Security, Arrays, Foreign Keys). *** ## Core Attribute Fields **Attribute Path (`ATTRIBUTE_PATH`)** * The full path to the attribute in the source data * Example: `customer.contact.email` * Used to reference the attribute in source queries **Attribute Path Embedded (`ATTRIBUTE_PATH_EMBEDDED`)** * Path for attributes found within embedded/stringified JSON * Tracks nested JSON structures within string fields **Attribute Name (`ATTRIBUTE_NAME`)** * The leaf name of the attribute * Example: For path `customer.contact.email`, the name is `email` **Attribute Level (`ATTRIBUTE_LEVEL`)** * The nesting depth of the attribute * Root level attributes have level 0 * Each nested level increments the count **Attribute Order (`ATTRIBUTE_ORDER`)** * Ordering information for attributes at the same level * Used for consistent presentation **Parent Object (`PARENT_OBJECT`)** * The parent object path containing this attribute * Example: For `customer.contact.email`, parent object is `customer.contact` **Parent Array (`PARENT_ARRAY`)** * The parent array path if this attribute is within an array * Example: For `orders[0].order_id`, parent array is `orders` * Used for foreign key relationship configuration **Parent Array Embedded (`PARENT_ARRAY_EMBEDDED`)** * Parent array path for attributes within embedded JSON arrays **Record Status (`RECORD_STATUS`)** * `'active'` - Attribute is active and can be used * `'inactive'` - Attribute is deactivated **Attribute Type (`ATTRIBUTE_TYPE`)** * `'Discovered'` - Found during scanning * `'Schema'` - Created from schema sample * `'Virtual'` - User-created custom attribute **Attribute Schema Component Type (`ATTRIBUTE_SCHEMA_COMPONENT_TYPE`)** * Classifies the schema component type * Used for internal organization **Is Primary Key (`IS_PRIMARY_KEY`)** * Boolean indicating if this attribute is part of the primary key * Used for metadata table generation and relationship tracking **Version Status Date (`VERSION_STATUS_DATE`)** * Timestamp when the polymorphic version was created or last activated * Used for tracking attribute lifecycle and schema evolution * Stored in `core.datasource_attribute_polymorphic_version` (not `core.datasource_attribute`) *** ## Polymorphic Version Fields ### Type Information **Source Data Type (`SOURCE_DATA_TYPE`)** * The inferred data type from the source data * Values: `'str'`, `'int'`, `'float'`, `'bool'`, `'object'`, `'array'`, `'null'` **Polymorphic Attribute Name (`POLYMORPHIC_ATTRIBUTE_NAME`)** * Unique name for this polymorphic version * Format: `{attribute_name}_{type}` or `{attribute_name}_array_{array_type}` **Version Status (`VERSION_STATUS`)** * `'active'` - Version is active and included in code generation * `'inactive'` - Version exists but is not currently active **Is Array (`IS_ARRAY`)** * Boolean indicating if this version is an array type **Array Type (`ARRAY_TYPE`)** * `'object'` - Array contains objects * `'primitive'` - Array contains primitive values * `'primitive,object'` - Array contains both primitives and objects (polymorphic array) **Array Primitive Type (`ARRAY_PRIMITIVE_TYPE`)** * For primitive arrays, the data type of array elements * Values: `'str'`, `'int'`, `'float'`, `'bool'` **Has Embedded Content (`HAS_EMBEDDED_CONTENT`)** * Boolean indicating if this attribute contains embedded/stringified JSON * Triggers recursive parsing of the embedded content **Sample Value (`SAMPLE_VALUE`)** * A sample value from the source data * For string types, defaults to `"string value"` for privacy/security * Helps users understand the data content *** ### Snowflake Data Type Configuration **Data Platform Data Type (`DATA_PLATFORM_DATA_TYPE`)** * The Snowflake data type to use for this attribute * Examples: `VARCHAR`, `NUMBER`, `TIMESTAMP_NTZ`, `BOOLEAN`, `VARIANT` * Editable — Users can change the inferred type **Data Platform Data Type Precision (`DATA_PLATFORM_DATA_TYPE_PRECISION`)** * For numeric types, the precision (total number of digits) * Editable — Users can adjust based on known data requirements **Data Platform Data Type Scale (`DATA_PLATFORM_DATA_TYPE_SCALE`)** * For numeric types, the scale (number of decimal places) * Editable — Users can adjust based on known data requirements **Data Platform Data Type DateTime Format (`DATA_PLATFORM_DATA_TYPE_DATETIMEFORMAT`)** * Comma-separated list of valid Snowflake datetime formats * Used for accurate datetime parsing * Editable — Users can specify formats if inference is incorrect * See Snowflake datetime formats documentation:\ **Use DateTime Format (`USE_DATETIME_FORMAT`)** * Boolean determining whether datetime formats are used in Dynamic Table SQL generation * If `FALSE`, uses `try_to_{data_type}` with `AUTO` detection * If `TRUE`, uses specified formats for more accurate parsing **Use Number Format (`USE_NUMBER_FORMAT`)** * Boolean for number format handling (similar to datetime format) *** ### Column Naming **Alias Name (`ALIAS_NAME`)** * Custom column name alias for the generated Dynamic Table column * Overrides the default polymorphic attribute name * Useful for creating more readable column names **Code Generated Column Name (`CODE_GENERATED_COLUMN_NAME`)** * The final column name used in generated SQL * Uses `ALIAS_NAME` if provided, otherwise `POLYMORPHIC_ATTRIBUTE_NAME` * Read-only (automatically generated) *** ### Transformation Configuration **Null Value Expression (`NULL_VALUE_EXPRESSION`)** * SQL expression for handling null/missing values * Default: `NULL` * Can use `{attribute_name}` placeholder to reference the polymorphic version * Not applicable if transformation type is `'SQL Expression'` Example: ```sql COALESCE({attribute_name}, 'N/A') ``` **Transformation Type (`TRANSFORMATION_TYPE`)** * `'No Transformation'` - Use the attribute as-is with null value expression * `'SQL Expression'` - Apply a custom SQL transformation expression **Transformation Expression (`TRANSFORMATION_EXPRESSION`)** * SQL expression used for Dynamic Table column definition when transformation type is `'SQL Expression'` * Can use `{attribute_name}` placeholder to reference the polymorphic version * Applied during materialization process Examples: ```sql UPPER({attribute_name}) -- or CASE WHEN {attribute_name} IS NULL THEN 'Unknown' ELSE {attribute_name} END ``` **Transformation Expression Comment (`TRANSFORMATION_EXPRESSION_COMMENT`)** * Documentation/comment for the transformation expression * Helps users understand the transformation logic *** ### Code Generation Control **Include In Code Gen (`INCLUDE_IN_CODE_GEN`)** * Boolean determining if this polymorphic version is included in SQL code generation * `TRUE` - Attribute will appear in generated Dynamic Tables * `FALSE` - Attribute is excluded from code generation **Parent Include In Code Gen (`PARENT_INCLUDE_IN_CODE_GEN`)** * Boolean indicating if the parent attribute/array is included in code generation * Child attributes are only generated if parent is included *** ### Semantic Layer Fields **Include In Semantic Layer (`INCLUDE_IN_SEMANTIC_LAYER`)** * Boolean for including this attribute in semantic layer views **Semantic Layer Expression (`SEMANTIC_LAYER_EXPRESSION`)** * Additional transformation expression for semantic layer views * Applied on top of materialized Dynamic Table columns **Semantic Layer Expression Comment (`SEMANTIC_LAYER_EXPRESSION_COMMENT`)** * Documentation for semantic layer transformation **Semantic Layer Alias Name (`SEMANTIC_LAYER_ALIAS_NAME`)** * Custom alias for semantic layer views *** ### Security Policy Fields **Include In Security Row Level Policy (`INCLUDE_IN_SECURITY_ROW_LEVEL_POLICY`)** * Boolean for row-level security policy integration **Masking Policy Name (`MASKING_POLICY_NAME`)** * Name of column masking policy to apply **Masking Policy Parameters (`MASKING_POLICY_PARAMETERS`)** * Parameters for the masking policy *** ### Foreign Key & Relationship Fields **Attribute Foreign Key Columns (`ATTRIBUTE_FOREIGN_KEY_COLUMNS`)** * Comma-separated list of foreign key column names * Used for nested array relationships Array Relationship Fields (for array attributes) **Array Relationship Name (`ARRAY_RELATIONSHIP_NAME`)** * Name of the relationship for this array **Array Relationship Description (`ARRAY_RELATIONSHIP_DESCRIPTION`)** * Description of the relationship **Array Relationship Type (`ARRAY_RELATIONSHIP_TYPE`)** * `'many_to_one'` or `'one_to_one'` **Array Relationship Join Type (`ARRAY_RELATIONSHIP_JOIN_TYPE`)** * `'left_outer'` or `'inner'` *** ### Cortex Analyst / Semantic Model Fields **W Question Category (`W_QUESTION_CATEGORY`)** * Category for Cortex Analyst semantic model * Values: `''`, `'dimensions'`, `'facts'`, `'filters'`, `'metrics'`, `'time_dimensions'` * Used to classify attributes in semantic models **Gen AI Semantic Model Expression (`GEN_AI_SEMANTIC_MODEL_EXPRESSION`)** * Expression for Cortex Analyst semantic model * Used when W\_QUESTION\_CATEGORY is set **Include In Gen AI Semantic Model (`INCLUDE_IN_GEN_AI_SEMANTIC_MODEL`)** * Boolean for including in Cortex Analyst semantic model **Attribute Version Comment (`ATTRIBUTE_VERSION_COMMENT`)** * Description/comment for the attribute version * Used in data dictionary and semantic models **Attribute Synonyms (`ATTRIBUTE_SYNONYMS`)** * Comma-separated synonyms for the attribute * Used in Cortex Analyst for natural language querying *** ### Additional Metadata Fields **Is Enum (`IS_ENUM`)** * Boolean indicating if this version contains only enum values * Useful for validation and documentation **Has All Unique Values (`HAS_ALL_UNIQUE_VALUES`)** * Boolean indicating if all values in this version are unique * Useful for identifying potential primary keys **Has All Null Values (`HAS_ALL_NULL_VALUES`)** * Boolean indicating if all values are null * Useful for identifying unused attributes **Add To Cluster By (`ADD_TO_CLUSTER_BY`)** * Boolean for adding to CLUSTER BY clause in Dynamic Tables * Improves query performance for large tables *** ### Consolidation Fields **Schema Insert Regular Expression Search (`SCHEMA_INSERT_REGULAR_EXPRESSION_SEARCH`)** * Regex pattern for attribute consolidation * Used to find portions of paths to replace **Schema Insert SQL Expression (`SCHEMA_INSERT_SQL_EXPRESSION`)** * SQL expression for attribute consolidation * Uses regex matches to produce unified attribute paths --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.datapancake.com/core-concepts/attribute-metadata/datapancake-metadata-fields.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.