# Attribute Metadata Details

## Overview

Attribute metadata is stored in:

* `core.datasource_attribute` - Core attribute information
* `core.datasource_attribute_polymorphic_version` - Version-specific metadata

**Source Schema Metadata:** Automatically discovered during scanning (read-only except `RECORD_STATUS`).

**Extended Metadata:** User-configurable fields for transformation, security, and code generation.

## Metadata Organization

Pipeline Designer organizes metadata by UI tabs:

* [**Source Schema**](/core-concepts/attribute-metadata/attribute-metadata-details/source-schema.md) - Discovered attributes, paths, types, structure
* [**Destination Schema**](/core-concepts/attribute-metadata/attribute-metadata-details/destination-schema.md) - Snowflake data type configuration, keys
* [**Transformation & Security**](/core-concepts/attribute-metadata/attribute-metadata-details/transformation-security.md) - Column naming, transformations, security policies
* [**Arrays**](/core-concepts/attribute-metadata/attribute-metadata-details/arrays.md) - Array-specific configuration and relationships
* [**Foreign Keys**](/core-concepts/attribute-metadata/attribute-metadata-details/foreign-keys.md) - Foreign key relationships for array tables

***

## Core Attribute Fields

Stored in `core.datasource_attribute` (applies to all polymorphic versions):

* `ATTRIBUTE_PATH` - Full path in source data (e.g., `customer.contact.email`)
* `ATTRIBUTE_PATH_EMBEDDED` - Path for attributes within embedded/stringified JSON
* `ATTRIBUTE_NAME` - Leaf name (e.g., `email` from `customer.contact.email`)
* `ATTRIBUTE_LEVEL` - Nesting depth (0 = root, increments per level)
* `ATTRIBUTE_ORDER` - Ordering for attributes at same level
* `PARENT_OBJECT` - Parent object path (e.g., `customer.contact` for `customer.contact.email`)
* `PARENT_ARRAY` - Parent array path if within array (e.g., `orders` for `orders[0].order_id`)
* `PARENT_ARRAY_EMBEDDED` - Parent array path for embedded JSON arrays
* `RECORD_STATUS` - `'active'` or `'inactive'` (editable)
* `ATTRIBUTE_TYPE` - `'Discovered'`, `'Schema'`, or `'Virtual'`
* `ATTRIBUTE_SCHEMA_COMPONENT_TYPE` - Internal classification
* `IS_PRIMARY_KEY` - Boolean for primary key identification

***

## Polymorphic Version Fields

Stored in `core.datasource_attribute_polymorphic_version` (version-specific):

**Type Information:**

* `SOURCE_DATA_TYPE` - Inferred type (`'str'`, `'int'`, `'float'`, `'bool'`, `'object'`, `'array'`, `'null'`)
* `POLYMORPHIC_ATTRIBUTE_NAME` - Unique version name (e.g., `email_str`, `orders_array_object`)
* `VERSION_STATUS` - `'active'` or `'inactive'`
* `IS_ARRAY` - Boolean for array type
* `ARRAY_TYPE` - `'object'`, `'primitive'`, or `'primitive,object'`
* `ARRAY_PRIMITIVE_TYPE` - Element type for primitive arrays (`'str'`, `'int'`, `'float'`, `'bool'`)
* `HAS_EMBEDDED_CONTENT` - Boolean for embedded/stringified JSON
* `SAMPLE_VALUE` - Sample value (strings default to `"string value"` for privacy)
* `VERSION_STATUS_DATE` - Timestamp when version was created/last activated

**Snowflake Data Type:**

* `DATA_PLATFORM_DATA_TYPE` - Snowflake data type (editable)
* `DATA_PLATFORM_DATA_TYPE_PRECISION` - Precision for numeric types (editable)
* `DATA_PLATFORM_DATA_TYPE_SCALE` - Scale for numeric types (editable)
* `DATA_PLATFORM_DATA_TYPE_DATETIMEFORMAT` - DateTime format strings (editable)
* `USE_DATETIME_FORMAT` - Boolean for using formats (editable)

**Transformation & Naming:**

* `ALIAS_NAME` - Custom column alias (editable)
* `CODE_GENERATED_COLUMN_NAME` - Final column name (read-only, auto-generated)
* `NULL_VALUE_EXPRESSION` - SQL for null handling (editable, only when `TRANSFORMATION_TYPE = 'No Transformation'`)
* `TRANSFORMATION_TYPE` - `'No Transformation'` or `'SQL Expression'` (editable)
* `TRANSFORMATION_EXPRESSION` - Custom SQL transformation (editable)
* `TRANSFORMATION_EXPRESSION_COMMENT` - Documentation (editable)

**Semantic Layer:**

* `INCLUDE_IN_SEMANTIC_LAYER` - Boolean for view inclusion (editable)
* `SEMANTIC_LAYER_EXPRESSION` - View-level transformation (editable)
* `SEMANTIC_LAYER_EXPRESSION_COMMENT` - Documentation (editable)
* `SEMANTIC_LAYER_ALIAS_NAME` - View column alias (editable)

**Security:**

* `INCLUDE_IN_SECURITY_ROW_LEVEL_POLICY` - Boolean for row-level security (editable)
* `MASKING_POLICY_NAME` - Masking policy name (editable)
* `MASKING_POLICY_PARAMETERS` - Policy parameters (editable)

**Schema Consolidation:**

* `SCHEMA_INSERT_REGULAR_EXPRESSION_SEARCH` - Regex pattern (editable)
* `SCHEMA_INSERT_SQL_EXPRESSION` - SQL expression (editable)

**Code Generation:**

* `INCLUDE_IN_CODE_GEN` - Boolean for code generation (editable)
* `PARENT_INCLUDE_IN_CODE_GEN` - Boolean for parent inclusion (read-only)

**Additional:**

* `IS_ENUM` - Boolean for enum values (editable)
* `HAS_ALL_UNIQUE_VALUES` - Boolean for unique values (editable)
* `HAS_ALL_NULL_VALUES` - Boolean for null values (read-only)
* `ADD_TO_CLUSTER_BY` - Boolean for CLUSTER BY clause (editable)

***

## Metadata Workflow

1. **Discovery** - Source schema metadata discovered during scanning
2. **Configuration** - Extended metadata configured in Pipeline Designer
3. **Code Generation** - Metadata drives SQL generation for Dynamic Tables and Views
4. **Materialization** - Generated SQL creates materialized tables with transformations
5. **Semantic Layer** - Additional transformations applied in views

## Quick Reference

See individual pages for complete field details:

* [Source Schema](/core-concepts/attribute-metadata/attribute-metadata-details/source-schema.md) - `RECORD_STATUS` (editable)
* [Destination Schema](/core-concepts/attribute-metadata/attribute-metadata-details/destination-schema.md) - Data types, precision, scale, datetime formats, keys
* [Transformation & Security](/core-concepts/attribute-metadata/attribute-metadata-details/transformation-security.md) - Naming, transformations, semantic layer, security
* [Arrays](/core-concepts/attribute-metadata/attribute-metadata-details/arrays.md) - Array configuration, relationships, aliases
* [Foreign Keys](/core-concepts/attribute-metadata/attribute-metadata-details/foreign-keys.md) - Foreign key relationships


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.datapancake.com/core-concepts/attribute-metadata/attribute-metadata-details.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.