Attribute Metadata Bulk Updates - Execute

Export the attribute metadata to a .csv file, edit in a spreadsheet like Google Sheets or Excel, then use to perform a bulk update in DataPancake.

Ensure you have completed How to Setup Attribute Metadata Bulk Updates

1. Open a new Snowflake worksheet

2. Run a select statement for datapancake.core.vw_datasource_all_attributes

The minimum field requirements to select are attribute_id, polymorphic_version_id - for example:

select attribute_id, polymorphic_version_id
from datapancake.core.vw_datasource_all_attributes
where version_status = 'active';

This SQL statement includes key non-editable fields along with all editable fields:

select attribute_id, polymorphic_version_id, datasource_name, attribute_path,
attribute_name, source_data_type, data_platform_data_type, numeric_precision,
numeric_scale, use_datetime_format, datetime_format, attribute_record_status,
is_unique, is_primary_key, contains_enum_values, include_column_in_secure_view, include_array_in_code_gen, array_relationship_name, array_relationship_description,
array_relationship_type, array_relationship_join_type, raw_transformation_type, raw_transformation_expression,
raw_transformation_expression_comment, raw_null_value_expression,
semantic_layer_transformation_expression, semantic_layer_transformation_expression_comment,
raw_alias_name, semantic_layer_alias_name,
consolidation_regex_search, consolidation_sql_expression, attribute_last_updated_by, attribute_last_updated_datetime
from datapancake.core.vw_datasource_all_attributes
where version_status = 'active'
order by attribute_path;

You can choose which editable fields you wish to include in your select statement.

These are all of the editable columns you can select and modify in a bulk update:

NUMERIC_PRECISION  
NUMERIC_SCALE  
USE_DATETIME_FORMAT  
DATETIME_FORMAT  
ATTRIBUTE_RECORD_STATUS  
IS_UNIQUE  
IS_PRIMARY_KEY  
CONTAINS_ENUM_VALUES  
INCLUDE_COLUMN_IN_SECURE_VIEW  
INCLUDE_ARRAY_IN_CODE_GEN  
ARRAY_RELATIONSHIP_NAME  
ARRAY_RELATIONSHIP_DESCRIPTION  
ARRAY_RELATIONSHIP_TYPE  
ARRAY_RELATIONSHIP_JOIN_TYPE  
RAW_TRANSFORMATION_TYPE  
RAW_TRANSFORMATION_EXPRESSION  
RAW_TRANSFORMATION_EXPRESSION_COMMENT  
RAW_NULL_VALUE_EXPRESSION  
SEMANTIC_LAYER_TRANSFORMATION_EXPRESSION  
SEMANTIC_LAYER_TRANSFORMATION_EXPRESSION_COMMENT  
RAW_ALIAS_NAME  
SEMANTIC_LAYER_ALIAS_NAME  
CONSOLIDATION_REGEX_SEARCH  
CONSOLIDATION_SQL_EXPRESSION

These are all of the view-only columns you can select:

DATASOURCE_ID  
DATASOURCE_NAME  
DATASOURCE_TAG  
DATASOURCE_VERSION_ID  
VERSION_NUMBER  
ATTRIBUTE_ID  
ATTRIBUTE_PATH  
ATTRIBUTE_NAME  
PARENT_OBJECT  
PARENT_ARRAY  
ATTRIBUTE_LEVEL  
ATTRIBUTE_ORDER  
ATTRIBUTE_LAST_UPDATED_DATETIME  
ATTRIBUTE_LAST_UPDATED_BY  
POLYMORPHIC_VERSION_ID  
SOURCE_DATA_TYPE  
POLYMORPHIC_ATTRIBUTE_NAME  
IS_ARRAY  
ARRAY_TYPE  
ARRAY_PRIMITIVE_TYPE  
VERSION_STATUS_DATE  
SAMPLE_VALUE  
HAS_EMBEDDED_CONTENT  
VERSION_STATUS  
DATA_PLATFORM_DATA_TYPE

3. Verify the query successfully returns a list of attribute metadata

4. Download the results as a .csv file

5. Open the .csv file in either Excel or Google Sheets

6. Edit any editable columns as needed

7. Save (Excel) or download (Google Sheets) the file

8. Load the .csv into your stage

9. Run the bulk update stored procedure

When the stored procedure completes, the attribute metadata file will be moved to the archive folder.

call <database>.<schema>.sp_update_datapancake_attribute_metadata_core(
    '@<stage_name>/<staged_attribute_metadata_filename>.csv',
    '@<stage_name>/<archive_folder>/',
    '<snowflake_username>'
);

10. Verify the integration completed successfully by running the following query

select * from datapancake.core.vw_integration_process
where user_name = 'BWOLFF'
order by integration_process_date desc;

Last updated