Data Source Quick Start

This quick start will help you implement your first DataPancake pipeline for semi-structured data.

Prerequisite: Complete the Install & Setup process

1

Create, Scan, & Generate Code for your First Data Source

1. Load your data into Snowflake or access your data via iceberg tables.

2. Use the single data source script builder to create and scan your first data source.

3. Monitor the progress of the scan on the "Scans in Process" page. The scanning process will often take several minutes depending on the size of your data and the warehouse used.

DataPancake recursively scans your entire data source unless you specifically limit the amount of data scanned.

2

Deploy Dynamic Tables and Views (if generated)

1. Navigate to the "Generated SQL" page and select your data source.

2. Copy the generated SQL DDL.

3. In a new Snowflake worksheet or workspace, paste and execute all of the statements to deploy your dynamic tables and views (if generated).

You're now ready to start selecting data from your deployed objects!

Next Steps

Schema Shaping

Pipline Design

Scan Configuration

Code Generation & Deployment Options

Data Dictionary Builder

Semantic Model Generator

JSON Tutorial