# How to Manage a Scan Configuration

<h2 align="center">Configure Required Scan Configuration Settings</h2>

### 1. Navigate to the Scan Configurations Page

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/6855b824-dc44-4506-9993-093c1390a350/ascreenshot.jpeg?tl_px=0,0\&br_px=1376,769\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=134,229)

### 2. Select a data source

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/e43acb34-1c67-4d26-85b1-9deebb8623c8/ascreenshot.jpeg?tl_px=0,18\&br_px=1376,787\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=467,277)

### 3. Enter the Configuration Name

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/873f6d4b-b298-486a-a960-2629470b070a/ascreenshot.jpeg?tl_px=22,0\&br_px=1399,769\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=524,172)

### 4. Select the Virtual Warehouse

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/8f11c5b0-d9c4-4a6d-a94f-5bb4b9630619/ascreenshot.jpeg?tl_px=164,223\&br_px=1541,993\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=714,414)

### 5. Ensure the Warehouse is Connected

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/b880aa14-f776-4db6-80a6-0104dbf4e193/ascreenshot.jpeg?tl_px=0,1\&br_px=1376,770\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=451,277)

### 6. (Optional) Configure additional settings

[Base Configuration Settings](#optional-base-configuration-settings)

[Scan Schedule Settings](#optional-scan-schedule-settings)

[Data Source Settings](#optional-data-source-settings)

[Vertical Scale Settings](#optional-vertical-scale-settings)

[Source Stream Settings](#optional-source-stream-settings)

### 7. Save the scan configuration

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/06882023-1d27-4119-8b06-439335982411/ascreenshot.jpeg?tl_px=0,223\&br_px=1376,993\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=344,514)

***

<h2 align="center">Optional Base Configuration Settings</h2>

### Set the Attribute Create Type

The default Discover create type uses scanned data to create attribute metadata.

The Schema create type uses the Data Source Object Schema Sample data to create the attribute metadata.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/a80d45d6-f212-4ac5-9bee-00d75fdf4182/ascreenshot.jpeg?tl_px=0,119\&br_px=1376,888\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=454,277)

### Update the Scan Configuration Status

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/e298cea1-5df6-49a9-869e-8317b4a499e5/ascreenshot.jpeg?tl_px=0,94\&br_px=1376,863\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=462,276)

### Enable Auto Code Generate

If enabled, DataPancake will automatically generate dynamic table SQL code if the data source's schema or polymorphic state changes.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/01ed1775-4960-46f2-b4a9-9517115a212d/ascreenshot.jpeg?tl_px=0,1\&br_px=1376,770\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=315,277)

***

<h2 align="center">(Optional) Scan Schedule Settings</h2>

### 1. Enable a Scan Schedule

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/2a73768b-3d47-4657-bd34-b0e40f2d2f1e/ascreenshot.jpeg?tl_px=164,0\&br_px=1541,769\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=655,121)

### 2. Enter a Cron Schedule

A cron schedule is required if you enable scheduling. See the examples below for valid cron schedules:

Hourly at 30 minutes after the hour

```python
30 * * * *
```

Daily at 3:00 am

```python
0 3 * * *
```

Weekly at 3:00 am every Monday

```python
0 3 * * MON
```

Monthly at 3:00 am on the 1st of each month

```python
0 3 1 * *
```

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/23cda880-5941-445a-952e-71818bd00401/ascreenshot.jpeg?tl_px=164,0\&br_px=1541,769\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=817,187)

### 3. Select a Cron Time Zone

A cron timezone is required in addition to the cron schedule if you enable scheduling.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/8e9f7cd2-3912-4a4b-bbf7-c5bd81375c06/ascreenshot.jpeg?tl_px=164,223\&br_px=1541,993\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=780,283)

***

<h2 align="center">(Optional) Data Source Settings</h2>

### Enter a Record Limit

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/4618ecf2-704f-4a91-adf4-464f118f2855/ascreenshot.jpeg?tl_px=0,0\&br_px=1376,769\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=489,209)

### Enter a Where Clause

The where clause can only be used with a single procedure call.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/73903adb-add3-40a7-9973-580d207c779a/ascreenshot.jpeg?tl_px=60,2\&br_px=1437,771\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=524,277)

***

<h2 align="center">(Optional) Vertical Scale Settings</h2>

### Modify the Number of Threads

The number of threads will default to the maximum number of threads available to the virtual warehouse chosen.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/26942d58-3e63-447a-8d0e-934c6aca07e0/ascreenshot.jpeg?tl_px=0,115\&br_px=1376,884\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=484,277)

### Modify the Number of Procedure Calls

The number of scan procedure calls required to process the entire dataset.

Use multiple procedure calls when a single call cannot be completed in under sixty minutes which is the default timeout for the Snowpark Python Sandbox.

The where clause parameter is not available if the number of procedure calls is greater than 1.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/ce556e31-b88f-4606-a8c7-861cad8a679b/ascreenshot.jpeg?tl_px=0,207\&br_px=1376,976\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=500,277)

### Modify the Record Count Per Procedure Call

The record count per procedure is required when the number of procedure calls is greater than 1.

The record count chosen needs to be large enough to process all the rows in the data source but not too large as to create procedure calls with no rows to process based on the criteria you have provided and the number of calls entered.

Ex. 2,000,000 rows can successfully be divided into two calls with 1m records per call. But it cannot be divided into 2 calls with 500,000 rows per call or 3 calls with 2.5m rows per call.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/500061a7-5574-4bcc-aead-2c2d8360a121/ascreenshot.jpeg?tl_px=0,223\&br_px=1376,993\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=489,329)

***

<h2 align="center">(Optional) Source Stream Settings</h2>

### Modify the Last Scanned Timestamp

The last timestamp scanned. This value will be used as part of the where clause when scanning data from this data source. To scan the entire datasource remove the timestamp if one exists.

![](https://ajeuwbhvhr.cloudimg.io/https://colony-recorder.s3.amazonaws.com/files/2025-06-21/36f20587-4433-47a8-911b-769164f631ff/ascreenshot.jpeg?tl_px=164,123\&br_px=1541,892\&force_format=jpeg\&q=100\&width=1120.0\&wat=1\&wat_opacity=1\&wat_gravity=northwest\&wat_url=https://colony-recorder.s3.amazonaws.com/images/watermarks/FB923C_standard.png\&wat_pad=816,277)

<h2 align="center"></h2>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.datapancake.com/guides/how-to-guides-ui/creating-and-configuring-data-sources/how-to-manage-a-scan-configuration.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
