From time to time, Stitch may run into problems when attempting to load data into your destination. When data is deemed incompatible by the destination, the record will be “rejected” and logged in a table called _sdc_rejected.
Reasons for Rejection
Reasons for rejection will depend on the type of data warehouse you’re using, as each has its own data requirements and restrictions.
There are some common causes for rejection, however:
- Column names contain data type suffixes (ex:
__bigint), which are reserved by Stitch - Table and/or column names contain
_rjmor_sdcprefixes, which are reserved by Stitch - Table or column names exceed the supported character limit
- Integer data that falls outside the range supported by the data warehouse
For a detailed rollup of how each destination handles data - including what situations will result in rejected records - refer to the Data Loading guide for the destination you’re using.
_sdc_rejected Schema
In every integration schema created by Stitch is a table called _sdc_rejected which acts as a log for a particular integration’s rejected records.
This table contains information about why a record was rejected, the date of the rejection, and the name of the table in the integration schema that the data was destined for.
| Column Name | Description | Example |
|---|---|---|
_sdc_rejected_at
|
This column contains the date the loading error occurred. | 2016-11-29 10:50:55.816 |
table_name
|
This column contains the name of the table the record was destined for. |
order
|
reason
|
This column contains the reason why the record was rejected. |
Field collision on id
|
</tr>
record
|
This column contains as much of the raw Transit data of the record as Stitch can fit. |
"{""id"":1,""ID"":""Paul""}"
|
Take a look at the data in the Example column. If Stitch was attempting to load this record into a Redshift data warehouse, it would be rejected. Why?
The answer: Redshift is case-insensitive. Because id and Id canonicalize to the same name - that is, they differ only by case - a collision error surfaced when Stitch attempted to load the data.
Resolving Record Rejection Issues
In some cases, you may be able to pinpoint and resolve the root cause of the rejection.
Consider the id and Id example from the previous section. If these fields came from a database integration, you could re-name the columns - for example: customer_id and first_order_id - in the source database and re-sync the data. This would resolve the field collision error and allow Stitch to load the data.
However, it may not be possible to resolve every rejected record issue. While you may be able to resolve the issue in a database integration, the majority of SaaS integrations don’t provide users with the ability to define and control the structure of their data.
| Related | Troubleshooting |
Questions? Feedback?
Did this article help? If you have questions or feedback, please reach out to us.