Batch CSV Integration
- Adam Carter
- Michael Konomanyi
This integration type is used when you have to send files containing lead information directly to either an Email Address, FTP, FTPS, or SFTP connection or to a specific Dropbox folder.
With this integration, you will be able to schedule sends for any day of the week, with the option to ZIP and password protect
Include Fields
The section of the configuration allows users to map the Convertr Fields to the File Output
On the left-hand side, this is the naming of the columns on the right-hand side these are the fields that those columns will be referencing.
This can include all: Form Fields, Value Transforms and Convertr Meta Data (see below)
Field Name | Descirption |
|---|---|
| The unique Lead ID generated for all leads in Convertr |
| The IP address where the lead was generated from |
| The Form ID that the lead entered the platform from |
| The ID of the publisher that generated the lead |
| The Sub ID that a publisher may submit along with their lead data |
| The ID of the publisher that generated the lead |
| If the lead came in via a tracking link, this will show the channel associated to that link |
| The name of the publisher that generated the lead |
| The email domain on the lead's email field |
| The date at time of lead delivery |
| The date and time of lead delivery |
| The time of lead delivery |
| The date that the lead entered Convertr |
| The date and time that the lead entered Convertr |
| The unique hash generated for all leads in Convertr |
| The Form ID that the lead entered the platform from |
| The lead's current status |
| The unique Lead ID generated for all leads in Convertr |
| The IP address where the lead was generated from |
| The Form ID that the lead entered the platform from |
| The ID of the publisher that generated the lead |
| The Sub ID that a publisher may submit along with their lead data |
| The ID of the publisher contract |
| If the lead came in via a tracking link, this will show the channel associated to that link |
Integration Filters
When we pass data through via our integrations, the platform usually sends the data across in the format that it has been received. However, in some cases, you may need to format the data differently in order for it to go through to a client's CRM successfully.
When mapping an integration you are able to embed filters into selected fields. This will ensure the correct format is pulled through. You can do this by adding the following values to the end of the form field names in the integration:
Example
Filter | Description |
|---|---|
| send numeric values as integers |
| send values as string (this is done by default if no filter is used) |
| send date value as unix timestamp (seconds) |
| send date value as unix timestamp (milliseconds) |
| force data to be sent as lower case (e.g. “This Is a Headline” will be sent as “this is a headline”) |
| force data to be sent as upper case (e.g. This Is a Headline” will be sent as “THIS IS A HEADLINE”) |
| force data to be sent as title case (e.g. “this is a headline” will be sent as “This Is A Headline”) |
| leaves the field blank if there is no value stored in the lead details |
| force integration to ignore data in the field and just submit the field name itself |
| forces a field to have a NULL value on submission if the field is empty |
| forces the values true and false to be submitted as a boolean value instead of a string |
File Format
For the file format, users have three options here:
xlsx
csv
txt
File Name
Here is where you can name the file that the user will download.
Below are all the placeholders:
Placeholders | Description |
|---|---|
%campID% | This will pull through the Convertr Campaign ID |
%campaignName% | This will pull through the Convertr Campaign Name |
%Y% | the year that file was created eg 2020 |
%y% | the year that file was created showing the last 2 digits of the year eg 20 |
%D% | eg the year that file was created (non-numeric) eg Mon, Tue, Wed |
%d% | the day that file was created (numeric) eg 01,02,03 |
%H% | the hour that file was created in a 24-hour format eg 23:00 |
%h% | the hour that file was created in a 12-hour format eg 11 |
%i% | the minute that file was created eg 00:01:00 |
%M% | the month that file was created (non-numeric) eg JAN, FEB |
%m% | the month that file was created (numeric) eg ) 01, 02 ,03 etc |
%s% | the seconds on when the file was created with leading zeros eg 00:00:01 |
Finally, the file can be saved with .XLSX , .CSV or .TXT conventions.
File Type
Important: Please be aware that if this file_type is changed mid-campaign this will interrupt the current process and may affect your next file send.
There are two options here, default and cumulative.
a. Default: This file type means leads will be sent on to the endpoint in batches according to the schedule set. This file will only include the new leads from the platform since the last batch email was sent.
The leads sent to this file type will be stored in the database and encrypted, and only be appended to the new CSV file when the reports runs at the scheduled time. This means that all leads that have been deleted, invalidated, marked as test or undelivered will all be removed from pending batches, right up until the scheduled delivery date and time.
b. Cumulative: This file type means the report generated will contain all of the lead data within the campaign from the date that this file type was set to Cumulative. Any leads in this campaign prior to this file type change will not be included in the file sent on.
The leads sent to this file type will be stored in the database and encrypted. They will only be appended to the existing CSV file when the reports runs at the scheduled time. This means that any of the new leads that have been deleted, invalidated, marked as test or undelivered will all be removed from pending batches, right up until the scheduled delivery date and time.
Scenarios
As a user of Batch CSV Integration, I should be able to:
Scenario |
|
|---|---|
“default” type | Schedule my batches to send all the valid “On hold - pending batch” leads between each batch window. E.g If I set my batch to run 9 am everyday, on Tuesday I would expect all the compiled leads since 9 am Monday → to 8:59 Tuesday morning to be in a single file |
“cumulative” type | Each schedule should append the “On hold - pending batch” leads to a single file and send it. Using the example above, by Friday I would see all of the leads from Monday, Tuesday, Wednesday, Thursday and Friday morning in a single file. The file would still be sent to its destination as normal at 9 am each day with all leads for the time period. |
I should be able to run multiple batches on a single campaign without the files conflicting with each other, either using job groups or value-based routing, or just having two batches which collect different information.
I should be able to zip and password protect my files - this should work with cumulative and default batch types and not impact the expected behaviour of each type - this seems to be broken with cumulative files. “Password protecting” seems to just send the new leads, and not append to the existing file.
When the file is sent it should use the configured filename in the settings to create the actual filename.
CSV Delimiter
The CSV delimiter will only be used if the file format is set to CSV. This setting determines how the CSV is separated.
Number | Type | Example |
|---|---|---|
0 | None |
|
1 | Comma | value1,value2,value3 |
2 | Semicolon | value1;value2;value3 |
3 | Tab | value1 value2 value3 |
4 | Hypen | value1-value2-value3 |
CSV Encapsulation
The CSV delimiter will only be used if the file format is set to CSV. This setting determines how the CSV is encapsulated.
Number | Type | Example |
|---|---|---|
0 | None |
|
1 | Double Quote | “value1” |
2 | Single Quote | 'value1' |
3 | Tab | value1 , value2 |
Path
This setting is used when configuring a connection for either FTP, SFTP, SFTPS or Dropbox.
When using the path the user can specify, in the directory when the file needs to be saved.
Send ZIP (recommended)
For the send ZIP, this allows the user to put a password on the file.
If send_zip is true then to add a password you will have to fill in the ZIP Password configuration to set it.
If set to no, no password will be needed to be set.
Notification Emails
This is where a user can set the emails of the users who need to receive a notification.
If the connection is Email the email addresses in the notification emails will receive the download link or the file.
If set to any other connection the user will receive an email informing them that the file was sent with the lead ids.
To add multiples it needs to be formatted like this:
BCC Emails
This is the same as notification emails but when sent the email they will be in BCC:
Schedule
Batch Hour (mandatory) is the hour in which the file integration will commence, eg. if it is set to 11 this will compile all leads before 11 only so for instance if the lead came in at 11:01 it would not be included in the file being sent.
Days of the week(1 selection mandatory): the days of the week when the file will be sent.
Connections
This connection type will send an email to the email configured in the notification emails section.
The user will receive an email like this for example:
FTP & FTPS
These connection types allow the user of the platform to place the file onto an FTP and FTPS Server.
The below details will have to be sent from the customer who is receiving the data.
Username
Password
Server is the URL on the FTP/FTPS server
Port the number of the port on the server
Passive - this can be set to true or false
SFTP
This connection type allows the user of the platform to place the file onto an SFTP Server.
The below details will have to be sent from the customer who is receiving the data.
Username
Password
Server is the URL on the FTP/FTPS server
Port the number of the port on the server
Dropbox
This connection type allows the user of the platform to place the file on to a Dropbox account.
Here there are two configurations that need to be applied.
API token: this is generated by going to the top of the configuration and clicking Get Dropbox Token:
Once clicked, you will have to grant access
Once granted the user will then be displayed their access token, this is what the users will need to copy and paste in the configuration.
Access Type
app Folder Access - the user will only have access on a folder level
Full Access - Full Admin rights
Example
For all of the connections below, when the integration is complete the configured email will receive an email such as below which will not include a download link just the file that has been sent and to where.
FTP
SFTP
FTPS
Dropbox