Syncrofy File Transfer Visibility (FTV) tracks file transfers by breaking them into three key stages:
- Receive – File is received by the system.
- Processing – File is processed (e.g. unzipped, validated).
- Send/Delivery – File is delivered to the final destination.
Each stage includes events that provide status updates and details about the file transfer.
¶ How It Works
A single transfer is tracked across multiple events using a shared ARRIVEDFILE_KEY. Each event for the same transfer must include the same ARRIVEDFILE_KEY.
Each event has a unique EVENT_KEY, a timestamp (TIME), and metadata about the event source.
¶ Typical Event Flow
Here's a sample flow of a full file transfer:
| Stage | Event | Required | Purpose |
|---|---|---|---|
ARRIVED_FILE |
StartTransfer |
Required | Initiates the transfer, includes sender info |
ARRIVED_FILE |
ProcessDetails |
Optional | Details about the received file |
PROCESSING |
ProcessDetails |
Optional | File processing steps (e.g. checksum, normalization) |
DELIVERY |
StartedDelivery |
Required | Delivery begins |
DELIVERY |
ProcessDetails |
Optional | Delivery-specific details |
DELIVERY |
CompleteDelivery |
Required | File successfully delivered |
ARRIVED_FILE |
CompleteTransfer |
Required | Marks the transfer as complete |
¶ Event Data Structure
Each event has standard data describing the event itself:
"STAGE": "ARRIVED_FILE",
"Event": "StartTransfer",
"TIME": "1745964341771",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429185005401657002"
STAGE tells us which of the stages this event will provide data for.
Event tells us the type of action in the stage.
ARRIVEDFILE_KEY is an ID used to link all events for the same transfer together. All events for the same transfer must have the same ARRIVEDFILE_KEY.
EVENT_KEY is a unique ID for each event. Events for the same delivery should all have the same event_key.
TIME is ms Unix timestamp of each event action.
EVENT_SOURCE_NAME, EVENT_SOURCE_URL, EVENT_SOURCE_TYPE provide reference information to the event source. This is information about the event generator (not the protocol transfer). Set the name to include test/prod and the node ID. The URL can be the URL to SFG.
Let's say you have test and prod SFG and each one has 3 nodes in the cluster. You would use the NAME to say "SFG TEST Node 1" or "SFG Prod Node 3". The URL should correspond to that node and point to the SFG admin login URL. All events from that node have that same info. It is unrelated to the protocol transfer happening.
Each event also has event specific fields based on that event type. For example, the StartTransfer event tells us all about the sender and file.
"ProducerUserId": "LP_Producer",
"ProducerMailboxPath": "/LP_Producer/Outbound",
"ProducerFileSize": "1249",
"ProducerFilename": "payments_21360685.dat",
"ProducerName": "LP_Producer",
"ProducerOperation": "PUT",
"ProducerPattern": "PUSH",
"ProducerRemoteHost": "172.172.51.6",
"ProducerPort": "22",
"ProducerProtocol": "SFTP",
"Direction": "inbound",
"Status": "SUCCESS",
"ProducerUserId": "LP_Producer",
Direction is INBOUND for an end-to-end transfers from a partner or OUTBOUND for and end-to-end transfers to a partner.
Status can be any of Unknown, Success, Failed, Retry, InProgress.
ProducerOperation can be any text string and should indicate the specific protocol operation used to transfer the file. For example "PUT" indicates an ftp/sftp connection was initiated by the client and delivered this file into the server.
ProducerFileSize is the file size in bytes of the file.
ProducerFilename is the file name of the file.
ProducerRemoteHostname is the hostname of the remote system that the file originated from.
ProducerProtocol indicates the protocol used to initiate the file transfer. Valid values are: AS2, SFTP, FTP, FTPS, HTTP, HTTPS, CD, MBX, Mailbox, DB, FS, Bus, AWS, Email, API, RN, OFTP, Other.
ProducerPattern indicates if the producer or the gateway initiated the connection. Use PUSH for remote initiated and PULL for gateway initiated.
Additional fields vary by event type.
¶ Sample: StartTransfer Event
Provides details about the sender and the received file:
{
"STAGE": "ARRIVED_FILE",
"Event": "StartTransfer",
"TIME": "1745964341771",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429185005401657002",
"ProducerUserId": "LP_Producer",
"ProducerMailboxPath": "/LP_Producer/Outbound",
"ProducerFileSize": "1249",
"ProducerFilename": "payments_21360685.dat",
"ProducerName": "LP_Producer",
"ProducerOperation": "Put",
"ProducerRemoteHost": "172.172.51.6",
"ProducerPort": "22",
"ProducerProtocol": "SFTP",
"ProducerPattern": "PUSH",
"Direction": "inbound",
"Status": "SUCCESS",
"ProducerUserId": "LP_Producer",
"ProducerTransferSpeedBPS": "200",
"EVENT_SOURCE_NAME": "System name",
"EVENT_SOURCE_URL": "http://your.source.url",
"EVENT_SOURCE_TYPE": "SFG"
}
¶ Sample: Processing Events
Used to track intermediate steps like validation, compression, or transformation.
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"TIME": "1745964365871",
"EVENT_KEY": "20250429186206011657005",
"LayerType": "Normalize Data",
"LayerFilename": "Test1.txt",
"LayerStatus": "Success",
"LayerMessage": "Message data is ready",
"EVENT_SOURCE_NAME": "COE Lab SFG System",
"EVENT_SOURCE_URL": "http://cs-sandbox2.coenterprise.com",
"EVENT_SOURCE_TYPE": "SFG"
}
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"TIME": "1745964365871",
"EVENT_KEY": "20250429186206011657006",
"LayerType": "Generate Checksum",
"LayerFilename": "Test2.txt",
"LayerStatus": "Success",
"LayerMessage": "Checksum: af69451cda314fb49d06a165bd397b28",
"EVENT_SOURCE_NAME": "COE Lab SFG System",
"EVENT_SOURCE_URL": "http://cs-sandbox2.coenterprise.com",
"EVENT_SOURCE_TYPE": "SFG"
}
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"TIME": "1745964365871",
"EVENT_KEY": "20250429186206011657007",
"LayerType": "Validate Checksum",
"LayerFilename": "Test3.txt",
"LayerStatus": "Success",
"LayerMessage": "Validation complete",
"EVENT_SOURCE_NAME": "COE Lab SFG System",
"EVENT_SOURCE_URL": "http://cs-sandbox2.coenterprise.com",
"EVENT_SOURCE_TYPE": "SFG"
}
¶ Sample: Delivery Events
Track the delivery process and outcome.
A delivery consists of at least a StartedDelivery event and a CompleteDelivery event and can include ProcessDetails and FailedDelivery events.
Events for the same delivery should all have the same EVENT_KEY. If sending events for multiple deliveries, use a different EVENT_KEY for each separate delivery.
¶ Sample: Start Delivery
Reports the intent to deliver and provides transfer details:
{
"STAGE": "DELIVERY",
"Event": "StartedDelivery",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429186206011657004",
"TIME": "1745964365871",
"ConsumerName": "LP_Consumer",
"ConsumerContentType": "application/x-octet",
"ConsumerFilename": "payments_21360685.dat",
"ConsumerDocumentId": "4099021968389ad5anode1",
"ConsumerFileSize": "1249",
"ConsumerOperation": "Put",
"ConsumerPattern": "PUSH",
"ConsumerRemoteHost": "cs-sandbox2.coenterprise.com",
"ConsumerProtocol": "SFTP",
"ConsumerUserId": "sftp_user",
"ConsumerPort": "22",
"ConsumerPath": "/LP_Consumer/Inbox",
"EVENT_SOURCE_NAME": "COE Lab SFG System",
"EVENT_SOURCE_URL": "http://cs-sandbox2.coenterprise.com",
"EVENT_SOURCE_TYPE": "SFG"
}
¶ Sample: Delivery Processing Step
Adds a processing step to the delivery:
{
"STAGE": "DELIVERY",
"Event": "ProcessDetails",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429186206011657004",
"TIME": "1745964356440",
"LayerType": "ZIP",
"LayerFilename": "payments_21360685.zip",
"EVENT_SOURCE_NAME": "COE Lab SFG System",
"EVENT_SOURCE_URL": "http://cs-sandbox2.coenterprise.com",
"EVENT_SOURCE_TYPE": "SFG"
}
¶ Sample: Failed Delivery
Marks the delivery as failed with an error message:
{
"STAGE": "DELIVERY",
"Event": "FailedDelivery",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429186206011657004",
"TIME": "1745964399904",
"ConsumerFilename": "payments_21360685.dat",
"ErrorMessage": "Couldn't resolve host cs-sandbox.coenterprise.com",
"ConsumerOperation": "Put",
"ConsumerRemoteHost": "cs-sandbox2.coenterprise.com",
"ConsumerUserId": "sftp_user",
"ConsumerPort": "22",
"ConsumerPath": "/LP_Consumer/Inbox",
"ConsumerProtocol": "SFTP",
"Direction": "outbound"
"EVENT_SOURCE_NAME": "COE Lab SFG System",
"EVENT_SOURCE_URL": "http://cs-sandbox2.coenterprise.com",
"EVENT_SOURCE_TYPE": "SFG",
}
¶ Sample: Complete Delivery
Marks the delivery as completed:
{
"STAGE": "DELIVERY",
"Event": "CompleteDelivery",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429186206011657004",
"TIME": "1745964399994",
"ConsumerFilename": "payments_21360685.dat",
"ConsumerOperation": "Put",
"ConsumerRemoteHost": "cs-sandbox2.coenterprise.com",
"ConsumerProtocol": "SFTP",
"Direction": "outbound"
"ConsumerTransferSpeedBPS": "400",
"ConsumerUserId": "sftp_user",
"ConsumerPort": "22",
"ConsumerPath": "/LP_Consumer/Inbox",
"EVENT_SOURCE_NAME": "COE Lab SFG System",
"EVENT_SOURCE_URL": "http://cs-sandbox2.coenterprise.com",
"EVENT_SOURCE_TYPE": "SFG",
}
¶ Sample: Final Transfer Completion (Success)
Marks the entire transfer as completed. Use this to indicate the transfer is successful and no additional events are expected. The MESSAGE field is optional.
{
"STAGE": "ARRIVED_FILE",
"Event": "CompleteTransfer",
"TIME": "1745964341771",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429185005401657002",
'MESSAGE': 'Transfer Successful",
"EVENT_SOURCE_NAME": "System name",
"EVENT_SOURCE_URL": "http://your.source.url",
"EVENT_SOURCE_TYPE": "SFG"
}
¶ Sample: Final Transfer Completion (Failed)
Marks the entire transfer as failed. Use this to indicate the transfer is failed and no additional events are expected. This is an alternative to CompleteTransfer.
{
"STAGE": "ARRIVED_FILE",
"Event": "FailTransfer",
"TIME": "1745964341771",
"ARRIVEDFILE_KEY": "20250429185005401657002",
"EVENT_KEY": "20250429185005401657002",
'ERROR_MESSAGE': 'Exceeded Retries",
"EVENT_SOURCE_NAME": "System name",
"EVENT_SOURCE_URL": "http://your.source.url",
"EVENT_SOURCE_TYPE": "SFG"
}
¶ Sample: Optional Processing Events
{
"ARRIVEDFILE_KEY": "20250429185005401657002",
"Event": "PROCESSING",
"STAGE": "ARRIVED_FILE",
"TIME": "1745964341771",
"EVENT_KEY": "20250429185005401657002",
"LayerType": "Normalize Data",
"LayerFilename": "Test1.txt",
"LayerStatus": "Success",
"LayerMessage": "Message data is ready"
"EVENT_SOURCE_NAME": "System name",
"EVENT_SOURCE_URL": "http://your.source.url",
"EVENT_SOURCE_TYPE": "SFG"
}
{
"ARRIVEDFILE_KEY": "20250429185005401657002",
"Event": "PROCESSING",
"STAGE": "PROCESSING",
"TIME": "1745964341772",
"EVENT_KEY": "20250429185005401657002",
"LayerType": "Normalize Data",
"LayerFilename": "Test1.txt",
"LayerStatus": "Success",
"LayerMessage": "Message data is ready"
"EVENT_SOURCE_NAME": "System name",
"EVENT_SOURCE_URL": "http://your.source.url",
"EVENT_SOURCE_TYPE": "SFG"
}
{
"ARRIVEDFILE_KEY": "20250429185005401657002",
"Event": "PROCESSING",
"STAGE": "DELIVERY",
"TIME": "1745964341773",
"EVENT_KEY": "20250429185005401657002",
"LayerType": "Normalize Data",
"LayerFilename": "Test1.txt",
"LayerStatus": "Success",
"LayerMessage": "Message data is ready"
"EVENT_SOURCE_NAME": "System name",
"EVENT_SOURCE_URL": "http://your.source.url",
"EVENT_SOURCE_TYPE": "SFG"
}
¶ Submitting Events
Events can be submitted:
• Individually or in batches
• Via HTTPS POST
• Kafka support is coming soon
¶ HTTPS
HTTPS event upload to: https://events.prod.syncrofy.com:443
Hosted on IBM cloud with dynamic DNS and IPs change periodically.
If you need a static endpoint, this static proxy is available.
https://nginx-api.prod.syncrofy.com:443/events/
https://52.118.189.238:443/events/
For firewall setup, this is considered REST not standard user web traffic.
The endpoint is used only for HTTPS POST.
¶ Kafka support
Coming soon.
¶ Event Batch Size
When sending events to Syncrofy in a batch, keep batches to a "reasonable" size to make debug and handler easier. Less than 1000 events per batch and less than 1 megabyte per batch is a good place to start.
¶ Partner and System Identification
Syncrofy keeps an inventory of Partners and Systems and associates each transfer with a sender and receiver that is either a partner or system.
Syncrofy automatically identifies Partners and Systems by matching the protocol, remote address and login credential from the event records to the same data in the Partner and System inventory. If a match is found, the new event is associated with the matching Partner or System. If no match is found, a new Partner is created in the inventory.
¶ Full Event Sample
Following is a complete event upload for sample file transfer.
[
{
"Event": "StartTransfer",
"STAGE": "ARRIVED_FILE",
"TIME": "1756415876001",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"EVENT_KEY": "20250507185005401622888",
"ProducerUserId": "zombie_user",
"ProducerPath": "/zombie/Outbound",
"ProducerFileSize": "1249",
"ProducerFilename": "zombie_407.zip",
"ProducerOperation": "Put",
"ProducerRemoteHost": "172.172.51.6",
"ProducerPort": "22",
"ProducerProtocol": "SFTP",
"ProducerPattern": "PUSH",
"Direction": "inbound",
"Status": "SUCCESS",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
},
{
"STAGE": "ARRIVED_FILE",
"Event": "ProcessDetails",
"TIME": "1756415876002",
"EVENT_KEY": "20250507185005401622888",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"LayerType": "ZIP",
"LayerFilename": "zombie_407.pgp",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
},
{
"STAGE": "ARRIVED_FILE",
"Event": "ProcessDetails",
"TIME": "1756415876003",
"EVENT_KEY": "20250507185005401622888",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"LayerType": "PGP",
"LayerFilename": "zombie_407.dat",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
},
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"TIME": "1756415876004",
"EVENT_KEY": "20250507185005401622888",
"LayerType": "Normalize Data",
"LayerFilename": "data.txt",
"LayerStatus": "Success",
"LayerMessage": "Message data is ready"
},
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"TIME": "1756415876005",
"EVENT_KEY": "20250507185005401622888",
"LayerType": "Generate Checksum",
"LayerFilename": "data.txt",
"LayerStatus": "Success",
"LayerMessage": "Checksum: af69451cda314fb49d06a165bd397b28"
},
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"TIME": "1756415876006",
"EVENT_KEY": "20250507185005401622888",
"LayerType": "Validate Checksum",
"LayerFilename": "data_valid.txt",
"LayerStatus": "Success",
"LayerMessage": "Checksum is valid"
},
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"TIME": "1756415876007",
"EVENT_KEY": "20250507185005401622888",
"LayerType": "Store Checksum",
"LayerFilename": "data_valid.txt",
"LayerStatus": "Success",
"LayerMessage": "Checksum storage was successful"
},
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"TIME": "1756415876008",
"EVENT_KEY": "20250507185005401622888",
"LayerType": "Add to Archive",
"LayerFilename": "data_valid.txt",
"LayerStatus": "Success",
"LayerMessage": "Payload added to archive"
},
{
"STAGE": "PROCESSING",
"Event": "PROCESSING",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"TIME": "1756415876009",
"EVENT_KEY": "20250507185005401622888",
"LayerType": "Validate Archive",
"LayerFilename": "data_valid.txt",
"LayerStatus": "Success",
"LayerMessage": "Archive valid"
},
{
"STAGE": "DELIVERY",
"Event": "StartedDelivery",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"EVENT_KEY": "20250507185005401622888",
"TIME": "1756415876010",
"ConsumerName": "LoansApp",
"ConsumerFilename": "report_zombie.dat",
"ConsumerFileSize": "1279",
"ConsumerOperation": "Put",
"ConsumerRemoteHost": "sftp.loans.company.com",
"ConsumerPath": "/Loans/Inbox",
"ConsumerProtocol": "SFTP",
"ConsumerUserId": "sftp_user",
"ConsumerPort": "22",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
},
{
"STAGE": "DELIVERY",
"Event": "ProcessDetails",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"EVENT_KEY": "20250507185005401622888",
"TIME": "1756415876011",
"LayerType": "ZIP",
"LayerFilename": "report_zombie.zip",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
},
{
"STAGE": "DELIVERY",
"Event": "ProcessDetails",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"EVENT_KEY": "20250507185005401622888",
"TIME": "1756415876012",
"LayerType": "PGP",
"LayerFilename": "report_zombie.pgp",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
},
{
"STAGE": "DELIVERY",
"Event": "CompleteDelivery",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"EVENT_KEY": "20250507185005401622888",
"TIME": "1756415876013",
"ConsumerFilename": "report_zombie_2048.dat",
"ConsumerOperation": "Put",
"ConsumerRemoteHost": "sftp.loans.company.com",
"ConsumerPath": "/Loans/Inbox",
"Mailbox_MailboxPath": "/Loans/Inbox",
"ConsumerProtocol": "SFTP",
"ConsumerUserId": "sftp_user",
"ConsumerPort": "22",
"Direction": "outbound",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
},
{
"STAGE": "ARRIVED_FILE",
"Event": "CompleteTransfer",
"ARRIVEDFILE_KEY": "20250507185005401622888",
"EVENT_KEY": "20250507185005401622888",
"TIME": "1756415876014",
"EVENT_SOURCE_NAME": "Lab SFG System",
"EVENT_SOURCE_URL": "http://gateway.company.com",
"EVENT_SOURCE_TYPE": "SFG"
}
]