Everflow

Use the Everflow connector to sync conversion events, click-level data, aggregated entity reports, and dimension tables (affiliates, offers, advertisers, campaigns) into your Athena lakehouse. This is a Blotout custom connector built on the Airbyte Connector Builder (declarative manifest).

Prerequisites

• A tag with Blotout Cloud set up (green checkmark in the Tags table)
• An Everflow Network API key — generate it in Everflow under Control Center → Security → API keys. Must be a Network key (not affiliate, advertiser, or marketplace). The key is shown once at creation.
• Know your region — standard networks use https://api.eflow.team/v1; EU-hosted networks use https://api-eu.eflow.team/v1

Caution

Everflow issues four key types — network, affiliate, advertiser, marketplace — and each one only reaches its own path prefix. Only a network key can read /v1/networks/* endpoints. Using the wrong key type will fail the connection check.

Create the pipeline

Follow the shared Create a pipeline steps until the source catalogue appears, then select Everflow.

Source configuration

On the Source step, configure the connector:

Field	Purpose
Schedule Type	How the pipeline runs — typically Scheduled
Sync Frequency	How often Airbyte syncs (for example, Every 24 hours)
Source Name	Name for this pipeline — used in the Athena schema <SOURCE_NAME>_<ENV>

Authentication (API Key)

Every request carries a single header — X-Eflow-API-Key. No OAuth flow is available for the Network API; API keys over HTTPS are the documented mechanism. Keys are long-lived with no documented expiry. Rotating a key means creating a new one in Everflow and editing the source — cursor state is untouched, no re-sync is needed.

1. Under Everflow Configuration, paste your Network API Key into the API Key field.

Configuration fields

Field	Type	Required	Default	Description
api_key	secret string	Yes	—	Everflow Network API key. Sent as the X-Eflow-API-Key header. Generate under Control Center → Security → API keys.
start_date	string	Yes	—	Earliest date to pull (YYYY-MM-DD). Conversion and click reporting is limited to the prior 365 days — an earlier start date returns nothing for out-of-range slices.
end_date	string	No	—	Latest date to pull (YYYY-MM-DD). Leave blank to always pull through now (recommended for scheduled syncs). Set only for a bounded historical backfill.
timezone_id	integer	No	67	Everflow timezone id for the report window. 67 = UTC. Response timestamps are always UTC epoch seconds, so non-UTC values make the cursor and request window disagree by the offset. Change only to match the network’s reporting timezone, and raise lookback_days to compensate. Full list: GET /meta/timezones.
currency_id	string	No	"USD"	Currency for payout/revenue amounts in conversions and entity_report (ISO code). Full list: GET /meta/currencies.
page_size	integer	No	500	Page size for conversions and the affiliates/offers/advertisers table endpoints (max 2000). Ignored by clicks and entity_report, which are unpaginated.
report_step_days	integer	No	7	Size of each conversions time slice. Smaller windows mean more requests but smaller responses; the API allows up to a 1-year range per call.
clicks_step_hours	integer	No	24	Size of each clicks time slice. The click endpoint returns at most 10,000 rows per request and truncates silently beyond that. If a network does more than 10k clicks/day, lower this to 6 or 1 and re-backfill.
lookback_days	integer	No	1	Days to rewind before the stored cursor on each incremental sync. Covers conversions that are approved, scrubbed, or adjusted after the fact. Records are appended — deduplicate latest-per-key downstream.
entity_columns	array	No	["date", "offer", "affiliate"]	Columns to group the aggregated entity_report by. Common values: date, hour, offer, affiliate, advertiser, campaign, sub1–sub10, source_id, country, platform, device_type. High-cardinality columns count against a separate 1,000 queries/hour granular-report quota.
include_events	boolean	No	false	When true, sets show_events on the conversion report. Events arrive in the same array as conversions with is_event = true — filter on that downstream.
base_url	string	No	"https://api.eflow.team/v1"	Override for EU-hosted networks. Default is https://api.eflow.team/v1 — EU networks use https://api-eu.eflow.team/v1.

Supported sync modes

The Everflow source connector supports the following sync modes:

• Incremental Sync — Append

Note

Conversions mutate after the fact (pending → approved → scrubbed, payout adjustments), which is exactly what lookback_days re-fetches. The same key can land more than once — always deduplicate latest-per-key downstream using _airbyte_extracted_at.

Supported streams

Stream	Endpoint	Grain	Cursor Field	Primary Key
conversions	POST /networks/reporting/conversions	Per conversion	conversion_unix_timestamp	conversion_id
clicks	POST /networks/reporting/clicks/stream	Per click	unix_timestamp	transaction_id
entity_report	POST /networks/reporting/entity/table	Day x grouping	report_date	None
affiliates	POST /networks/affiliatestable	Per affiliate	time_saved	network_affiliate_id
offers	POST /networks/offerstable	Per offer	time_saved	network_offer_id
advertisers	POST /networks/advertiserstable	Per advertiser	time_saved	network_advertiser_id
campaigns	GET /networks/campaigns	Per Smart Link	time_saved	network_campaign_id

All 7 streams support Incremental | Append sync mode. conversions, clicks, and entity_report use server-side cursors; affiliates, offers, advertisers, and campaigns use client-side cursors.

How incremental works

• Server-side (conversions, clicks, entity_report): the cursor value is written into the request window, so the API only returns the slice requested.
• Client-side (affiliates, offers, advertisers, campaigns): these endpoints have no date filter and always return the full set. The connector fetches in one pass and drops rows whose time_saved is older than the stored cursor.
• lookback_days (default 1) rewinds the window each run so late corrections are picked up.

Unverified stream

campaigns is the one unverified stream. Everflow’s published response schema lists no fields, so the time_saved cursor is assumed. If the field is missing, the CDK logs “Could not find cursor field” once and syncs every record anyway — the stream degrades to a full snapshot each run. Confirm on the first live run and pin the real cursor if it differs.

conversions

One row per conversion. With include_events = true, post-conversion events arrive in the same array with is_event = true. The relationship object contains offer, affiliate, advertiser, campaign, and account manager blocks — flatten in dbt.

Field	JSON Type	Athena Type	Notes
conversion_id	string	varchar	Primary key
conversion_unix_timestamp	integer	bigint	UTC epoch seconds — cursor field
click_unix_timestamp	integer	bigint
transaction_id	string	varchar
order_id	string	varchar
status	string	varchar
payout	number	double
revenue	number	double
sale_amount	number	double
payout_type	string	varchar
revenue_type	string	varchar
currency_id	string	varchar
is_event	boolean	boolean
event	string	varchar
is_scrub	boolean	boolean
is_view_through	boolean	boolean
error_code	integer	bigint
error_message	string	varchar
coupon_code	string	varchar
email	string	varchar	PII — present only if the network passes it
notes	string	varchar
referer	string	varchar
url	string	varchar
source_id	string	varchar
network_offer_payout_revenue_id	integer	bigint
previous_network_offer_id	integer	bigint
conversion_user_ip	string	varchar	PII
session_user_ip	string	varchar	PII
http_user_agent	string	varchar
platform	string	varchar
os_version	string	varchar
browser	string	varchar
brand	string	varchar
device_model	string	varchar
device_type	string	varchar
carrier	string	varchar
isp	string	varchar
language	string	varchar
country	string	varchar
region	string	varchar
city	string	varchar
dma	integer	bigint
app_id	string	varchar
idfa	string	varchar	PII — device identifier
idfa_md5	string	varchar
idfa_sha1	string	varchar
google_ad_id	string	varchar	PII — device identifier
google_ad_id_md5	string	varchar
google_ad_id_sha1	string	varchar
android_id	string	varchar	PII — device identifier
android_id_md5	string	varchar
android_id_sha1	string	varchar
sub1 – sub10	string	varchar	Sub-tracking parameters
adv1 – adv10	string	varchar	Advertiser-side tracking parameters
relationship	object	struct / json	offer / affiliate / advertiser / campaign / account_manager blocks

clicks

One row per click. No pagination — uses window sizing only. The endpoint returns at most 10,000 rows per request and truncates silently beyond that cap.

Field	JSON Type	Athena Type	Notes
transaction_id	string	varchar	Primary key
unix_timestamp	integer	bigint	UTC epoch seconds — cursor field
is_unique	boolean	boolean
has_conversion	boolean	boolean
tracking_url	string	varchar
url	string	varchar
referer	string	varchar
source_id	string	varchar
payout	number	double
revenue	number	double
payout_type	string	varchar
revenue_type	string	varchar
currency_id	string	varchar
coupon_code	string	varchar
error_code	integer	bigint
error_message	string	varchar
error_filter_id	integer	bigint
project_id	string	varchar
user_ip	string	varchar	PII
is_view_through	boolean	boolean
is_async	boolean	boolean
is_pass_through	boolean	boolean
is_sdk_click	boolean	boolean
is_test_mode	boolean	boolean
server_side_url	string	varchar
server_side_output	string	varchar
custom_landing_page_id	integer	bigint
redirect_method	string	varchar
creative_id	integer	bigint
category_id	integer	bigint
previous_network_offer_id	integer	bigint
idfa	string	varchar	PII — device identifier
idfa_md5	string	varchar
idfa_sha1	string	varchar
google_ad_id	string	varchar	PII — device identifier
google_ad_id_md5	string	varchar
google_ad_id_sha1	string	varchar
android_id	string	varchar	PII — device identifier
android_id_md5	string	varchar
android_id_sha1	string	varchar
sub1 – sub10	string	varchar	Sub-tracking parameters
relationship	object	struct / json	offer / advertiser / affiliate / device_information / geolocation / campaign

entity_report

The aggregated report behind the Everflow UI, one request per day. Rows are structured — columns[] holds the grouping values (in entity_columns order) and reporting{} holds the metrics. report_date is stamped by the connector because the API returns no top-level date.

Field	JSON Type	Athena Type	Notes
report_date	string	varchar	YYYY-MM-DD, stamped from the request window — cursor field
columns	array<object>	array	Grouping values in entity_columns order
custom_metric_columns	array<object>	array
usm_columns	array<object>	array
reporting	object	struct / json	Aggregated metrics

affiliates

Dimension table — the partners promoting offers. Join to conversions.relationship.affiliate or the affiliate id inside entity_report.columns.

Field	JSON Type	Athena Type	Notes
network_affiliate_id	integer	bigint	Primary key
network_id	integer	bigint
name	string	varchar
account_status	string	varchar
account_manager_id	integer	bigint
account_manager_name	string	varchar
account_executive_id	integer	bigint
account_executive_name	string	varchar
today_revenue	string	varchar	Formatted string, e.g. "$1,234.00"
balance	number	double
is_payable	boolean	boolean
payment_type	string	varchar
payment_terms	string	varchar
network_country_code	string	varchar
network_traffic_source_id	integer	bigint
global_tracking_domain_url	string	varchar
last_login	integer	bigint	UTC epoch seconds
time_created	integer	bigint	UTC epoch seconds
time_saved	integer	bigint	UTC epoch seconds — cursor field
labels	array<string>	array
relationship	object	struct / json

offers

Dimension table — what is being promoted, with payout/revenue terms and caps.

Field	JSON Type	Athena Type	Notes
network_offer_id	integer	bigint	Primary key
network_id	integer	bigint
network_advertiser_id	integer	bigint
network_offer_group_id	integer	bigint
name	string	varchar
offer_status	string	varchar
visibility	string	varchar
currency_id	string	varchar
payout_type	string	varchar
revenue_type	string	varchar
default_payout	number	double
default_revenue	number	double
percent_payout	number	double
percent_revenue	number	double
destination_url	string	varchar
preview_url	string	varchar
thumbnail_url	string	varchar
category	string	varchar
today_clicks	integer	bigint
today_revenue	string	varchar
is_caps_enabled	boolean	boolean
daily_conversion_cap	integer	bigint
weekly_conversion_cap	integer	bigint
monthly_conversion_cap	integer	bigint
global_conversion_cap	integer	bigint
daily_payout_cap	number	double
weekly_payout_cap	number	double
monthly_payout_cap	number	double
global_payout_cap	number	double
date_live_until	string	varchar
time_created	integer	bigint	UTC epoch seconds
time_saved	integer	bigint	UTC epoch seconds — cursor field
labels	array<string>	array
relationship	object	struct / json

advertisers

Dimension table — who owns the offers. Also the stream Airbyte’s connection check hits to validate the API key.

Field	JSON Type	Athena Type	Notes
network_advertiser_id	integer	bigint	Primary key
network_id	integer	bigint
name	string	varchar
account_status	string	varchar
account_manager_id	integer	bigint
account_manager_name	string	varchar
sales_manager_id	integer	bigint
sales_manager_name	string	varchar
today_revenue	string	varchar
verification_token	string	varchar
time_created	integer	bigint	UTC epoch seconds
time_saved	integer	bigint	UTC epoch seconds — cursor field
labels	array<string>	array
relationship	object	struct / json

campaigns

Smart Links. Everflow publishes no field list for this payload, so the schema and time_saved cursor are assumed — verify on the first live run.

Field	JSON Type	Athena Type	Notes
network_campaign_id	integer	bigint	Primary key
network_id	integer	bigint
name	string	varchar
campaign_status	string	varchar
time_created	integer	bigint	UTC epoch seconds
time_saved	integer	bigint	UTC epoch seconds — cursor field (assumed)
relationship	object	struct / json

API limits

Limit	Value	How the connector handles it
Request rate	30 req/s, burst 50 (network key)	429 responses carry X-RateLimit-Reset (unix ts); the connector waits until then, up to 5 retries with exponential backoff for 5xx.
Reporting concurrency	10 concurrent /reporting/*	Streams are read sequentially — never approached.
Click response size	10,000 rows, no pagination	One clicks_step_hours window per request (default 24h). Lower to 6 or 1 for high-volume networks.
Entity report size	10,000 rows / 367-day range	One day per request, which also makes report_date exact.
Conversion history	Prior 365 days, 1-year window	report_step_days (default 7) per request; a start_date older than 365 days simply yields empty slices.
Granular report quota	1,000 queries/hour	Only triggered by high-cardinality entity_columns (geo, device, sub1–10). Keep the default grouping unless a dashboard needs more.

Timezones

from/to in the request body are interpreted as midnight in the requested timezone_id, but every *_unix_timestamp in the response is UTC epoch seconds — and that is what the incremental cursor stores. timezone_id = 67 is UTC and keeps the two aligned.

Setting a non-UTC timezone_id makes the stored cursor and the requested window disagree by the offset, which clips records at each slice boundary. If a network insists on local-time reporting, set the id and raise lookback_days to cover the offset.

PII considerations

conversions can carry email, conversion_user_ip, session_user_ip, http_user_agent, and raw or hashed device ids (idfa, google_ad_id, android_id) when the network passes them through. clicks carries user_ip and the same device ids. They are declared in the schema so they land typed rather than silently. Drop or hash them in dbt if the client’s agreement does not cover storing them.

Downstream notes

• Schema: everflow_<client>_prod.*, one table per stream
• conversions.relationship already carries the offer / affiliate / advertiser / campaign blocks, so attribution queries work off that one table. The dimension streams add names, statuses, and payout terms.
• Deduplication example for conversions:

SELECT * FROM (
  SELECT *, ROW_NUMBER() OVER (
    PARTITION BY conversion_id
    ORDER BY _airbyte_extracted_at DESC
  ) AS rn
  FROM everflow_<client>_prod.conversions
) WHERE rn = 1

Table	Partition by
conversions	conversion_id
clicks	transaction_id
entity_report	report_date + the grouping ids from columns[]
affiliates / offers / advertisers / campaigns	Their network_*_id

Setup checklist

• Client creates a Network API key and sends it over a secure channel (shown once)
• Confirm region — EU networks need base_url = https://api-eu.eflow.team/v1
• Local smoke test: cd connectors/source-everflow/v1 && ./test_local.sh <key>
• Publish the image: ./build_and_deploy.sh (or merge to main — CI builds it)
• Airbyte → Settings → Sources → New → Add Docker Connector → blotout/source-everflow, tag 1.0.0
• Create the source with the key + start date; run check and discover
• Create the connection with all 7 streams on Incremental | Append
• First sync: confirm click windows never return exactly 10,000 rows; lower clicks_step_hours if they do
• Confirm entity_report rows carry report_date and the columns / reporting split
• Confirm campaigns records actually contain time_saved
• Point dbt at everflow_<client>_prod and build the dedupe views

Changelog

Version	Date	Description
1.0.0	—	Initial release with 7 streams: conversions, clicks, entity_report, affiliates, offers, advertisers, campaigns

Complete Schema and Summary

Continue through the Schema and Summary steps to choose streams and confirm the connection. Save and run the first sync.

After a successful sync, Airflow creates an organization-scoped DAG for the pipeline, and dbt models feed downstream analytics.

Related documentation

• Data Pipelines overview — Tags-first create flow