Fivetran Automated Data Movement Platform for Secure Data Transformations

It starts with the data sources themselves – Fivetran has 100s of pre-built, fully-managed connectors that support every major department, team, and business. After you authenticate into your source(s), often with one click, the process of Extracting starts.  For all connectors – they begin with an initial historical load of all your data that exists in your source. This initial historical sync is free for any data sources so you can get all the data you need right out of the box. And, all of these connectors (exception of a few file connectors) use CDC (Change Data Capture), which provides low impact, high volume data replication.  

And as your team grows and changes tools, you can add new connectors in just minutes with Fivetran’s out-of-the-box options. None of these processes will require engineering resources as they are all set up code free.  

But it’s not just automation working behind the scene, Fivetran also have a massive team of engineers building and maintaining all of these pipelines. Because APIs are constantly evolving – Fivetran connectors are proactively being updated on the backend by engineering teams to keep up with all new objects, field changes, and more so you and your team don’t have to.  

And that automation continues through our load process. After the initial load, Fivetran will scan for incremental updates using system columns for applications or transaction logs for database sources. As a result, you have the freshest data always available but without having to do any manual intervention – or without reloading all that data every time which can be costly as well as time consuming. 

Behind the scenes, as that data is moving, Fivetran is normalizing the data into schemas in your destination. Fivetran will handle the schema creation process, including data type mapping and underlying table creation. Fivetran will also manage any schema drift that may occur, such as creating net new custom objects from the data source, or simply changing a data type as we read a change in the source.  Again, this level of automation is really what makes Fivetran unique and saves you hundreds of engineering hours every month.  

And Fivetran brings that same automation to transformations. Once loaded into your destination, you can then trigger transformations via dbt. Fivetran has pre-built models for many major connectors and integrated scheduling that triggers only when new data is loaded so your transformations are up to date and ensure your data is business ready asap.