Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What solutions exist to manage data movement efficiently and effectively across multiple databases?


There's AWS Glue and Azure Data Factory, integration tools like Qlik Replicate or FiveTran Replication, or ConnX solutions if you have enterprise-scale money to burn.

We don't use any of them though, we have custom C# ingestion code that can talk to most databases and Odata API's. .Net's lazy collections make it quite easy to create a consumer/producer pipeline for streaming large datasets without too much overhead.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: