ephemeral404@lemmy.mlM to Data Engineering@lemmy.mlEnglish · 1 year agoData Engineering roadmaplemmy.mlimagemessage-square4fedilinkarrow-up16file-text
arrow-up16imageData Engineering roadmaplemmy.mlephemeral404@lemmy.mlM to Data Engineering@lemmy.mlEnglish · 1 year agomessage-square4fedilinkfile-text
minus-squareAndy@lemmy.mllinkfedilinkEnglisharrow-up1·1 year agoIsn’t Apache Arrow a table format rather than a batch processing tool? You could add SQLMesh as an alternative to dbt.
minus-squareephemeral404@lemmy.mlOPMlinkfedilinkEnglisharrow-up1·1 year agothanks for suggesting sqlmesh. regarding arrow, have you read this - https://arrow.apache.org/docs/format/Columnar.html
Isn’t Apache Arrow a table format rather than a batch processing tool?
You could add SQLMesh as an alternative to dbt.
thanks for suggesting sqlmesh. regarding arrow, have you read this - https://arrow.apache.org/docs/format/Columnar.html