DataPipe
DataPipe as a Backend service
The DataPipe service is a high-performance data processing service written in Golang. It is responsible for consuming data from topics with the pattern example.com/+/pepeunit, filtering it, and then saving it into ClickHouse and PostgreSQL.
DataPipe as a data processing paradigm
DataPipe is a data processing instruction described in YML format and is designed for efficient accumulation of data from UnitNode topics.
In Pepeunit, there is a deep integration with Grafana to visualize data collected by DataPipe.