Skip to content

Latest commit

 

History

History
27 lines (19 loc) · 826 Bytes

index.rst

File metadata and controls

27 lines (19 loc) · 826 Bytes

smallpond

Smallpond is a lightweight distributed data processing framework. It uses duckdb as the compute engine and stores data in parquet format on a distributed file system (e.g. 3FS).

Why smallpond?

  • Performance: Smallpond uses DuckDB to deliver native-level performance for efficient data processing.
  • Scalability: Leverages high-performance distributed file systems for intermediate storage, enabling PB-scale data handling without memory bottlenecks.
  • Simplicity: No long-running services or complex dependencies, making it easy to deploy and maintain.
.. toctree::
   :maxdepth: 1

   getstarted
   internals


.. toctree::
   :maxdepth: 3

   api