79 docs tagged with "Docs"

Column ingestion

Notice:

Notice:

Notice:

This tutorial guides users through the basic operations of creating tables, importing data, and querying data using ByConity in a Docker deployment environment.

Notice:

Release Date: May 24, 2023

Release Date: September 9, 2023

Release Date: December 17, 2023

Release Date: Apirl 15, 2024

Upgrading Clusters Based on S3 Storage (Applicable Only to Versions Before 0.2.0)

Data Caching (Disk Cache)

Document Type: Tutorial

Document Type: Descriptive

Notice:

Complex Query Execution Model

Notice:

ByConity supports codecs for lists, which are methods for compressing and decompressing data aimed at saving storage space and improving I/O performance.

Data Sources

This tutorial aims to guide users on how to export data from tables in ByConity to different file systems or databases, as well as how to export query results. Currently, ByConity supports exporting data to local file systems and HDFS (Hadoop Distributed FileSystem).

Direct Write Method Tuning

Columnar Storage Design Principles

The data types provided in ByConity are adapted from ClickHouse. Visit this page for more information on ClickHouse data types.

Tutorial Objectives:

Notice:

This article demonstrates how to deploy a ByConity cluster on a Kubernetes (k8s) environment.

Currently, the way to install ByConity on physical machines is through package managers, such as installing Debian packages for Debian OS or rpm packages for Centos OS. Since ByConity uses FoundationDB as its metadata storage and HDFS as its data storage, before deploying ByConity, we need to deploy FoundationDB and HDFS first. The steps are to install FoundationDB (FDB) first, then install HDFS, and finally deploy the ByConity software package. Details are as follows.

Software Requirements

Refer to the Admin Guide - Cluster Configuration Parameters

Our developers connect to a Debian 11 development environment running on a virtual machine using a plugin called Dev Container in VS Code. Therefore, we also recommend that you use the same approach for development to avoid many dependency and compatibility issues. For more information, refer to the official dev set.

Importing from Hive External Tables

Importing via MySQL

This tutorial guides users to quickly deploy ByConity locally using Docker. Users can follow the steps below to deploy a simple ByConity cluster for a quick start experience. If you want to fully try and understand ByConity, it is recommended that you skip this section and refer directly to ByConity Cluster Deployment to experience the powerful performance of ByConity.

ClickHouse in OLAP Scenarios and Its Problems

Notice:

Notice:

In addition to supporting access to Hive data using external tables, Byconity also supports access through the External Catalog feature.

CnchHive

Deployment-related

Elastic Scaling

ByConity provides two SQL dialects, (1) ClickHouse and (2) ANSI.

Notice:

Notice:

Notice:

ByConity supports deploying multiple ByConity Server, Resource Manager, and TSO processes within the same cluster to achieve high availability. This is achieved by determining the leader through a "Compare And Swap" operation in the shared FoundationDB.

Dependencies

Importing from HDFS

Importing from Kafka

Full-Text Search

Notice:

Notice:

Notice:

Configuring Background Tasks

map

Function Definition

Notice:

Common Monitoring Metrics

Notice:

MACNumToString

RBAC Model

Introduction

The optimizer is the core of the database system, and an excellent optimizer can greatly improve query performance, especially in complex query scenarios where the optimizer can bring several times to hundreds of times performance improvement.

Notice:

The Resource Manager (RM) component is responsible for the unified management and scheduling of ByConity's computational resources. It is a core component for achieving resource elasticity and improving resource utilization.

ByConity supports containerized and modular deployment, allowing for elastic scaling. It can scale computing resources in real-time and on-demand, ensuring efficient resource utilization. This section describes how to scale ByConity on a Kubernetes cluster.

The supported statements in ByConity are similar to ClickHouse, but it is still recommended to follow the ByConity manual to ensure proper use. Some of the examples below are referenced from ClickHouse Documentation but have been adapted and modified to work in ByConity.

Streaming Data Import (Local Files and In-Memory Data)

Notice:

The table engine determines the way data is organized and stored, the method and type of indexing, which queries are supported, and how they are supported, as well as some other specific features and configurations. In ByConity, the most commonly used table engines are CnchMergeTree, CnchKafka, and CnchMaterializedMySql. This article focuses on explaining the principles of these three table engines.

Technical Architecture

With the increasing amount and complexity of data, enterprises are increasingly turning to OLAP (On-Line Analytical Processing) engines to process large-scale data and deliver real-time analytical insights. Performance is a very important factor when choosing an OLAP engine. Therefore, this article aims to offer valuable insights by conducting a performance comparison among four prominent open-source OLAP engines: ClickHouse, Doris, Presto, and ByConity. To achieve this, we employ a set of 99 query statements sourced from the TPC-DS benchmark. Our aim is to provide reference for enterprises in their journey to select the most fitting OLAP engine for their specific needs.

Transaction overview

Notice:

Introduction

Notice:

Notice:

Document Type: Tutorial

What is ByConity

ByConity supports the standard syntax of window functions. A list of window related features are explained below.