Prepare 0.3.2

[zhicwu]

• more tests for ClickHouseInputStream
• use separate thread pool for I/O(instead of commonPool)
• remove log4j dependency
• new options
  • clickhouse-http-client: http_connection_provider defaults to HTTP_URL_CONNECTION, can be changed to HTTP_CLIENT in JDK 11 or above
  • clickhouse-jdbc: customTypeMappings to define mapping between ClickHouse data type and Java class
• correct behaviour of handling named parameters
  • select :p1(DateTime64(3)) p1, :p2 p2, :p1 p3 has only two parameters: p1 and p2
  • select ? p1, ? p2, ? p3 has 3 parameters
• optimize LZ4 input and output stream for memory effciency
• add new data type Bool(instead of treating it as alias to Int8)
• only show functions under system schema
• enhance prepared statement for below two use cases
  • insert into ... values
  • insert into ... format ...
• support more AggregateFunction types

[github-actions]

Benchmark                                (client)  (connection)  (statement)   (type)   Mode  Cnt     Score    Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20   245.516 ± 29.278  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20   239.286 ± 25.687  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20   237.156 ± 27.516  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20   235.730 ± 30.784  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20   247.038 ± 28.548  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20   246.739 ± 28.367  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20   248.922 ± 31.022  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20   244.815 ± 28.263  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20  1050.045 ± 94.970  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20  1012.921 ± 76.739  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20  1065.005 ± 65.120  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20  1055.477 ± 53.199  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20   738.557 ± 52.523  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20   707.033 ± 54.532  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20   711.309 ± 76.160  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20   731.522 ± 56.784  ops/s

[enqueue]

Bear in mind that a JDBC connection pool might create some dozens of connections ahead of time for reuse. If each of these connections spawns another threadpool, we might run into issues.

[zhicwu]

Bear in mind that a JDBC connection pool might create some dozens of connections ahead of time for reuse. If each of these connections spawns another threadpool, we might run into issues.

max_threads_per_client defaults to zero, but when it's configured with a number greater than zero, each client will have its own thread pool, instead of reusing ClickHouseClient.defaultExecutor. Maybe keep alive timeout should be set for core pool as well to avoid the situation of having too many idle threads.

[github-actions]

Benchmark                                (client)  (connection)  (statement)   (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20   287.056 ±  35.974  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20   275.543 ±  28.798  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20   272.975 ±  24.080  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20   274.608 ±  29.392  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20   281.707 ±  35.551  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20   284.496 ±  31.182  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20   284.215 ±  31.433  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20   286.415 ±  30.166  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20  1626.552 ±  77.295  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20  1575.407 ±  81.022  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20  1628.521 ±  57.435  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20  1589.268 ±  62.086  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20  1104.106 ±  73.055  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20  1066.350 ±  80.787  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20  1077.503 ± 100.513  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20  1020.707 ±  94.470  ops/s

[github-actions]

Benchmark                                (client)  (connection)  (statement)   (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20   279.993 ±  30.792  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20   267.521 ±  27.470  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20   247.821 ±  45.978  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20   268.745 ±  27.913  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20   281.262 ±  33.734  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20   283.223 ±  33.768  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20   280.931 ±  36.562  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20   279.858 ±  33.696  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20  1599.161 ± 100.538  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20  1594.226 ±  62.095  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20  1647.391 ±  75.005  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20  1581.111 ±  66.437  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20  1066.236 ±  96.858  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20  1077.183 ±  72.847  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20  1084.858 ± 103.049  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20  1066.591 ±  85.452  ops/s

[github-actions]

Benchmark                                (client)  (connection)  (statement)   (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20   280.841 ±  30.689  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20   273.023 ±  31.249  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20   273.944 ±  28.472  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20   269.603 ±  27.688  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20   282.478 ±  35.168  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20   286.499 ±  30.990  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20   284.089 ±  31.715  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20   285.575 ±  29.491  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20  1625.814 ±  92.580  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20  1586.369 ±  82.836  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20  1648.988 ±  71.028  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20  1637.752 ±  75.794  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20  1017.791 ± 149.155  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20  1088.756 ±  84.440  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20  1121.829 ±  78.587  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20  1113.738 ± 105.829  ops/s

[github-actions]

Benchmark                                (client)  (connection)  (statement)   (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20   266.786 ±  31.967  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20   271.687 ±  29.301  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20   266.762 ±  25.757  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20   266.166 ±  27.591  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20   281.392 ±  33.433  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20   280.805 ±  28.723  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20   277.415 ±  32.561  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20   281.632 ±  31.947  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse       normal  default  thrpt   20  1584.398 ± 140.062  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  default  thrpt   20  1555.086 ±  74.058  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new       normal  default  thrpt   20  1594.057 ±  94.675  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new     prepared  default  thrpt   20  1603.031 ±  58.615  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  default  thrpt   20  1062.549 ±  77.554  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  default  thrpt   20  1046.488 ±  92.378  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new       normal  default  thrpt   20  1095.806 ± 112.619  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  default  thrpt   20  1091.176 ±  83.580  ops/s