File data insert using CSVWithNames is failing since v0.3.2-patch10

[rsim]

We are using ClickHouse JDBC driver to import CSV files that include a header line with column names. Here is the JRuby code that we use https:/rsim/mondrian-olap/blob/master/spec/rake_tasks.rb#L377-L380

conn.jdbc_connection.createStatement.write.
  query("INSERT INTO #{table_name}(#{columns_string})").
  format(Java::com.clickhouse.client.ClickHouseFormat::CSVWithNames).
  data(file_path).send

It was working until the version v0.3.2-patch9 but since the version v0.3.2-patch10 it is failing with:

Java::JavaUtilConcurrent::CompletionException: com.clickhouse.client.ClickHouseException: Code: 27. DB::ParsingException: Cannot parse input: expected ',' before: 'id,the_date,the_day,the_month,the_year,day_of_month,week_of_year,month_of_year,quarter\n1,2010-01-01 02:00:00,Friday,January,2010,1,0,1,Q1\n2,2010-01-02 02:00:00,':
Row 1:
Column 0,   name: id,            type: Int32,    ERROR: text "id,the_dat" is not like Int32

: While executing ParallelParsingBlockInputFormat: (at row 1)
. (CANNOT_PARSE_INPUT_ASSERTION_FAILED) (version 22.1.4.1)
, server ClickHouseNode [uri=http://localhost:8123/mondrian_test]com.clickhouse.client.ClickHouseClientBuilder$Agent.handle(com/clickhouse/client/ClickHouseClientBuilder.java:181)
com.clickhouse.client.ClickHouseClientBuilder$Agent.send(com/clickhouse/client/ClickHouseClientBuilder.java:204)
com.clickhouse.client.ClickHouseClientBuilder$Agent.execute(com/clickhouse/client/ClickHouseClientBuilder.java:234)
com.clickhouse.client.ClickHouseRequest$Mutation.execute(com/clickhouse/client/ClickHouseRequest.java:241)
com.clickhouse.client.ClickHouseRequest$Mutation.send(com/clickhouse/client/ClickHouseRequest.java:292)
java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)

It seems that now the CSV format (without the header line) is used instead of CSVWithNames and it complains about the first file line with column headers.

I was browsing the changes between patch9 and patch10 but couldn't identify so far which change could be causing this.

[rsim]

I found with git bisect that this commit 75ddd37 (with many different changes, the corresponding pull request #956) broke this.
/cc @zhicwu

[zhicwu]

Hi @rsim, thanks for the report and sorry for the inconvenience. There's a bad assumption forcing insert to use headless data format, which works for some formats like RowBinary but apparently not others. Will try to fix it today.

[rsim]

@zhicwu I built the JDBC driver from the latest develop branch and tested it. I still get the same error with it:

Java::JavaUtilConcurrent::CompletionException: com.clickhouse.client.ClickHouseException: Code: 27. DB::ParsingException: Cannot parse input: expected ',' before: 'id,the_date,the_day,the_month,the_year,day_of_month,week_of_year,month_of_year,quarter\n1,2010-01-01 02:00:00,Friday,January,2010,1,0,1,Q1\n2,2010-01-02 02:00:00,':
Row 1:
Column 0,   name: id,            type: Int32,    ERROR: text "id,the_dat" is not like Int32

: While executing ParallelParsingBlockInputFormat: (at row 1)
. (CANNOT_PARSE_INPUT_ASSERTION_FAILED) (version 22.1.4.1)
, server ClickHouseNode [uri=http://localhost:8123/mondrian_test]@-1740047039com.clickhouse.client.ClickHouseClientBuilder$Agent.handle(com/clickhouse/client/ClickHouseClientBuilder.java:274)
com.clickhouse.client.ClickHouseClientBuilder$Agent.send(com/clickhouse/client/ClickHouseClientBuilder.java:298)
com.clickhouse.client.ClickHouseClientBuilder$Agent.execute(com/clickhouse/client/ClickHouseClientBuilder.java:351)
com.clickhouse.client.ClickHouseRequest$Mutation.execute(com/clickhouse/client/ClickHouseRequest.java:272)
com.clickhouse.client.ClickHouseRequest$Mutation.send(com/clickhouse/client/ClickHouseRequest.java:326)

[zhicwu]

Sorry @rsim, I thought it's covered by below case but it didn't. Let me fix it now.

https:/ClickHouse/clickhouse-jdbc/blob/94a5e7c2921047bfe907fbdf6aea2feaed3c2417/clickhouse-client/src/test/java/com/clickhouse/client/ClientIntegrationTest.java#L1416-L1420

[rsim]

@zhicwu I just built the driver with this fix from the develop branch and tested that now this bug is fixed.
Thank you.

[zhicwu]

Thanks for confirming. You can download nightly build here after 4pm UTC today.