-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
master分支
What's Wrong?
比如上个步骤表输入,包含 id,age,name doris输出的目标表 字段顺序是id,name,age。
在序列化的时候,因为age和name的顺序是调换了的,但是他按照record的下标来取数据,实际生成的csv或者json数据age和name没有调换位置。
并且因为name一般包含中文,导致 name的value在进行convertExternal的时候会报 string无法转换为integer。

What You Expected?
在DorisStreamLoader中看到,data.keynrs 记录了正确的顺序,所以在构造DorisRecordSerializer把这个data.keynrs 传进来。然后通过 下面截图的代码进行数据匹配。这才是正确的顺序。

How to Reproduce?
以下为复现步骤:
1 在mysql里创建表:
CREATE TABLE yutao_user
(
id
int NOT NULL COMMENT 'id',
name
varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL COMMENT '姓名',
age
int DEFAULT NULL COMMENT '年龄',
PRIMARY KEY (id
)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci COMMENT='鱼涛用户';
insert 几条数据:
INSERT INTO yutao_user
(id
, name
, age
) VALUES (1, 'Hui Fat', 222);
INSERT INTO yutao_user
(id
, name
, age
) VALUES (2, 'Dong Zhiyuan', 467);
INSERT INTO yutao_user
(id
, name
, age
) VALUES (3, 'Hao Ziyi', 448);
2 在doris里创建表
CREATE TABLE yutao_user
(
id
VARCHAR(48) NOT NULL COMMENT '主键',
name
VARCHAR(32) NULL COMMENT '名称',
age
INT NULL COMMENT '年龄'
) ENGINE=OLAP
UNIQUE KEY(id
)
DISTRIBUTED BY HASH(id
) BUCKETS AUTO
PROPERTIES (
"replication_allocation" = "tag.location.default: 3")
3 表输入写:select id,age,name from yutao_user
4 doris输出里打乱其顺序, 字段匹配为id,name,age
Anything Else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct