Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

windows10 ppyoloe plus trt=True 的模型自动压缩报错 #1709

Open
truthsun22 opened this issue Mar 31, 2023 · 2 comments
Open

windows10 ppyoloe plus trt=True 的模型自动压缩报错 #1709

truthsun22 opened this issue Mar 31, 2023 · 2 comments

Comments

@truthsun22
Copy link

系统：windows10
paddle最新版
训练模型为ppyoloe+
导出模型时，不设置trt=True，模型自动压缩不会报错
导出模型时，设置了trt=True，模型自动压缩就会报错。
错误信息：
2023-03-31 15:05:18,030-INFO: devices: gpu
2023-03-31 15:05:30,720-INFO: Selected strategies: ['qat_dis']
2023-03-31 15:05:39,768-INFO: train config.distill_node_pair: ['teacher_conv2d_305.tmp_1', 'conv2d_305.tmp_1', 'teacher_conv2d_309.tmp_0', 'conv2d_309.tmp_0', 'teacher_conv2d_312.tmp_1', 'conv2d_312.tmp_1', 'teacher_conv2d_316.tmp_0', 'conv2d_316.tmp_0', 'teacher_conv2d_319.tmp_1', 'conv2d_319.tmp_1', 'teacher_conv2d_323.tmp_0', 'conv2d_323.tmp_0']
2023-03-31 15:05:40,647-INFO: quant_aware config {'weight_quantize_type': 'channel_wise_abs_max', 'activation_quantize_type': 'moving_average_abs_max', 'weight_bits': 8, 'activation_bits': 8, 'not_quant_pattern': ['skip_quant'], 'quantize_op_types': ['conv2d', 'depthwise_conv2d', 'conv2d_transpose', 'mul', 'matmul', 'matmul_v2'], 'dtype': 'int8', 'window_size': 10000, 'moving_rate': 0.9, 'for_tensorrt': False, 'is_full_quantize': False, 'onnx_format': False, 'quant_post_first': False, 'scale_trainable': True, 'name': 'Distillation', 'loss': 'l2', 'node': [], 'alpha': 1.0, 'teacher_model_dir': 'D:\ai\paddle\models\ppyoloe_plus_crn_l_80e_coco_trt_animal8', 'teacher_model_filename': 'model.pdmodel', 'teacher_params_filename': 'model.pdiparams'}
Adding quant op with weight:|██████████████████████████████████████████| 585/585
Adding OutScale op:|███████████████████████████████████████████████████| 578/578
2023-03-31 15:05:47,042-INFO: quant_aware config {'weight_quantize_type': 'channel_wise_abs_max', 'activation_quantize_type': 'moving_average_abs_max', 'weight_bits': 8, 'activation_bits': 8, 'not_quant_pattern': ['skip_quant'], 'quantize_op_types': ['conv2d', 'depthwise_conv2d', 'conv2d_transpose', 'mul', 'matmul', 'matmul_v2'], 'dtype': 'int8', 'window_size': 10000, 'moving_rate': 0.9, 'for_tensorrt': False, 'is_full_quantize': False, 'onnx_format': False, 'quant_post_first': False, 'scale_trainable': True, 'name': 'Distillation', 'loss': 'l2', 'node': [], 'alpha': 1.0, 'teacher_model_dir': 'D:\ai\paddle\models\ppyoloe_plus_crn_l_80e_coco_trt_animal8', 'teacher_model_filename': 'model.pdmodel', 'teacher_params_filename': 'model.pdiparams'}
Adding quant op with weight:|████████████████████████████████████████| 2275/2275
Adding OutScale op:|█████████████████████████████████████████████████| 1181/1181
2023-03-31 15:07:02,888-INFO: Total iter: 0, epoch: 0, batch: 0, loss: [1.5197452]l2: [1.5197452]
Traceback (most recent call last):
File "D:\ai\paddle\PaddleSlim-develop\example\auto_compression\detection\run.py", line 199, in
main()
File "D:\ai\paddle\PaddleSlim-develop\example\auto_compression\detection\run.py", line 189, in main
ac.compress()
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddleslim\auto_compression\compressor.py", line 593, in compress
self.single_strategy_compress(strategy, config, strategy_idx,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddleslim\auto_compression\compressor.py", line 775, in single_strategy_compress
test_program_info = self._start_train(
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddleslim\auto_compression\compressor.py", line 794, in _start_train
loss = self._exe.run(train_program_info.program,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\executor.py", line 1463, in run
six.reraise(*sys.exc_info())
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\six.py", line 719, in reraise
raise value
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\executor.py", line 1450, in run
res = self._run_impl(program=program,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\executor.py", line 1720, in _run_impl
return self._run_parallel(program,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\executor.py", line 1254, in _run_parallel
tensors = exe.run(fetch_var_names, return_merged)._move_to_list()
RuntimeError: In user code:

File "D:\ai\paddle\PaddleDetection-release-2.5\tools\export_model.py", line 108, in <module>
  main()
File "D:\ai\paddle\PaddleDetection-release-2.5\tools\export_model.py", line 104, in main
  run(FLAGS, cfg)
File "D:\ai\paddle\PaddleDetection-release-2.5\tools\export_model.py", line 73, in run
  trainer.export(FLAGS.output_dir)
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\engine\trainer.py", line 1059, in export
  static_model, pruned_input_spec = self._get_infer_cfg_and_input_spec(
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\engine\trainer.py", line 1019, in _get_infer_cfg_and_input_spec
  input_spec, static_model.forward.main_program,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 734, in main_program
  concrete_program = self.concrete_program
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 537, in concrete_program
  return self.concrete_program_specify_input_spec(input_spec=None)
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 577, in concrete_program_specify_input_spec
  concrete_program, _ = self.get_concrete_program(
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 485, in get_concrete_program
  concrete_program, partial_program_layer = self._program_cache[cache_key]
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 955, in __getitem__
  self._caches[item_id] = self._build_once(item)
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 939, in _build_once
  concrete_program = ConcreteProgram.from_func_spec(
File "<decorator-gen-119>", line 2, in from_func_spec

File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\wrapped_decorator.py", line 26, in __impl__
  return wrapped_func(*args, **kwargs)
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\base.py", line 67, in __impl__
  return func(*args, **kwargs)
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\program_translator.py", line 888, in from_func_spec
  outputs = static_func(*inputs)
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\meta_arch.py", line 58, in forward
  if self.training:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\meta_arch.py", line 68, in forward
  for inp in inputs_list:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 106, in convert_while_loop
  _run_py_while(cond, body, getter, setter)
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 165, in _run_py_while
  body()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\meta_arch.py", line 75, in forward
  outs.append(self.get_pred())
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\yolo.py", line 127, in get_pred
  return self._forward()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\yolo.py", line 87, in _forward
  if self.training:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\yolo.py", line 98, in _forward
  if self.for_mot:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\yolo.py", line 109, in _forward
  if self.return_idx:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\yolo.py", line 112, in _forward
  elif self.post_process is not None:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\architectures\yolo.py", line 117, in _forward
  bbox, bbox_num = self.yolo_head.post_process(
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\heads\ppyoloe_head.py", line 373, in post_process
  if self.exclude_post_process:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\heads\ppyoloe_head.py", line 383, in post_process
  if self.exclude_nms:
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\heads\ppyoloe_head.py", line 387, in post_process
  bbox_pred, bbox_num, _ = self.nms(pred_bboxes, pred_scores)
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\layers.py", line 553, in __call__
  if self.trt and (int(paddle.version.major) == 0 or
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 323, in convert_ifelse
  out = _run_py_ifelse(pred, true_fn, false_fn, get_args, set_args,
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\dygraph\dygraph_to_static\convert_operators.py", line 380, in _run_py_ifelse
  py_outs = true_fn() if pred else false_fn()
File "D:\ai\paddle\PaddleDetection-release-2.5\ppdet\modeling\layers.py", line 559, in __call__
  bbox = bbox.reshape([1, -1, 6])
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\tensor\manipulation.py", line 3658, in reshape
  helper.append_op(
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\layer_helper.py", line 45, in append_op
  return self.main_program.current_block().append_op(*args, **kwargs)
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\framework.py", line 4017, in append_op
  op = Operator(
File "D:\Anaconda3\envs\paddle2.4\lib\site-packages\paddle\fluid\framework.py", line 2858, in __init__
  for frame in traceback.extract_stack():

PreconditionNotMetError: The meta data must be valid when call the mutable data function. (at ..\paddle\phi\core\dense_tensor.cc:111)
  [operator < reshape2 > error]
@panp4n
Copy link

panp4n commented Oct 20, 2023

你好，请问这个问题解决了吗，我也遇到了同样的问题。

@truthsun22
Copy link
Author

你好，请问这个问题解决了吗，我也遇到了同样的问题。
没有解决，始终没有回复

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@truthsun22 @panp4n