feat(kafka): add support for Confluence Producers

[leandrodamascena]

Issue number:

Summary

Changes

This is an important fix to support customers using Confluent schema integration on the producer side of Protobuf messages. This only applies for Protobuf and not for Avro.

According to the wire format of the Confluent serializers message-indices are injected as additional bytes after magic bytes and schema id:

magic-byte, schema-id, message-indexes, protobuf-payload

We need to strip these message-indexes before attempting to parse the data using the plain Google protobuf library. This is what this PR is doing:

1. Attempt to parse using plain protobuf
2. If it fails
   1. Attempt to strip message-indexes and parse again
3. If it still fails we have some corrupted data or a schema mismatch and raise an exception

User experience

Please share what the user experience looks like before and after this change

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.20%. Comparing base (fe9bdcb) to head (25c66e9).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #6833   +/-   ##
========================================
  Coverage    96.19%   96.20%           
========================================
  Files          273      273           
  Lines        12626    12641   +15     
  Branches       938      940    +2     
========================================
+ Hits         12146    12161   +15     
  Misses         375      375           
  Partials       105      105           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud