From a879d8292b90becc582d5c0ee1e5edf1072dea10 Mon Sep 17 00:00:00 2001 From: Ivana Kellyer Date: Wed, 8 Oct 2025 08:32:40 +0200 Subject: [PATCH 1/9] ci: Remove toxgen check (#4892) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### Description Removing the Check CI Config step altogether as well as associated parts of the toxgen script (`fail_on_changes`). Added a BIG ALL CAPS WARNING to `tox.ini` instead. Also updated the toxgen readme a bit. Removing the check should be fine because we haven't actually seen cases of people trying to edit `tox.ini` directly -- if this happens in the future it's easy to notice in the PR. If we don't notice it then, we can notice it during the weekly toxgen update. And if don't notice it then, the file simply gets overwritten. 🤷🏻‍♀️ ### The Problem With Checking `tox.ini`: The Long Read In order to check manual changes to `tox.ini` on a PR, we hash the committed file, then run toxgen, hash the result, and compare. If the hashes differ, we fail the check. This works fine as long as there have been no new releases between the two points in time when `tox.ini` was last committed and when we ran the check. This is usually not the case. There are new releases all the time. When we then rerun toxgen, the resulting `tox.ini` is different from the committed one because it contains the new releases. So the hashes are different without any manual changes to the file. One solution to this is always saving the timestamp of the last time `tox.ini` was generated, and then when rerunning toxgen for the purposes of the check, ignoring all new releases past the timestamp. This means any changes we detect were actually made by the user. However, the explicit timestamp is prone to merge conflicts. Anytime `master` has had a toxgen update, and a PR is made that also ran toxgen, the PR will have a merge conflict on the timestamp field that needs to be sorted out manually. This is annoying and unnecessary. (An attempt was made to use an implicit timestamp instead in the form of the commit timestamp, but this doesn't work since we squash commits on master, so the timestamp of the last commit that touched `tox.ini` is actually much later than the change was made. There are also other problems, like someone running toxgen but committing the change much later, etc.) ### Solutions considered - using a custom merge driver to resolve the timestamp conflict automatically (doesn't work on GH PRs) - running toxgen in CI on each PR and committing the change (would work but we're essentially already doing this with the cron job every week) - not checking in `tox.ini` at all, but running toxgen on each PR (introduces new package releases unrelated to the PR, no test setup committed -- contributors and package index maintainers also need to run our tests) - finding a different commit to use as the implicit timestamp (doesn't work because we squash commits on `master`) - ... In the end I decided to just get rid of the check. If people modifying `tox.ini` manually becomes a problem, we can deal with it then. I've added a big warning to `tox.ini` to discourage this. #### Issues Closes https://github.com/getsentry/sentry-python/issues/4886 #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --- .github/workflows/ci.yml | 22 -------- scripts/populate_tox/README.md | 37 +++++++------ scripts/populate_tox/config.py | 5 +- scripts/populate_tox/populate_tox.py | 81 +++++----------------------- scripts/populate_tox/releases.jsonl | 8 +-- scripts/populate_tox/tox.jinja | 22 ++++---- tox.ini | 44 +++++++-------- 7 files changed, 74 insertions(+), 145 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 50ab1d39ce..8ea11db711 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -33,28 +33,6 @@ jobs: pip install tox tox -e linters - check-ci-config: - name: Check CI config - runs-on: ubuntu-latest - timeout-minutes: 10 - - steps: - - uses: actions/checkout@v5.0.0 - with: - ref: ${{ github.event.pull_request.head.sha }} - fetch-depth: 0 - - uses: actions/setup-python@v6 - with: - python-version: 3.12 - - - name: Detect unexpected changes to tox.ini or CI - run: | - pip install -e . - pip install -r scripts/populate_tox/requirements.txt - python scripts/populate_tox/populate_tox.py --fail-on-changes - pip install -r scripts/split_tox_gh_actions/requirements.txt - python scripts/split_tox_gh_actions/split_tox_gh_actions.py --fail-on-changes - build_lambda_layer: name: Build Package runs-on: ubuntu-latest diff --git a/scripts/populate_tox/README.md b/scripts/populate_tox/README.md index 9bdb3567b8..d6c4e52147 100644 --- a/scripts/populate_tox/README.md +++ b/scripts/populate_tox/README.md @@ -14,7 +14,7 @@ combination of hardcoded and generated entries. The `populate_tox.py` script fills out the auto-generated part of that template. It does this by querying PyPI for each framework's package and its metadata and -then determining which versions make sense to test to get good coverage. +then determining which versions it makes sense to test to get good coverage. By default, the lowest supported and latest version of a framework are always tested, with a number of releases in between: @@ -22,17 +22,16 @@ tested, with a number of releases in between: - If the package doesn't have multiple majors, we pick two versions in between lowest and highest. -#### Caveats +Each test suite requires at least some configuration to be added to +`TEST_SUITE_CONFIG` in `scripts/populate_tox/config.py`. If you're adding a new +integration, check out the [Add a new test suite](#add-a-new-test-suite) section. -- Make sure the integration name is the same everywhere. If it consists of - multiple words, use an underscore instead of a hyphen. +## Test suite config -## Defining constraints - -The `TEST_SUITE_CONFIG` dictionary defines, for each integration test suite, -the main package (framework, library) to test with; any additional test -dependencies, optionally gated behind specific conditions; and optionally -the Python versions to test on. +The `TEST_SUITE_CONFIG` dictionary in `scripts/populate_tox/config.py` defines, +for each integration test suite, the main package (framework, library) to test +with; any additional test dependencies, optionally gated behind specific +conditions; and optionally the Python versions to test on. Constraints are defined using the format specified below. The following sections describe each key. @@ -58,7 +57,7 @@ in [packaging.specifiers](https://packaging.pypa.io/en/stable/specifiers.html). ### `package` -The name of the third party package as it's listed on PyPI. The script will +The name of the third-party package as it's listed on PyPI. The script will be picking different versions of this package to test. This key is mandatory. @@ -69,7 +68,7 @@ The test dependencies of the test suite. They're defined as a dictionary of `rule: [package1, package2, ...]` key-value pairs. All packages in the package list of a rule will be installed as long as the rule applies. -`rule`s are predefined. Each `rule` must be one of the following: +Each `rule` must be one of the following: - `*`: packages will be always installed - a version specifier on the main package (e.g. `<=0.32`): packages will only be installed if the main package falls into the version bounds specified @@ -77,7 +76,7 @@ in the package list of a rule will be installed as long as the rule applies. installed if the Python version matches one from the list Rules can be used to specify version bounds on older versions of the main -package's dependencies, for example. If e.g. Flask tests generally need +package's dependencies, for example. If Flask tests generally need Werkzeug and don't care about its version, but Flask older than 3.0 needs a specific Werkzeug version to work, you can say: @@ -176,7 +175,7 @@ be expressed like so: ### `integration_name` Sometimes, the name of the test suite doesn't match the name of the integration. -For example, we have the `openai_base` and `openai_notiktoken` test suites, both +For example, we have the `openai-base` and `openai-notiktoken` test suites, both of which are actually testing the `openai` integration. If this is the case, you can use the `integration_name` key to define the name of the integration. If not provided, it will default to the name of the test suite. @@ -193,6 +192,11 @@ greater than 2, as the oldest and latest supported versions will always be picked. Additionally, if there is a recent prerelease, it'll also always be picked (this doesn't count towards `num_versions`). +For instance, `num_versions` set to `2` will only test the first supported and +the last release of the package. `num_versions` equal to `3` will test the first +supported, the last release, and one release in between; `num_versions` set to `4` +will test an additional release in between. In all these cases, if there is +a recent prerelease, it'll be picked as well in addition to the picked versions. ## How-Tos @@ -202,9 +206,10 @@ picked (this doesn't count towards `num_versions`). in `integrations/__init__.py`. This should be the lowest version of the framework that we can guarantee works with the SDK. If you've just added the integration, you should generally set this to the latest version of the framework - at the time. + at the time, unless you've verified the integration works for earlier versions + as well. 2. Add the integration and any constraints to `TEST_SUITE_CONFIG`. See the - "Defining constraints" section for the format. + [Test suite config](#test-suite-config) section for the format. 3. Add the integration to one of the groups in the `GROUPS` dictionary in `scripts/split_tox_gh_actions/split_tox_gh_actions.py`. 4. Run `scripts/generate-test-files.sh` and commit the changes. diff --git a/scripts/populate_tox/config.py b/scripts/populate_tox/config.py index 0ff0e9b434..f6b90e75e6 100644 --- a/scripts/populate_tox/config.py +++ b/scripts/populate_tox/config.py @@ -1,7 +1,6 @@ # The TEST_SUITE_CONFIG dictionary defines, for each integration test suite, -# the main package (framework, library) to test with; any additional test -# dependencies, optionally gated behind specific conditions; and optionally -# the Python versions to test on. +# at least the main package (framework, library) to test with. Additional +# test dependencies, Python versions to test on, etc. can also be defined here. # # See scripts/populate_tox/README.md for more info on the format and examples. diff --git a/scripts/populate_tox/populate_tox.py b/scripts/populate_tox/populate_tox.py index c0bf7f1a9f..453823f39d 100644 --- a/scripts/populate_tox/populate_tox.py +++ b/scripts/populate_tox/populate_tox.py @@ -130,7 +130,8 @@ def _save_to_cache(package: str, version: Version, release: Optional[dict]) -> N def _prefilter_releases( - integration: str, releases: dict[str, dict], older_than: Optional[datetime] = None + integration: str, + releases: dict[str, dict], ) -> tuple[list[Version], Optional[Version]]: """ Filter `releases`, removing releases that are for sure unsupported. @@ -178,9 +179,6 @@ def _prefilter_releases( uploaded = datetime.fromisoformat(meta["upload_time_iso_8601"]) - if older_than is not None and uploaded > older_than: - continue - if CUTOFF is not None and uploaded < CUTOFF: continue @@ -224,7 +222,7 @@ def _prefilter_releases( def get_supported_releases( - integration: str, pypi_data: dict, older_than: Optional[datetime] = None + integration: str, pypi_data: dict ) -> tuple[list[Version], Optional[Version]]: """ Get a list of releases that are currently supported by the SDK. @@ -236,9 +234,6 @@ def get_supported_releases( We return the list of supported releases and optionally also the newest prerelease, if it should be tested (meaning it's for a version higher than the current stable version). - - If an `older_than` timestamp is provided, no release newer than that will be - considered. """ package = pypi_data["info"]["name"] @@ -246,7 +241,8 @@ def get_supported_releases( # (because that might require an additional API call for some # of the releases) releases, latest_prerelease = _prefilter_releases( - integration, pypi_data["releases"], older_than + integration, + pypi_data["releases"], ) def _supports_lowest(release: Version) -> bool: @@ -665,32 +661,10 @@ def _normalize_release(release: dict) -> dict: return normalized -def main(fail_on_changes: bool = False) -> dict[str, list]: +def main() -> dict[str, list]: """ Generate tox.ini from the tox.jinja template. - - The script has two modes of operation: - - fail on changes mode (if `fail_on_changes` is True) - - normal mode (if `fail_on_changes` is False) - - Fail on changes mode is run on every PR to make sure that `tox.ini`, - `tox.jinja` and this script don't go out of sync because of manual changes - in one place but not the other. - - Normal mode is meant to be run as a cron job, regenerating tox.ini and - proposing the changes via a PR. """ - print(f"Running in {'fail_on_changes' if fail_on_changes else 'normal'} mode.") - last_updated = get_last_updated() - if fail_on_changes: - # We need to make the script ignore any new releases after the last updated - # timestamp so that we don't fail CI on a PR just because a new package - # version was released, leading to unrelated changes in tox.ini. - print( - f"Since we're in fail_on_changes mode, we're only considering " - f"releases before the last tox.ini update at {last_updated.isoformat()}." - ) - global MIN_PYTHON_VERSION, MAX_PYTHON_VERSION meta = _fetch_sdk_metadata() sdk_python_versions = _parse_python_versions_from_classifiers( @@ -736,12 +710,7 @@ def main(fail_on_changes: bool = False) -> dict[str, list]: # Get the list of all supported releases - # If in fail-on-changes mode, ignore releases newer than `last_updated` - older_than = last_updated if fail_on_changes else None - - releases, latest_prerelease = get_supported_releases( - integration, pypi_data, older_than - ) + releases, latest_prerelease = get_supported_releases(integration, pypi_data) if not releases: print(" Found no supported releases.") @@ -778,9 +747,6 @@ def main(fail_on_changes: bool = False) -> dict[str, list]: } ) - if fail_on_changes: - old_file_hash = get_file_hash() - write_tox_file(packages) # Sort the release cache file @@ -798,36 +764,13 @@ def main(fail_on_changes: bool = False) -> dict[str, list]: ): releases_cache.write(json.dumps(release) + "\n") - if fail_on_changes: - new_file_hash = get_file_hash() - if old_file_hash != new_file_hash: - raise RuntimeError( - dedent( - """ - Detected that `tox.ini` is out of sync with - `scripts/populate_tox/tox.jinja` and/or - `scripts/populate_tox/populate_tox.py`. This might either mean - that `tox.ini` was changed manually, or the `tox.jinja` - template and/or the `populate_tox.py` script were changed without - regenerating `tox.ini`. - - Please don't make manual changes to `tox.ini`. Instead, make the - changes to the `tox.jinja` template and/or the `populate_tox.py` - script (as applicable) and regenerate the `tox.ini` file by - running scripts/generate-test-files.sh - """ - ) - ) - print("Done checking tox.ini. Looking good!") - else: - print( - "Done generating tox.ini. Make sure to also update the CI YAML " - "files to reflect the new test targets." - ) + print( + "Done generating tox.ini. Make sure to also update the CI YAML " + "files to reflect the new test targets." + ) return packages if __name__ == "__main__": - fail_on_changes = len(sys.argv) == 2 and sys.argv[1] == "--fail-on-changes" - main(fail_on_changes) + main() diff --git a/scripts/populate_tox/releases.jsonl b/scripts/populate_tox/releases.jsonl index bd04eb7c28..9f937e5e77 100644 --- a/scripts/populate_tox/releases.jsonl +++ b/scripts/populate_tox/releases.jsonl @@ -20,7 +20,7 @@ {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Typing :: Typed"], "name": "UnleashClient", "requires_python": ">=3.8", "version": "6.0.1", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Typing :: Typed"], "name": "UnleashClient", "requires_python": ">=3.8", "version": "6.3.0", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Framework :: AsyncIO", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Internet :: WWW/HTTP"], "name": "aiohttp", "requires_python": ">=3.8", "version": "3.10.11", "yanked": false}} -{"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Framework :: AsyncIO", "Intended Audience :: Developers", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.9", "Topic :: Internet :: WWW/HTTP"], "name": "aiohttp", "requires_python": ">=3.9", "version": "3.12.15", "yanked": false}} +{"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Framework :: AsyncIO", "Intended Audience :: Developers", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", "Programming Language :: Python :: 3.9", "Topic :: Internet :: WWW/HTTP"], "name": "aiohttp", "requires_python": ">=3.9", "version": "3.13.0", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Framework :: AsyncIO", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Internet :: WWW/HTTP"], "name": "aiohttp", "requires_python": ">=3.5.3", "version": "3.4.4", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Framework :: AsyncIO", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Internet :: WWW/HTTP"], "name": "aiohttp", "requires_python": ">=3.6", "version": "3.7.4", "yanked": false}} {"info": {"classifiers": ["Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: MacOS", "Operating System :: Microsoft :: Windows", "Operating System :: OS Independent", "Operating System :: POSIX", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "anthropic", "requires_python": ">=3.7", "version": "0.16.0", "yanked": false}} @@ -46,7 +46,7 @@ {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7"], "name": "boto3", "requires_python": "", "version": "1.12.49", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9"], "name": "boto3", "requires_python": ">= 3.6", "version": "1.20.54", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9"], "name": "boto3", "requires_python": ">= 3.7", "version": "1.28.85", "yanked": false}} -{"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", "Programming Language :: Python :: 3.9"], "name": "boto3", "requires_python": ">=3.9", "version": "1.40.45", "yanked": false}} +{"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", "Programming Language :: Python :: 3.9"], "name": "boto3", "requires_python": ">=3.9", "version": "1.40.46", "yanked": false}} {"info": {"classifiers": ["Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2.5", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.2", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Topic :: Internet :: WWW/HTTP :: Dynamic Content :: CGI Tools/Libraries", "Topic :: Internet :: WWW/HTTP :: HTTP Servers", "Topic :: Internet :: WWW/HTTP :: WSGI", "Topic :: Internet :: WWW/HTTP :: WSGI :: Application", "Topic :: Internet :: WWW/HTTP :: WSGI :: Middleware", "Topic :: Internet :: WWW/HTTP :: WSGI :: Server", "Topic :: Software Development :: Libraries :: Application Frameworks"], "name": "bottle", "requires_python": "", "version": "0.12.25", "yanked": false}} {"info": {"classifiers": ["Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Internet :: WWW/HTTP :: Dynamic Content :: CGI Tools/Libraries", "Topic :: Internet :: WWW/HTTP :: HTTP Servers", "Topic :: Internet :: WWW/HTTP :: WSGI", "Topic :: Internet :: WWW/HTTP :: WSGI :: Application", "Topic :: Internet :: WWW/HTTP :: WSGI :: Middleware", "Topic :: Internet :: WWW/HTTP :: WSGI :: Server", "Topic :: Software Development :: Libraries :: Application Frameworks"], "name": "bottle", "requires_python": null, "version": "0.13.4", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: Software Development :: Object Brokering", "Topic :: System :: Distributed Computing"], "name": "celery", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*", "version": "4.4.7", "yanked": false}} @@ -111,7 +111,7 @@ {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: System :: Logging"], "name": "loguru", "requires_python": "<4.0,>=3.5", "version": "0.7.3", "yanked": false}} {"info": {"classifiers": ["Intended Audience :: Developers", "Operating System :: MacOS", "Operating System :: Microsoft :: Windows", "Operating System :: OS Independent", "Operating System :: POSIX", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "openai", "requires_python": ">=3.7.1", "version": "1.0.1", "yanked": false}} {"info": {"classifiers": ["Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS", "Operating System :: Microsoft :: Windows", "Operating System :: OS Independent", "Operating System :: POSIX", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "openai", "requires_python": ">=3.8", "version": "1.109.1", "yanked": false}} -{"info": {"classifiers": ["Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS", "Operating System :: Microsoft :: Windows", "Operating System :: OS Independent", "Operating System :: POSIX", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "openai", "requires_python": ">=3.8", "version": "2.1.0", "yanked": false}} +{"info": {"classifiers": ["Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS", "Operating System :: Microsoft :: Windows", "Operating System :: OS Independent", "Operating System :: POSIX", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "openai", "requires_python": ">=3.8", "version": "2.2.0", "yanked": false}} {"info": {"classifiers": ["Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "openai-agents", "requires_python": ">=3.9", "version": "0.0.19", "yanked": false}} {"info": {"classifiers": ["Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "openai-agents", "requires_python": ">=3.9", "version": "0.1.0", "yanked": false}} {"info": {"classifiers": ["Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries :: Python Modules", "Typing :: Typed"], "name": "openai-agents", "requires_python": ">=3.9", "version": "0.2.11", "yanked": false}} @@ -191,7 +191,7 @@ {"info": {"classifiers": ["Intended Audience :: Developers", "Programming Language :: Python", "Programming Language :: Python :: 3", "Topic :: Software Development :: Libraries"], "name": "statsig", "requires_python": ">=3.7", "version": "0.55.3", "yanked": false}} {"info": {"classifiers": ["Intended Audience :: Developers", "Programming Language :: Python", "Programming Language :: Python :: 3", "Topic :: Software Development :: Libraries"], "name": "statsig", "requires_python": ">=3.7", "version": "0.65.0", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Python Modules"], "name": "strawberry-graphql", "requires_python": ">=3.8,<4.0", "version": "0.209.8", "yanked": false}} -{"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Python Modules"], "name": "strawberry-graphql", "requires_python": "<4.0,>=3.9", "version": "0.283.0", "yanked": false}} +{"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Libraries :: Python Modules"], "name": "strawberry-graphql", "requires_python": "<4.0,>=3.9", "version": "0.283.1", "yanked": false}} {"info": {"classifiers": ["License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy"], "name": "tornado", "requires_python": ">= 3.5", "version": "6.0.4", "yanked": false}} {"info": {"classifiers": ["License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy"], "name": "tornado", "requires_python": ">=3.9", "version": "6.5.2", "yanked": false}} {"info": {"classifiers": ["Development Status :: 5 - Production/Stable", "Environment :: No Input/Output (Daemon)", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License (GPL)", "Natural Language :: English", "Natural Language :: French", "Natural Language :: German", "Natural Language :: Spanish", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Software Development :: Libraries :: Application Frameworks"], "name": "trytond", "requires_python": null, "version": "1.2.10", "yanked": false}} diff --git a/scripts/populate_tox/tox.jinja b/scripts/populate_tox/tox.jinja index 2a33e7790d..40ea309b08 100755 --- a/scripts/populate_tox/tox.jinja +++ b/scripts/populate_tox/tox.jinja @@ -1,14 +1,16 @@ -# Tox (http://codespeak.net/~hpk/tox/) is a tool for running tests -# in multiple virtualenvs. This configuration file will run the -# test suite on all supported python versions. To use it, "pip install tox" -# and then run "tox" from this directory. +# DON'T EDIT THIS FILE BY HAND. This file has been generated from a template by +# `scripts/populate_tox/populate_tox.py`. # -# This file has been generated from a template -# by "scripts/populate_tox/populate_tox.py". Any changes to the file should -# be made in the template (if you want to change a hardcoded part of the file) -# or in the script (if you want to change the auto-generated part). -# The file (and all resulting CI YAMLs) then needs to be regenerated via -# "scripts/generate-test-files.sh". +# Any changes to the test matrix should be made +# - either in the script config in `scripts/populate_tox/config.py` (if you want +# to change the auto-generated part) +# - or in the template in `scripts/populate_tox/tox.jinja` (if you want to change +# a hardcoded part of the file) +# +# This file (and all resulting CI YAMLs) then needs to be regenerated via +# `scripts/generate-test-files.sh`. +# +# See also `scripts/populate_tox/README.md` for more info. [tox] requires = diff --git a/tox.ini b/tox.ini index 8eb04550fb..2c77edd07c 100644 --- a/tox.ini +++ b/tox.ini @@ -1,14 +1,16 @@ -# Tox (http://codespeak.net/~hpk/tox/) is a tool for running tests -# in multiple virtualenvs. This configuration file will run the -# test suite on all supported python versions. To use it, "pip install tox" -# and then run "tox" from this directory. +# DON'T EDIT THIS FILE BY HAND. This file has been generated from a template by +# `scripts/populate_tox/populate_tox.py`. # -# This file has been generated from a template -# by "scripts/populate_tox/populate_tox.py". Any changes to the file should -# be made in the template (if you want to change a hardcoded part of the file) -# or in the script (if you want to change the auto-generated part). -# The file (and all resulting CI YAMLs) then needs to be regenerated via -# "scripts/generate-test-files.sh". +# Any changes to the test matrix should be made +# - either in the script config in `scripts/populate_tox/config.py` (if you want +# to change the auto-generated part) +# - or in the template in `scripts/populate_tox/tox.jinja` (if you want to change +# a hardcoded part of the file) +# +# This file (and all resulting CI YAMLs) then needs to be regenerated via +# `scripts/generate-test-files.sh`. +# +# See also `scripts/populate_tox/README.md` for more info. [tox] requires = @@ -67,11 +69,11 @@ envlist = {py3.8,py3.11,py3.12}-openai-base-v1.0.1 {py3.8,py3.12,py3.13}-openai-base-v1.109.1 - {py3.8,py3.12,py3.13}-openai-base-v2.1.0 + {py3.8,py3.12,py3.13}-openai-base-v2.2.0 {py3.8,py3.11,py3.12}-openai-notiktoken-v1.0.1 {py3.8,py3.12,py3.13}-openai-notiktoken-v1.109.1 - {py3.8,py3.12,py3.13}-openai-notiktoken-v2.1.0 + {py3.8,py3.12,py3.13}-openai-notiktoken-v2.2.0 {py3.9,py3.12,py3.13}-langgraph-v0.6.8 {py3.10,py3.12,py3.13}-langgraph-v1.0.0a4 @@ -92,7 +94,7 @@ envlist = {py3.6,py3.7}-boto3-v1.12.49 {py3.6,py3.9,py3.10}-boto3-v1.20.54 {py3.7,py3.11,py3.12}-boto3-v1.28.85 - {py3.9,py3.12,py3.13}-boto3-v1.40.45 + {py3.9,py3.12,py3.13}-boto3-v1.40.46 {py3.6,py3.7,py3.8}-chalice-v1.16.0 {py3.9,py3.12,py3.13}-chalice-v1.32.0 @@ -151,7 +153,7 @@ envlist = {py3.8,py3.12,py3.13}-graphene-v3.4.3 {py3.8,py3.10,py3.11}-strawberry-v0.209.8 - {py3.9,py3.12,py3.13}-strawberry-v0.283.0 + {py3.9,py3.12,py3.13}-strawberry-v0.283.1 # ~~~ Network ~~~ @@ -227,7 +229,7 @@ envlist = {py3.7}-aiohttp-v3.4.4 {py3.7,py3.8,py3.9}-aiohttp-v3.7.4 {py3.8,py3.12,py3.13}-aiohttp-v3.10.11 - {py3.9,py3.12,py3.13}-aiohttp-v3.12.15 + {py3.9,py3.12,py3.13}-aiohttp-v3.13.0 {py3.6,py3.7}-bottle-v0.12.25 {py3.8,py3.12,py3.13}-bottle-v0.13.4 @@ -365,14 +367,14 @@ deps = openai-base-v1.0.1: openai==1.0.1 openai-base-v1.109.1: openai==1.109.1 - openai-base-v2.1.0: openai==2.1.0 + openai-base-v2.2.0: openai==2.2.0 openai-base: pytest-asyncio openai-base: tiktoken openai-base-v1.0.1: httpx<0.28 openai-notiktoken-v1.0.1: openai==1.0.1 openai-notiktoken-v1.109.1: openai==1.109.1 - openai-notiktoken-v2.1.0: openai==2.1.0 + openai-notiktoken-v2.2.0: openai==2.2.0 openai-notiktoken: pytest-asyncio openai-notiktoken-v1.0.1: httpx<0.28 @@ -398,7 +400,7 @@ deps = boto3-v1.12.49: boto3==1.12.49 boto3-v1.20.54: boto3==1.20.54 boto3-v1.28.85: boto3==1.28.85 - boto3-v1.40.45: boto3==1.40.45 + boto3-v1.40.46: boto3==1.40.46 {py3.7,py3.8}-boto3: urllib3<2.0.0 chalice-v1.16.0: chalice==1.16.0 @@ -475,7 +477,7 @@ deps = {py3.6}-graphene: aiocontextvars strawberry-v0.209.8: strawberry-graphql[fastapi,flask]==0.209.8 - strawberry-v0.283.0: strawberry-graphql[fastapi,flask]==0.283.0 + strawberry-v0.283.1: strawberry-graphql[fastapi,flask]==0.283.1 strawberry: httpx strawberry-v0.209.8: pydantic<2.11 @@ -618,10 +620,10 @@ deps = aiohttp-v3.4.4: aiohttp==3.4.4 aiohttp-v3.7.4: aiohttp==3.7.4 aiohttp-v3.10.11: aiohttp==3.10.11 - aiohttp-v3.12.15: aiohttp==3.12.15 + aiohttp-v3.13.0: aiohttp==3.13.0 aiohttp: pytest-aiohttp aiohttp-v3.10.11: pytest-asyncio - aiohttp-v3.12.15: pytest-asyncio + aiohttp-v3.13.0: pytest-asyncio bottle-v0.12.25: bottle==0.12.25 bottle-v0.13.4: bottle==0.13.4 From b1dd2dcf3b63c73ab91f25ca217a28cf3ea7f6e9 Mon Sep 17 00:00:00 2001 From: Simon Hellmayr Date: Wed, 8 Oct 2025 12:41:55 +0200 Subject: [PATCH 2/9] fix(ai): add mapping for gen_ai message roles (#4884) - Add a constant that contains the allowed message roles according to OTEL and a mapping - Apply that mapping to all gen_ai integrations - We will track input roles that do not conform to expectations via a Sentry issue in agent monitoring to make sure we continually update the mappings --------- Co-authored-by: Ivana Kellyer --- sentry_sdk/ai/__init__.py | 7 + sentry_sdk/ai/utils.py | 48 ++++++ sentry_sdk/integrations/anthropic.py | 12 +- sentry_sdk/integrations/langchain.py | 33 +++- sentry_sdk/integrations/langgraph.py | 8 +- sentry_sdk/integrations/openai.py | 5 +- .../openai_agents/spans/invoke_agent.py | 12 +- .../integrations/openai_agents/utils.py | 53 ++++--- .../integrations/anthropic/test_anthropic.py | 67 +++++++++ .../integrations/langchain/test_langchain.py | 141 ++++++++++++++++++ .../integrations/langgraph/test_langgraph.py | 71 +++++++++ tests/integrations/openai/test_openai.py | 53 +++++++ .../openai_agents/test_openai_agents.py | 46 ++++++ 13 files changed, 525 insertions(+), 31 deletions(-) diff --git a/sentry_sdk/ai/__init__.py b/sentry_sdk/ai/__init__.py index e69de29bb2..fbcb9c061d 100644 --- a/sentry_sdk/ai/__init__.py +++ b/sentry_sdk/ai/__init__.py @@ -0,0 +1,7 @@ +from .utils import ( + set_data_normalized, + GEN_AI_MESSAGE_ROLE_MAPPING, + GEN_AI_MESSAGE_ROLE_REVERSE_MAPPING, + normalize_message_role, + normalize_message_roles, +) # noqa: F401 diff --git a/sentry_sdk/ai/utils.py b/sentry_sdk/ai/utils.py index d0ccf1bed3..0c0b937006 100644 --- a/sentry_sdk/ai/utils.py +++ b/sentry_sdk/ai/utils.py @@ -10,6 +10,26 @@ from sentry_sdk.utils import logger +class GEN_AI_ALLOWED_MESSAGE_ROLES: + SYSTEM = "system" + USER = "user" + ASSISTANT = "assistant" + TOOL = "tool" + + +GEN_AI_MESSAGE_ROLE_REVERSE_MAPPING = { + GEN_AI_ALLOWED_MESSAGE_ROLES.SYSTEM: ["system"], + GEN_AI_ALLOWED_MESSAGE_ROLES.USER: ["user", "human"], + GEN_AI_ALLOWED_MESSAGE_ROLES.ASSISTANT: ["assistant", "ai"], + GEN_AI_ALLOWED_MESSAGE_ROLES.TOOL: ["tool", "tool_call"], +} + +GEN_AI_MESSAGE_ROLE_MAPPING = {} +for target_role, source_roles in GEN_AI_MESSAGE_ROLE_REVERSE_MAPPING.items(): + for source_role in source_roles: + GEN_AI_MESSAGE_ROLE_MAPPING[source_role] = target_role + + def _normalize_data(data, unpack=True): # type: (Any, bool) -> Any # convert pydantic data (e.g. OpenAI v1+) to json compatible format @@ -40,6 +60,34 @@ def set_data_normalized(span, key, value, unpack=True): span.set_data(key, json.dumps(normalized)) +def normalize_message_role(role): + # type: (str) -> str + """ + Normalize a message role to one of the 4 allowed gen_ai role values. + Maps "ai" -> "assistant" and keeps other standard roles unchanged. + """ + return GEN_AI_MESSAGE_ROLE_MAPPING.get(role, role) + + +def normalize_message_roles(messages): + # type: (list[dict[str, Any]]) -> list[dict[str, Any]] + """ + Normalize roles in a list of messages to use standard gen_ai role values. + Creates a deep copy to avoid modifying the original messages. + """ + normalized_messages = [] + for message in messages: + if not isinstance(message, dict): + normalized_messages.append(message) + continue + normalized_message = message.copy() + if "role" in message: + normalized_message["role"] = normalize_message_role(message["role"]) + normalized_messages.append(normalized_message) + + return normalized_messages + + def get_start_span_function(): # type: () -> Callable[..., Any] current_span = sentry_sdk.get_current_span() diff --git a/sentry_sdk/integrations/anthropic.py b/sentry_sdk/integrations/anthropic.py index d9898fa1d1..46c6b2a766 100644 --- a/sentry_sdk/integrations/anthropic.py +++ b/sentry_sdk/integrations/anthropic.py @@ -3,7 +3,11 @@ import sentry_sdk from sentry_sdk.ai.monitoring import record_token_usage -from sentry_sdk.ai.utils import set_data_normalized, get_start_span_function +from sentry_sdk.ai.utils import ( + set_data_normalized, + normalize_message_roles, + get_start_span_function, +) from sentry_sdk.consts import OP, SPANDATA, SPANSTATUS from sentry_sdk.integrations import _check_minimum_version, DidNotEnable, Integration from sentry_sdk.scope import should_send_default_pii @@ -140,8 +144,12 @@ def _set_input_data(span, kwargs, integration): else: normalized_messages.append(message) + role_normalized_messages = normalize_message_roles(normalized_messages) set_data_normalized( - span, SPANDATA.GEN_AI_REQUEST_MESSAGES, normalized_messages, unpack=False + span, + SPANDATA.GEN_AI_REQUEST_MESSAGES, + role_normalized_messages, + unpack=False, ) set_data_normalized( diff --git a/sentry_sdk/integrations/langchain.py b/sentry_sdk/integrations/langchain.py index fdba26569d..724d908665 100644 --- a/sentry_sdk/integrations/langchain.py +++ b/sentry_sdk/integrations/langchain.py @@ -4,7 +4,12 @@ import sentry_sdk from sentry_sdk.ai.monitoring import set_ai_pipeline_name -from sentry_sdk.ai.utils import set_data_normalized, get_start_span_function +from sentry_sdk.ai.utils import ( + GEN_AI_ALLOWED_MESSAGE_ROLES, + normalize_message_roles, + set_data_normalized, + get_start_span_function, +) from sentry_sdk.consts import OP, SPANDATA from sentry_sdk.integrations import DidNotEnable, Integration from sentry_sdk.scope import should_send_default_pii @@ -209,8 +214,18 @@ def on_llm_start( _set_tools_on_span(span, all_params.get("tools")) if should_send_default_pii() and self.include_prompts: + normalized_messages = [ + { + "role": GEN_AI_ALLOWED_MESSAGE_ROLES.USER, + "content": {"type": "text", "text": prompt}, + } + for prompt in prompts + ] set_data_normalized( - span, SPANDATA.GEN_AI_REQUEST_MESSAGES, prompts, unpack=False + span, + SPANDATA.GEN_AI_REQUEST_MESSAGES, + normalized_messages, + unpack=False, ) def on_chat_model_start(self, serialized, messages, *, run_id, **kwargs): @@ -262,6 +277,8 @@ def on_chat_model_start(self, serialized, messages, *, run_id, **kwargs): normalized_messages.append( self._normalize_langchain_message(message) ) + normalized_messages = normalize_message_roles(normalized_messages) + set_data_normalized( span, SPANDATA.GEN_AI_REQUEST_MESSAGES, @@ -740,8 +757,12 @@ def new_invoke(self, *args, **kwargs): and should_send_default_pii() and integration.include_prompts ): + normalized_messages = normalize_message_roles([input]) set_data_normalized( - span, SPANDATA.GEN_AI_REQUEST_MESSAGES, [input], unpack=False + span, + SPANDATA.GEN_AI_REQUEST_MESSAGES, + normalized_messages, + unpack=False, ) output = result.get("output") @@ -791,8 +812,12 @@ def new_stream(self, *args, **kwargs): and should_send_default_pii() and integration.include_prompts ): + normalized_messages = normalize_message_roles([input]) set_data_normalized( - span, SPANDATA.GEN_AI_REQUEST_MESSAGES, [input], unpack=False + span, + SPANDATA.GEN_AI_REQUEST_MESSAGES, + normalized_messages, + unpack=False, ) # Run the agent diff --git a/sentry_sdk/integrations/langgraph.py b/sentry_sdk/integrations/langgraph.py index df3941bb13..11aa1facf4 100644 --- a/sentry_sdk/integrations/langgraph.py +++ b/sentry_sdk/integrations/langgraph.py @@ -2,7 +2,7 @@ from typing import Any, Callable, List, Optional import sentry_sdk -from sentry_sdk.ai.utils import set_data_normalized +from sentry_sdk.ai.utils import set_data_normalized, normalize_message_roles from sentry_sdk.consts import OP, SPANDATA from sentry_sdk.integrations import DidNotEnable, Integration from sentry_sdk.scope import should_send_default_pii @@ -180,10 +180,11 @@ def new_invoke(self, *args, **kwargs): ): input_messages = _parse_langgraph_messages(args[0]) if input_messages: + normalized_input_messages = normalize_message_roles(input_messages) set_data_normalized( span, SPANDATA.GEN_AI_REQUEST_MESSAGES, - input_messages, + normalized_input_messages, unpack=False, ) @@ -230,10 +231,11 @@ async def new_ainvoke(self, *args, **kwargs): ): input_messages = _parse_langgraph_messages(args[0]) if input_messages: + normalized_input_messages = normalize_message_roles(input_messages) set_data_normalized( span, SPANDATA.GEN_AI_REQUEST_MESSAGES, - input_messages, + normalized_input_messages, unpack=False, ) diff --git a/sentry_sdk/integrations/openai.py b/sentry_sdk/integrations/openai.py index e8b3b30ab2..e9bd2efa23 100644 --- a/sentry_sdk/integrations/openai.py +++ b/sentry_sdk/integrations/openai.py @@ -3,7 +3,7 @@ import sentry_sdk from sentry_sdk import consts from sentry_sdk.ai.monitoring import record_token_usage -from sentry_sdk.ai.utils import set_data_normalized +from sentry_sdk.ai.utils import set_data_normalized, normalize_message_roles from sentry_sdk.consts import SPANDATA from sentry_sdk.integrations import DidNotEnable, Integration from sentry_sdk.scope import should_send_default_pii @@ -182,8 +182,9 @@ def _set_input_data(span, kwargs, operation, integration): and should_send_default_pii() and integration.include_prompts ): + normalized_messages = normalize_message_roles(messages) set_data_normalized( - span, SPANDATA.GEN_AI_REQUEST_MESSAGES, messages, unpack=False + span, SPANDATA.GEN_AI_REQUEST_MESSAGES, normalized_messages, unpack=False ) # Input attributes: Common diff --git a/sentry_sdk/integrations/openai_agents/spans/invoke_agent.py b/sentry_sdk/integrations/openai_agents/spans/invoke_agent.py index cf06120625..2a9c5ebe66 100644 --- a/sentry_sdk/integrations/openai_agents/spans/invoke_agent.py +++ b/sentry_sdk/integrations/openai_agents/spans/invoke_agent.py @@ -1,5 +1,9 @@ import sentry_sdk -from sentry_sdk.ai.utils import get_start_span_function, set_data_normalized +from sentry_sdk.ai.utils import ( + get_start_span_function, + set_data_normalized, + normalize_message_roles, +) from sentry_sdk.consts import OP, SPANDATA from sentry_sdk.scope import should_send_default_pii from sentry_sdk.utils import safe_serialize @@ -56,8 +60,12 @@ def invoke_agent_span(context, agent, kwargs): ) if len(messages) > 0: + normalized_messages = normalize_message_roles(messages) set_data_normalized( - span, SPANDATA.GEN_AI_REQUEST_MESSAGES, messages, unpack=False + span, + SPANDATA.GEN_AI_REQUEST_MESSAGES, + normalized_messages, + unpack=False, ) _set_agent_data(span, agent) diff --git a/sentry_sdk/integrations/openai_agents/utils.py b/sentry_sdk/integrations/openai_agents/utils.py index b0ad6bf903..125ff1175b 100644 --- a/sentry_sdk/integrations/openai_agents/utils.py +++ b/sentry_sdk/integrations/openai_agents/utils.py @@ -1,5 +1,10 @@ import sentry_sdk -from sentry_sdk.ai.utils import set_data_normalized +from sentry_sdk.ai.utils import ( + GEN_AI_ALLOWED_MESSAGE_ROLES, + normalize_message_roles, + set_data_normalized, + normalize_message_role, +) from sentry_sdk.consts import SPANDATA, SPANSTATUS, OP from sentry_sdk.integrations import DidNotEnable from sentry_sdk.scope import should_send_default_pii @@ -94,35 +99,47 @@ def _set_input_data(span, get_response_kwargs): # type: (sentry_sdk.tracing.Span, dict[str, Any]) -> None if not should_send_default_pii(): return + request_messages = [] - messages_by_role = { - "system": [], - "user": [], - "assistant": [], - "tool": [], - } # type: (dict[str, list[Any]]) system_instructions = get_response_kwargs.get("system_instructions") if system_instructions: - messages_by_role["system"].append({"type": "text", "text": system_instructions}) + request_messages.append( + { + "role": GEN_AI_ALLOWED_MESSAGE_ROLES.SYSTEM, + "content": [{"type": "text", "text": system_instructions}], + } + ) for message in get_response_kwargs.get("input", []): if "role" in message: - messages_by_role[message.get("role")].append( - {"type": "text", "text": message.get("content")} + normalized_role = normalize_message_role(message.get("role")) + request_messages.append( + { + "role": normalized_role, + "content": [{"type": "text", "text": message.get("content")}], + } ) else: if message.get("type") == "function_call": - messages_by_role["assistant"].append(message) + request_messages.append( + { + "role": GEN_AI_ALLOWED_MESSAGE_ROLES.ASSISTANT, + "content": [message], + } + ) elif message.get("type") == "function_call_output": - messages_by_role["tool"].append(message) - - request_messages = [] - for role, messages in messages_by_role.items(): - if len(messages) > 0: - request_messages.append({"role": role, "content": messages}) + request_messages.append( + { + "role": GEN_AI_ALLOWED_MESSAGE_ROLES.TOOL, + "content": [message], + } + ) set_data_normalized( - span, SPANDATA.GEN_AI_REQUEST_MESSAGES, request_messages, unpack=False + span, + SPANDATA.GEN_AI_REQUEST_MESSAGES, + normalize_message_roles(request_messages), + unpack=False, ) diff --git a/tests/integrations/anthropic/test_anthropic.py b/tests/integrations/anthropic/test_anthropic.py index 04ff12eb8b..e9065e2d32 100644 --- a/tests/integrations/anthropic/test_anthropic.py +++ b/tests/integrations/anthropic/test_anthropic.py @@ -1,5 +1,6 @@ import pytest from unittest import mock +import json try: from unittest.mock import AsyncMock @@ -878,3 +879,69 @@ def test_set_output_data_with_input_json_delta(sentry_init): assert span._data.get(SPANDATA.GEN_AI_USAGE_INPUT_TOKENS) == 10 assert span._data.get(SPANDATA.GEN_AI_USAGE_OUTPUT_TOKENS) == 20 assert span._data.get(SPANDATA.GEN_AI_USAGE_TOTAL_TOKENS) == 30 + + +def test_anthropic_message_role_mapping(sentry_init, capture_events): + """Test that Anthropic integration properly maps message roles like 'ai' to 'assistant'""" + sentry_init( + integrations=[AnthropicIntegration(include_prompts=True)], + traces_sample_rate=1.0, + send_default_pii=True, + ) + events = capture_events() + + client = Anthropic(api_key="z") + + def mock_messages_create(*args, **kwargs): + return Message( + id="msg_1", + content=[TextBlock(text="Hi there!", type="text")], + model="claude-3-opus", + role="assistant", + stop_reason="end_turn", + stop_sequence=None, + type="message", + usage=Usage(input_tokens=10, output_tokens=5), + ) + + client.messages._post = mock.Mock(return_value=mock_messages_create()) + + # Test messages with mixed roles including "ai" that should be mapped to "assistant" + test_messages = [ + {"role": "system", "content": "You are helpful."}, + {"role": "user", "content": "Hello"}, + {"role": "ai", "content": "Hi there!"}, # Should be mapped to "assistant" + {"role": "assistant", "content": "How can I help?"}, # Should stay "assistant" + ] + + with start_transaction(name="anthropic tx"): + client.messages.create( + model="claude-3-opus", max_tokens=10, messages=test_messages + ) + + (event,) = events + span = event["spans"][0] + + # Verify that the span was created correctly + assert span["op"] == "gen_ai.chat" + assert SPANDATA.GEN_AI_REQUEST_MESSAGES in span["data"] + + # Parse the stored messages + stored_messages = json.loads(span["data"][SPANDATA.GEN_AI_REQUEST_MESSAGES]) + + # Verify that "ai" role was mapped to "assistant" + assert len(stored_messages) == 4 + assert stored_messages[0]["role"] == "system" + assert stored_messages[1]["role"] == "user" + assert ( + stored_messages[2]["role"] == "assistant" + ) # "ai" should be mapped to "assistant" + assert stored_messages[3]["role"] == "assistant" # should stay "assistant" + + # Verify content is preserved + assert stored_messages[2]["content"] == "Hi there!" + assert stored_messages[3]["content"] == "How can I help?" + + # Verify no "ai" roles remain + roles = [msg["role"] for msg in stored_messages] + assert "ai" not in roles diff --git a/tests/integrations/langchain/test_langchain.py b/tests/integrations/langchain/test_langchain.py index ba49b2e508..661208432f 100644 --- a/tests/integrations/langchain/test_langchain.py +++ b/tests/integrations/langchain/test_langchain.py @@ -817,3 +817,144 @@ def test_langchain_integration_with_langchain_core_only(sentry_init, capture_eve assert llm_span["data"]["gen_ai.usage.total_tokens"] == 25 assert llm_span["data"]["gen_ai.usage.input_tokens"] == 10 assert llm_span["data"]["gen_ai.usage.output_tokens"] == 15 + + +def test_langchain_message_role_mapping(sentry_init, capture_events): + """Test that message roles are properly normalized in langchain integration.""" + global llm_type + llm_type = "openai-chat" + + sentry_init( + integrations=[LangchainIntegration(include_prompts=True)], + traces_sample_rate=1.0, + send_default_pii=True, + ) + events = capture_events() + + prompt = ChatPromptTemplate.from_messages( + [ + ("system", "You are a helpful assistant"), + ("human", "{input}"), + MessagesPlaceholder(variable_name="agent_scratchpad"), + ] + ) + + global stream_result_mock + stream_result_mock = Mock( + side_effect=[ + [ + ChatGenerationChunk( + type="ChatGenerationChunk", + message=AIMessageChunk(content="Test response"), + ), + ] + ] + ) + + llm = MockOpenAI( + model_name="gpt-3.5-turbo", + temperature=0, + openai_api_key="badkey", + ) + agent = create_openai_tools_agent(llm, [get_word_length], prompt) + agent_executor = AgentExecutor(agent=agent, tools=[get_word_length], verbose=True) + + # Test input that should trigger message role normalization + test_input = "Hello, how are you?" + + with start_transaction(): + list(agent_executor.stream({"input": test_input})) + + assert len(events) > 0 + tx = events[0] + assert tx["type"] == "transaction" + + # Find spans with gen_ai operation that should have message data + gen_ai_spans = [ + span for span in tx.get("spans", []) if span.get("op", "").startswith("gen_ai") + ] + + # Check if any span has message data with normalized roles + message_data_found = False + for span in gen_ai_spans: + span_data = span.get("data", {}) + if SPANDATA.GEN_AI_REQUEST_MESSAGES in span_data: + message_data_found = True + messages_data = span_data[SPANDATA.GEN_AI_REQUEST_MESSAGES] + + # Parse the message data (might be JSON string) + if isinstance(messages_data, str): + import json + + try: + messages = json.loads(messages_data) + except json.JSONDecodeError: + # If not valid JSON, skip this assertion + continue + else: + messages = messages_data + + # Verify that the input message is present and contains the test input + assert isinstance(messages, list) + assert len(messages) > 0 + + # The test input should be in one of the messages + input_found = False + for msg in messages: + if isinstance(msg, dict) and test_input in str(msg.get("content", "")): + input_found = True + break + elif isinstance(msg, str) and test_input in msg: + input_found = True + break + + assert input_found, ( + f"Test input '{test_input}' not found in messages: {messages}" + ) + break + + # The message role mapping functionality is primarily tested through the normalization + # that happens in the integration code. The fact that we can capture and process + # the messages without errors indicates the role mapping is working correctly. + assert message_data_found, "No span found with gen_ai request messages data" + + +def test_langchain_message_role_normalization_units(): + """Test the message role normalization functions directly.""" + from sentry_sdk.ai.utils import normalize_message_role, normalize_message_roles + + # Test individual role normalization + assert normalize_message_role("ai") == "assistant" + assert normalize_message_role("human") == "user" + assert normalize_message_role("tool_call") == "tool" + assert normalize_message_role("system") == "system" + assert normalize_message_role("user") == "user" + assert normalize_message_role("assistant") == "assistant" + assert normalize_message_role("tool") == "tool" + + # Test unknown role (should remain unchanged) + assert normalize_message_role("unknown_role") == "unknown_role" + + # Test message list normalization + test_messages = [ + {"role": "human", "content": "Hello"}, + {"role": "ai", "content": "Hi there!"}, + {"role": "tool_call", "content": "function_call"}, + {"role": "system", "content": "You are helpful"}, + {"content": "Message without role"}, + "string message", + ] + + normalized = normalize_message_roles(test_messages) + + # Verify the original messages are not modified + assert test_messages[0]["role"] == "human" # Original unchanged + assert test_messages[1]["role"] == "ai" # Original unchanged + + # Verify the normalized messages have correct roles + assert normalized[0]["role"] == "user" # human -> user + assert normalized[1]["role"] == "assistant" # ai -> assistant + assert normalized[2]["role"] == "tool" # tool_call -> tool + assert normalized[3]["role"] == "system" # system unchanged + assert "role" not in normalized[4] # Message without role unchanged + assert normalized[5] == "string message" # String message unchanged diff --git a/tests/integrations/langgraph/test_langgraph.py b/tests/integrations/langgraph/test_langgraph.py index 1510305b06..6ec6d9a96d 100644 --- a/tests/integrations/langgraph/test_langgraph.py +++ b/tests/integrations/langgraph/test_langgraph.py @@ -625,3 +625,74 @@ def original_invoke(self, *args, **kwargs): assert tool_calls_data[0]["function"]["name"] == "search" assert tool_calls_data[1]["id"] == "call_multi_2" assert tool_calls_data[1]["function"]["name"] == "calculate" + + +def test_langgraph_message_role_mapping(sentry_init, capture_events): + """Test that Langgraph integration properly maps message roles like 'ai' to 'assistant'""" + sentry_init( + integrations=[LanggraphIntegration(include_prompts=True)], + traces_sample_rate=1.0, + send_default_pii=True, + ) + events = capture_events() + + # Mock a langgraph message with mixed roles + class MockMessage: + def __init__(self, content, message_type="human"): + self.content = content + self.type = message_type + + # Create mock state with messages having different roles + state_data = { + "messages": [ + MockMessage("System prompt", "system"), + MockMessage("Hello", "human"), + MockMessage("Hi there!", "ai"), # Should be mapped to "assistant" + MockMessage("How can I help?", "assistant"), # Should stay "assistant" + ] + } + + compiled_graph = MockCompiledGraph("test_graph") + pregel = MockPregelInstance(compiled_graph) + + with start_transaction(name="langgraph tx"): + # Use the wrapped invoke function directly + from sentry_sdk.integrations.langgraph import _wrap_pregel_invoke + + wrapped_invoke = _wrap_pregel_invoke( + lambda self, state_data: {"result": "success"} + ) + wrapped_invoke(pregel, state_data) + + (event,) = events + span = event["spans"][0] + + # Verify that the span was created correctly + assert span["op"] == "gen_ai.invoke_agent" + + # If messages were captured, verify role mapping + if SPANDATA.GEN_AI_REQUEST_MESSAGES in span["data"]: + import json + + stored_messages = json.loads(span["data"][SPANDATA.GEN_AI_REQUEST_MESSAGES]) + + # Find messages with specific content to verify role mapping + ai_message = next( + (msg for msg in stored_messages if msg.get("content") == "Hi there!"), None + ) + assistant_message = next( + (msg for msg in stored_messages if msg.get("content") == "How can I help?"), + None, + ) + + if ai_message: + # "ai" should have been mapped to "assistant" + assert ai_message["role"] == "assistant" + + if assistant_message: + # "assistant" should stay "assistant" + assert assistant_message["role"] == "assistant" + + # Verify no "ai" roles remain + roles = [msg["role"] for msg in stored_messages if "role" in msg] + assert "ai" not in roles diff --git a/tests/integrations/openai/test_openai.py b/tests/integrations/openai/test_openai.py index e7fbf8a7d8..06e0a09fcf 100644 --- a/tests/integrations/openai/test_openai.py +++ b/tests/integrations/openai/test_openai.py @@ -1447,3 +1447,56 @@ def test_empty_tools_in_chat_completion(sentry_init, capture_events, tools): span = event["spans"][0] assert "gen_ai.request.available_tools" not in span["data"] + + +def test_openai_message_role_mapping(sentry_init, capture_events): + """Test that OpenAI integration properly maps message roles like 'ai' to 'assistant'""" + sentry_init( + integrations=[OpenAIIntegration(include_prompts=True)], + traces_sample_rate=1.0, + send_default_pii=True, + ) + events = capture_events() + + client = OpenAI(api_key="z") + client.chat.completions._post = mock.Mock(return_value=EXAMPLE_CHAT_COMPLETION) + + # Test messages with mixed roles including "ai" that should be mapped to "assistant" + test_messages = [ + {"role": "system", "content": "You are helpful."}, + {"role": "user", "content": "Hello"}, + {"role": "ai", "content": "Hi there!"}, # Should be mapped to "assistant" + {"role": "assistant", "content": "How can I help?"}, # Should stay "assistant" + ] + + with start_transaction(name="openai tx"): + client.chat.completions.create(model="test-model", messages=test_messages) + + (event,) = events + span = event["spans"][0] + + # Verify that the span was created correctly + assert span["op"] == "gen_ai.chat" + assert SPANDATA.GEN_AI_REQUEST_MESSAGES in span["data"] + + # Parse the stored messages + import json + + stored_messages = json.loads(span["data"][SPANDATA.GEN_AI_REQUEST_MESSAGES]) + + # Verify that "ai" role was mapped to "assistant" + assert len(stored_messages) == 4 + assert stored_messages[0]["role"] == "system" + assert stored_messages[1]["role"] == "user" + assert ( + stored_messages[2]["role"] == "assistant" + ) # "ai" should be mapped to "assistant" + assert stored_messages[3]["role"] == "assistant" # should stay "assistant" + + # Verify content is preserved + assert stored_messages[2]["content"] == "Hi there!" + assert stored_messages[3]["content"] == "How can I help?" + + # Verify no "ai" roles remain + roles = [msg["role"] for msg in stored_messages] + assert "ai" not in roles diff --git a/tests/integrations/openai_agents/test_openai_agents.py b/tests/integrations/openai_agents/test_openai_agents.py index e9a8372806..e647ce9fad 100644 --- a/tests/integrations/openai_agents/test_openai_agents.py +++ b/tests/integrations/openai_agents/test_openai_agents.py @@ -1031,3 +1031,49 @@ async def run(): assert txn2["transaction"] == "test_agent workflow" assert txn3["type"] == "transaction" assert txn3["transaction"] == "test_agent workflow" + + +def test_openai_agents_message_role_mapping(sentry_init, capture_events): + """Test that OpenAI Agents integration properly maps message roles like 'ai' to 'assistant'""" + sentry_init( + integrations=[OpenAIAgentsIntegration()], + traces_sample_rate=1.0, + send_default_pii=True, + ) + + # Test input messages with mixed roles including "ai" + test_input = [ + {"role": "system", "content": "You are helpful."}, + {"role": "user", "content": "Hello"}, + {"role": "ai", "content": "Hi there!"}, # Should be mapped to "assistant" + {"role": "assistant", "content": "How can I help?"}, # Should stay "assistant" + ] + + get_response_kwargs = {"input": test_input} + + from sentry_sdk.integrations.openai_agents.utils import _set_input_data + from sentry_sdk import start_span + + with start_span(op="test") as span: + _set_input_data(span, get_response_kwargs) + + # Verify that messages were processed and roles were mapped + from sentry_sdk.consts import SPANDATA + + if SPANDATA.GEN_AI_REQUEST_MESSAGES in span._data: + import json + + stored_messages = json.loads(span._data[SPANDATA.GEN_AI_REQUEST_MESSAGES]) + + # Verify roles were properly mapped + found_assistant_roles = 0 + for message in stored_messages: + if message["role"] == "assistant": + found_assistant_roles += 1 + + # Should have 2 assistant roles (1 from original "assistant", 1 from mapped "ai") + assert found_assistant_roles == 2 + + # Verify no "ai" roles remain in any message + for message in stored_messages: + assert message["role"] != "ai" From f32e391a6fe22cc1d26c093e63ac4499e52b9d68 Mon Sep 17 00:00:00 2001 From: Alex Alderman Webb Date: Wed, 8 Oct 2025 14:12:24 +0200 Subject: [PATCH 3/9] feat: Add concurrent.futures patch to threading integration (#4770) Automatically fork isolation and current scopes when running tasks with `concurrent.future`. Packages the implementation from https://github.com/getsentry/sentry-python/issues/4508#issuecomment-3003526348 as an integration. Closes https://github.com/getsentry/sentry-python/issues/4565 --------- Co-authored-by: Anton Pirker --- sentry_sdk/integrations/threading.py | 60 +++++++++++++++--- .../integrations/threading/test_threading.py | 61 +++++++++++++++++++ 2 files changed, 113 insertions(+), 8 deletions(-) diff --git a/sentry_sdk/integrations/threading.py b/sentry_sdk/integrations/threading.py index c031c51f50..cfe54c829c 100644 --- a/sentry_sdk/integrations/threading.py +++ b/sentry_sdk/integrations/threading.py @@ -2,6 +2,7 @@ import warnings from functools import wraps from threading import Thread, current_thread +from concurrent.futures import ThreadPoolExecutor, Future import sentry_sdk from sentry_sdk.integrations import Integration @@ -24,6 +25,7 @@ from sentry_sdk._types import ExcInfo F = TypeVar("F", bound=Callable[..., Any]) + T = TypeVar("T", bound=Any) class ThreadingIntegration(Integration): @@ -59,6 +61,15 @@ def setup_once(): django_version = None channels_version = None + is_async_emulated_with_threads = ( + sys.version_info < (3, 9) + and channels_version is not None + and channels_version < "4.0.0" + and django_version is not None + and django_version >= (3, 0) + and django_version < (4, 0) + ) + @wraps(old_start) def sentry_start(self, *a, **kw): # type: (Thread, *Any, **Any) -> Any @@ -67,14 +78,7 @@ def sentry_start(self, *a, **kw): return old_start(self, *a, **kw) if integration.propagate_scope: - if ( - sys.version_info < (3, 9) - and channels_version is not None - and channels_version < "4.0.0" - and django_version is not None - and django_version >= (3, 0) - and django_version < (4, 0) - ): + if is_async_emulated_with_threads: warnings.warn( "There is a known issue with Django channels 2.x and 3.x when using Python 3.8 or older. " "(Async support is emulated using threads and some Sentry data may be leaked between those threads.) " @@ -109,6 +113,9 @@ def sentry_start(self, *a, **kw): return old_start(self, *a, **kw) Thread.start = sentry_start # type: ignore + ThreadPoolExecutor.submit = _wrap_threadpool_executor_submit( # type: ignore + ThreadPoolExecutor.submit, is_async_emulated_with_threads + ) def _wrap_run(isolation_scope_to_use, current_scope_to_use, old_run_func): @@ -134,6 +141,43 @@ def _run_old_run_func(): return run # type: ignore +def _wrap_threadpool_executor_submit(func, is_async_emulated_with_threads): + # type: (Callable[..., Future[T]], bool) -> Callable[..., Future[T]] + """ + Wrap submit call to propagate scopes on task submission. + """ + + @wraps(func) + def sentry_submit(self, fn, *args, **kwargs): + # type: (ThreadPoolExecutor, Callable[..., T], *Any, **Any) -> Future[T] + integration = sentry_sdk.get_client().get_integration(ThreadingIntegration) + if integration is None: + return func(self, fn, *args, **kwargs) + + if integration.propagate_scope and is_async_emulated_with_threads: + isolation_scope = sentry_sdk.get_isolation_scope() + current_scope = sentry_sdk.get_current_scope() + elif integration.propagate_scope: + isolation_scope = sentry_sdk.get_isolation_scope().fork() + current_scope = sentry_sdk.get_current_scope().fork() + else: + isolation_scope = None + current_scope = None + + def wrapped_fn(*args, **kwargs): + # type: (*Any, **Any) -> Any + if isolation_scope is not None and current_scope is not None: + with use_isolation_scope(isolation_scope): + with use_scope(current_scope): + return fn(*args, **kwargs) + + return fn(*args, **kwargs) + + return func(self, wrapped_fn, *args, **kwargs) + + return sentry_submit + + def _capture_exception(): # type: () -> ExcInfo exc_info = sys.exc_info() diff --git a/tests/integrations/threading/test_threading.py b/tests/integrations/threading/test_threading.py index 799298910b..9c9a24aa63 100644 --- a/tests/integrations/threading/test_threading.py +++ b/tests/integrations/threading/test_threading.py @@ -276,3 +276,64 @@ def do_some_work(number): - op="outer-submit-4": description="Thread: main"\ """ ) + + +@pytest.mark.parametrize( + "propagate_scope", + (True, False), + ids=["propagate_scope=True", "propagate_scope=False"], +) +def test_spans_from_threadpool( + sentry_init, capture_events, render_span_tree, propagate_scope +): + sentry_init( + traces_sample_rate=1.0, + integrations=[ThreadingIntegration(propagate_scope=propagate_scope)], + ) + events = capture_events() + + def do_some_work(number): + with sentry_sdk.start_span( + op=f"inner-run-{number}", name=f"Thread: child-{number}" + ): + pass + + with sentry_sdk.start_transaction(op="outer-trx"): + with futures.ThreadPoolExecutor(max_workers=1) as executor: + for number in range(5): + with sentry_sdk.start_span( + op=f"outer-submit-{number}", name="Thread: main" + ): + future = executor.submit(do_some_work, number) + future.result() + + (event,) = events + + if propagate_scope: + assert render_span_tree(event) == dedent( + """\ + - op="outer-trx": description=null + - op="outer-submit-0": description="Thread: main" + - op="inner-run-0": description="Thread: child-0" + - op="outer-submit-1": description="Thread: main" + - op="inner-run-1": description="Thread: child-1" + - op="outer-submit-2": description="Thread: main" + - op="inner-run-2": description="Thread: child-2" + - op="outer-submit-3": description="Thread: main" + - op="inner-run-3": description="Thread: child-3" + - op="outer-submit-4": description="Thread: main" + - op="inner-run-4": description="Thread: child-4"\ +""" + ) + + elif not propagate_scope: + assert render_span_tree(event) == dedent( + """\ + - op="outer-trx": description=null + - op="outer-submit-0": description="Thread: main" + - op="outer-submit-1": description="Thread: main" + - op="outer-submit-2": description="Thread: main" + - op="outer-submit-3": description="Thread: main" + - op="outer-submit-4": description="Thread: main"\ +""" + ) From 55e903e2c52b3026d434e67a4b18b04878894a0f Mon Sep 17 00:00:00 2001 From: Ivana Kellyer Date: Wed, 8 Oct 2025 15:35:05 +0200 Subject: [PATCH 4/9] ci: Bump Python version for linting (#4897) ### Description Python 3.14 is out, let's use it for linting. #### Issues Ref https://github.com/getsentry/sentry-python/issues/4895 #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --- .github/workflows/ci.yml | 2 +- scripts/populate_tox/tox.jinja | 2 +- tox.ini | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8ea11db711..43f9d296a1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -27,7 +27,7 @@ jobs: - uses: actions/checkout@v5.0.0 - uses: actions/setup-python@v6 with: - python-version: 3.12 + python-version: 3.14 - run: | pip install tox diff --git a/scripts/populate_tox/tox.jinja b/scripts/populate_tox/tox.jinja index 40ea309b08..b86da57c24 100755 --- a/scripts/populate_tox/tox.jinja +++ b/scripts/populate_tox/tox.jinja @@ -181,7 +181,7 @@ basepython = # Python version is pinned here for consistency across environments. # Tools like ruff and mypy have options that pin the target Python # version (configured in pyproject.toml), ensuring consistent behavior. - linters: python3.12 + linters: python3.14 commands = {py3.7,py3.8}-boto3: pip install urllib3<2.0.0 diff --git a/tox.ini b/tox.ini index 2c77edd07c..4bfc90cee9 100644 --- a/tox.ini +++ b/tox.ini @@ -812,7 +812,7 @@ basepython = # Python version is pinned here for consistency across environments. # Tools like ruff and mypy have options that pin the target Python # version (configured in pyproject.toml), ensuring consistent behavior. - linters: python3.12 + linters: python3.14 commands = {py3.7,py3.8}-boto3: pip install urllib3<2.0.0 From 79973687822f0af0d7cda10e92a79aac7393a097 Mon Sep 17 00:00:00 2001 From: Ivana Kellyer Date: Thu, 9 Oct 2025 10:27:36 +0200 Subject: [PATCH 5/9] chore: Remove old metrics code (#4899) ### Description Remove old metrics code to make way for https://github.com/getsentry/sentry-python/pull/4898 Metrics was always an experimental feature and Sentry stopped accepting metrics a year ago. #### Issues #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --- sentry_sdk/_types.py | 22 - sentry_sdk/client.py | 27 -- sentry_sdk/consts.py | 7 - sentry_sdk/envelope.py | 4 +- sentry_sdk/metrics.py | 971 ---------------------------------------- sentry_sdk/tracing.py | 26 -- sentry_sdk/transport.py | 18 +- tests/test_envelope.py | 1 - tests/test_metrics.py | 971 ---------------------------------------- tests/test_transport.py | 111 ----- 10 files changed, 2 insertions(+), 2156 deletions(-) delete mode 100644 sentry_sdk/metrics.py delete mode 100644 tests/test_metrics.py diff --git a/sentry_sdk/_types.py b/sentry_sdk/_types.py index b28c7260ce..d057f215e4 100644 --- a/sentry_sdk/_types.py +++ b/sentry_sdk/_types.py @@ -210,7 +210,6 @@ class SDKInfo(TypedDict): "type": Literal["check_in", "transaction"], "user": dict[str, object], "_dropped_spans": int, - "_metrics_summary": dict[str, object], }, total=False, ) @@ -266,7 +265,6 @@ class SDKInfo(TypedDict): "internal", "profile", "profile_chunk", - "metric_bucket", "monitor", "span", "log_item", @@ -276,26 +274,6 @@ class SDKInfo(TypedDict): ContinuousProfilerMode = Literal["thread", "gevent", "unknown"] ProfilerMode = Union[ContinuousProfilerMode, Literal["sleep"]] - # Type of the metric. - MetricType = Literal["d", "s", "g", "c"] - - # Value of the metric. - MetricValue = Union[int, float, str] - - # Internal representation of tags as a tuple of tuples (this is done in order to allow for the same key to exist - # multiple times). - MetricTagsInternal = Tuple[Tuple[str, str], ...] - - # External representation of tags as a dictionary. - MetricTagValue = Union[str, int, float, None] - MetricTags = Mapping[str, MetricTagValue] - - # Value inside the generator for the metric value. - FlushedMetricValue = Union[int, float] - - BucketKey = Tuple[MetricType, str, MeasurementUnit, MetricTagsInternal] - MetricMetaKey = Tuple[MetricType, str, MeasurementUnit] - MonitorConfigScheduleType = Literal["crontab", "interval"] MonitorConfigScheduleUnit = Literal[ "year", diff --git a/sentry_sdk/client.py b/sentry_sdk/client.py index c06043ebe2..59a5013783 100644 --- a/sentry_sdk/client.py +++ b/sentry_sdk/client.py @@ -61,7 +61,6 @@ from sentry_sdk._types import Event, Hint, SDKInfo, Log from sentry_sdk.integrations import Integration - from sentry_sdk.metrics import MetricsAggregator from sentry_sdk.scope import Scope from sentry_sdk.session import Session from sentry_sdk.spotlight import SpotlightClient @@ -182,7 +181,6 @@ def __init__(self, options=None): self.transport = None # type: Optional[Transport] self.monitor = None # type: Optional[Monitor] - self.metrics_aggregator = None # type: Optional[MetricsAggregator] self.log_batcher = None # type: Optional[LogBatcher] def __getstate__(self, *args, **kwargs): @@ -361,26 +359,6 @@ def _capture_envelope(envelope): self.session_flusher = SessionFlusher(capture_func=_capture_envelope) - self.metrics_aggregator = None # type: Optional[MetricsAggregator] - experiments = self.options.get("_experiments", {}) - if experiments.get("enable_metrics", True): - # Context vars are not working correctly on Python <=3.6 - # with gevent. - metrics_supported = not is_gevent() or PY37 - if metrics_supported: - from sentry_sdk.metrics import MetricsAggregator - - self.metrics_aggregator = MetricsAggregator( - capture_func=_capture_envelope, - enable_code_locations=bool( - experiments.get("metric_code_locations", True) - ), - ) - else: - logger.info( - "Metrics not supported on Python 3.6 and lower with gevent." - ) - self.log_batcher = None if has_logs_enabled(self.options): @@ -467,7 +445,6 @@ def _capture_envelope(envelope): if ( self.monitor - or self.metrics_aggregator or self.log_batcher or has_profiling_enabled(self.options) or isinstance(self.transport, BaseHttpTransport) @@ -1019,8 +996,6 @@ def close( if self.transport is not None: self.flush(timeout=timeout, callback=callback) self.session_flusher.kill() - if self.metrics_aggregator is not None: - self.metrics_aggregator.kill() if self.log_batcher is not None: self.log_batcher.kill() if self.monitor: @@ -1045,8 +1020,6 @@ def flush( if timeout is None: timeout = self.options["shutdown_timeout"] self.session_flusher.flush() - if self.metrics_aggregator is not None: - self.metrics_aggregator.flush() if self.log_batcher is not None: self.log_batcher.flush() self.transport.flush(timeout=timeout, callback=callback) diff --git a/sentry_sdk/consts.py b/sentry_sdk/consts.py index 43c7e857ac..0f71a0d460 100644 --- a/sentry_sdk/consts.py +++ b/sentry_sdk/consts.py @@ -55,8 +55,6 @@ class CompressionAlgo(Enum): ProfilerMode, TracesSampler, TransactionProcessor, - MetricTags, - MetricValue, ) # Experiments are feature flags to enable and disable certain unstable SDK @@ -77,11 +75,6 @@ class CompressionAlgo(Enum): "transport_compression_algo": Optional[CompressionAlgo], "transport_num_pools": Optional[int], "transport_http2": Optional[bool], - "enable_metrics": Optional[bool], - "before_emit_metric": Optional[ - Callable[[str, MetricValue, MeasurementUnit, MetricTags], bool] - ], - "metric_code_locations": Optional[bool], "enable_logs": Optional[bool], "before_send_log": Optional[Callable[[Log, Hint], Optional[Log]]], }, diff --git a/sentry_sdk/envelope.py b/sentry_sdk/envelope.py index d9b2c1629a..b26c458d41 100644 --- a/sentry_sdk/envelope.py +++ b/sentry_sdk/envelope.py @@ -291,8 +291,6 @@ def data_category(self): return "profile" elif ty == "profile_chunk": return "profile_chunk" - elif ty == "statsd": - return "metric_bucket" elif ty == "check_in": return "monitor" else: @@ -354,7 +352,7 @@ def deserialize_from( # if no length was specified we need to read up to the end of line # and remove it (if it is present, i.e. not the very last char in an eof terminated envelope) payload = f.readline().rstrip(b"\n") - if headers.get("type") in ("event", "transaction", "metric_buckets"): + if headers.get("type") in ("event", "transaction"): rv = cls(headers=headers, payload=PayloadRef(json=parse_json(payload))) else: rv = cls(headers=headers, payload=payload) diff --git a/sentry_sdk/metrics.py b/sentry_sdk/metrics.py deleted file mode 100644 index d0041114ce..0000000000 --- a/sentry_sdk/metrics.py +++ /dev/null @@ -1,971 +0,0 @@ -import io -import os -import random -import re -import sys -import threading -import time -import warnings -import zlib -from abc import ABC, abstractmethod -from contextlib import contextmanager -from datetime import datetime, timezone -from functools import wraps, partial - -import sentry_sdk -from sentry_sdk.utils import ( - ContextVar, - now, - nanosecond_time, - to_timestamp, - serialize_frame, - json_dumps, -) -from sentry_sdk.envelope import Envelope, Item -from sentry_sdk.tracing import TransactionSource - -from typing import TYPE_CHECKING - -if TYPE_CHECKING: - from typing import Any - from typing import Callable - from typing import Dict - from typing import Generator - from typing import Iterable - from typing import List - from typing import Optional - from typing import Set - from typing import Tuple - from typing import Union - - from sentry_sdk._types import BucketKey - from sentry_sdk._types import DurationUnit - from sentry_sdk._types import FlushedMetricValue - from sentry_sdk._types import MeasurementUnit - from sentry_sdk._types import MetricMetaKey - from sentry_sdk._types import MetricTagValue - from sentry_sdk._types import MetricTags - from sentry_sdk._types import MetricTagsInternal - from sentry_sdk._types import MetricType - from sentry_sdk._types import MetricValue - - -warnings.warn( - "The sentry_sdk.metrics module is deprecated and will be removed in the next major release. " - "Sentry will reject all metrics sent after October 7, 2024. " - "Learn more: https://sentry.zendesk.com/hc/en-us/articles/26369339769883-Upcoming-API-Changes-to-Metrics", - DeprecationWarning, - stacklevel=2, -) - -_in_metrics = ContextVar("in_metrics", default=False) -_set = set # set is shadowed below - -GOOD_TRANSACTION_SOURCES = frozenset( - [ - TransactionSource.ROUTE, - TransactionSource.VIEW, - TransactionSource.COMPONENT, - TransactionSource.TASK, - ] -) - -_sanitize_unit = partial(re.compile(r"[^a-zA-Z0-9_]+").sub, "") -_sanitize_metric_key = partial(re.compile(r"[^a-zA-Z0-9_\-.]+").sub, "_") -_sanitize_tag_key = partial(re.compile(r"[^a-zA-Z0-9_\-.\/]+").sub, "") - - -def _sanitize_tag_value(value): - # type: (str) -> str - table = str.maketrans( - { - "\n": "\\n", - "\r": "\\r", - "\t": "\\t", - "\\": "\\\\", - "|": "\\u{7c}", - ",": "\\u{2c}", - } - ) - return value.translate(table) - - -def get_code_location(stacklevel): - # type: (int) -> Optional[Dict[str, Any]] - try: - frm = sys._getframe(stacklevel) - except Exception: - return None - - return serialize_frame( - frm, include_local_variables=False, include_source_context=True - ) - - -@contextmanager -def recursion_protection(): - # type: () -> Generator[bool, None, None] - """Enters recursion protection and returns the old flag.""" - old_in_metrics = _in_metrics.get() - _in_metrics.set(True) - try: - yield old_in_metrics - finally: - _in_metrics.set(old_in_metrics) - - -def metrics_noop(func): - # type: (Any) -> Any - """Convenient decorator that uses `recursion_protection` to - make a function a noop. - """ - - @wraps(func) - def new_func(*args, **kwargs): - # type: (*Any, **Any) -> Any - with recursion_protection() as in_metrics: - if not in_metrics: - return func(*args, **kwargs) - - return new_func - - -class Metric(ABC): - __slots__ = () - - @abstractmethod - def __init__(self, first): - # type: (MetricValue) -> None - pass - - @property - @abstractmethod - def weight(self): - # type: () -> int - pass - - @abstractmethod - def add(self, value): - # type: (MetricValue) -> None - pass - - @abstractmethod - def serialize_value(self): - # type: () -> Iterable[FlushedMetricValue] - pass - - -class CounterMetric(Metric): - __slots__ = ("value",) - - def __init__( - self, - first, # type: MetricValue - ): - # type: (...) -> None - self.value = float(first) - - @property - def weight(self): - # type: (...) -> int - return 1 - - def add( - self, - value, # type: MetricValue - ): - # type: (...) -> None - self.value += float(value) - - def serialize_value(self): - # type: (...) -> Iterable[FlushedMetricValue] - return (self.value,) - - -class GaugeMetric(Metric): - __slots__ = ( - "last", - "min", - "max", - "sum", - "count", - ) - - def __init__( - self, - first, # type: MetricValue - ): - # type: (...) -> None - first = float(first) - self.last = first - self.min = first - self.max = first - self.sum = first - self.count = 1 - - @property - def weight(self): - # type: (...) -> int - # Number of elements. - return 5 - - def add( - self, - value, # type: MetricValue - ): - # type: (...) -> None - value = float(value) - self.last = value - self.min = min(self.min, value) - self.max = max(self.max, value) - self.sum += value - self.count += 1 - - def serialize_value(self): - # type: (...) -> Iterable[FlushedMetricValue] - return ( - self.last, - self.min, - self.max, - self.sum, - self.count, - ) - - -class DistributionMetric(Metric): - __slots__ = ("value",) - - def __init__( - self, - first, # type: MetricValue - ): - # type(...) -> None - self.value = [float(first)] - - @property - def weight(self): - # type: (...) -> int - return len(self.value) - - def add( - self, - value, # type: MetricValue - ): - # type: (...) -> None - self.value.append(float(value)) - - def serialize_value(self): - # type: (...) -> Iterable[FlushedMetricValue] - return self.value - - -class SetMetric(Metric): - __slots__ = ("value",) - - def __init__( - self, - first, # type: MetricValue - ): - # type: (...) -> None - self.value = {first} - - @property - def weight(self): - # type: (...) -> int - return len(self.value) - - def add( - self, - value, # type: MetricValue - ): - # type: (...) -> None - self.value.add(value) - - def serialize_value(self): - # type: (...) -> Iterable[FlushedMetricValue] - def _hash(x): - # type: (MetricValue) -> int - if isinstance(x, str): - return zlib.crc32(x.encode("utf-8")) & 0xFFFFFFFF - return int(x) - - return (_hash(value) for value in self.value) - - -def _encode_metrics(flushable_buckets): - # type: (Iterable[Tuple[int, Dict[BucketKey, Metric]]]) -> bytes - out = io.BytesIO() - _write = out.write - - # Note on sanitization: we intentionally sanitize in emission (serialization) - # and not during aggregation for performance reasons. This means that the - # envelope can in fact have duplicate buckets stored. This is acceptable for - # relay side emission and should not happen commonly. - - for timestamp, buckets in flushable_buckets: - for bucket_key, metric in buckets.items(): - metric_type, metric_name, metric_unit, metric_tags = bucket_key - metric_name = _sanitize_metric_key(metric_name) - metric_unit = _sanitize_unit(metric_unit) - _write(metric_name.encode("utf-8")) - _write(b"@") - _write(metric_unit.encode("utf-8")) - - for serialized_value in metric.serialize_value(): - _write(b":") - _write(str(serialized_value).encode("utf-8")) - - _write(b"|") - _write(metric_type.encode("ascii")) - - if metric_tags: - _write(b"|#") - first = True - for tag_key, tag_value in metric_tags: - tag_key = _sanitize_tag_key(tag_key) - if not tag_key: - continue - if first: - first = False - else: - _write(b",") - _write(tag_key.encode("utf-8")) - _write(b":") - _write(_sanitize_tag_value(tag_value).encode("utf-8")) - - _write(b"|T") - _write(str(timestamp).encode("ascii")) - _write(b"\n") - - return out.getvalue() - - -def _encode_locations(timestamp, code_locations): - # type: (int, Iterable[Tuple[MetricMetaKey, Dict[str, Any]]]) -> bytes - mapping = {} # type: Dict[str, List[Any]] - - for key, loc in code_locations: - metric_type, name, unit = key - mri = "{}:{}@{}".format( - metric_type, _sanitize_metric_key(name), _sanitize_unit(unit) - ) - - loc["type"] = "location" - mapping.setdefault(mri, []).append(loc) - - return json_dumps({"timestamp": timestamp, "mapping": mapping}) - - -METRIC_TYPES = { - "c": CounterMetric, - "g": GaugeMetric, - "d": DistributionMetric, - "s": SetMetric, -} # type: dict[MetricType, type[Metric]] - -# some of these are dumb -TIMING_FUNCTIONS = { - "nanosecond": nanosecond_time, - "microsecond": lambda: nanosecond_time() / 1000.0, - "millisecond": lambda: nanosecond_time() / 1000000.0, - "second": now, - "minute": lambda: now() / 60.0, - "hour": lambda: now() / 3600.0, - "day": lambda: now() / 3600.0 / 24.0, - "week": lambda: now() / 3600.0 / 24.0 / 7.0, -} - - -class LocalAggregator: - __slots__ = ("_measurements",) - - def __init__(self): - # type: (...) -> None - self._measurements = {} # type: Dict[Tuple[str, MetricTagsInternal], Tuple[float, float, int, float]] - - def add( - self, - ty, # type: MetricType - key, # type: str - value, # type: float - unit, # type: MeasurementUnit - tags, # type: MetricTagsInternal - ): - # type: (...) -> None - export_key = "%s:%s@%s" % (ty, key, unit) - bucket_key = (export_key, tags) - - old = self._measurements.get(bucket_key) - if old is not None: - v_min, v_max, v_count, v_sum = old - v_min = min(v_min, value) - v_max = max(v_max, value) - v_count += 1 - v_sum += value - else: - v_min = v_max = v_sum = value - v_count = 1 - self._measurements[bucket_key] = (v_min, v_max, v_count, v_sum) - - def to_json(self): - # type: (...) -> Dict[str, Any] - rv = {} # type: Any - for (export_key, tags), ( - v_min, - v_max, - v_count, - v_sum, - ) in self._measurements.items(): - rv.setdefault(export_key, []).append( - { - "tags": _tags_to_dict(tags), - "min": v_min, - "max": v_max, - "count": v_count, - "sum": v_sum, - } - ) - return rv - - -class MetricsAggregator: - ROLLUP_IN_SECONDS = 10.0 - MAX_WEIGHT = 100000 - FLUSHER_SLEEP_TIME = 5.0 - - def __init__( - self, - capture_func, # type: Callable[[Envelope], None] - enable_code_locations=False, # type: bool - ): - # type: (...) -> None - self.buckets = {} # type: Dict[int, Any] - self._enable_code_locations = enable_code_locations - self._seen_locations = _set() # type: Set[Tuple[int, MetricMetaKey]] - self._pending_locations = {} # type: Dict[int, List[Tuple[MetricMetaKey, Any]]] - self._buckets_total_weight = 0 - self._capture_func = capture_func - self._running = True - self._lock = threading.Lock() - - self._flush_event = threading.Event() # type: threading.Event - self._force_flush = False - - # The aggregator shifts its flushing by up to an entire rollup window to - # avoid multiple clients trampling on end of a 10 second window as all the - # buckets are anchored to multiples of ROLLUP seconds. We randomize this - # number once per aggregator boot to achieve some level of offsetting - # across a fleet of deployed SDKs. Relay itself will also apply independent - # jittering. - self._flush_shift = random.random() * self.ROLLUP_IN_SECONDS - - self._flusher = None # type: Optional[threading.Thread] - self._flusher_pid = None # type: Optional[int] - - def _ensure_thread(self): - # type: (...) -> bool - """For forking processes we might need to restart this thread. - This ensures that our process actually has that thread running. - """ - if not self._running: - return False - - pid = os.getpid() - if self._flusher_pid == pid: - return True - - with self._lock: - # Recheck to make sure another thread didn't get here and start the - # the flusher in the meantime - if self._flusher_pid == pid: - return True - - self._flusher_pid = pid - - self._flusher = threading.Thread(target=self._flush_loop) - self._flusher.daemon = True - - try: - self._flusher.start() - except RuntimeError: - # Unfortunately at this point the interpreter is in a state that no - # longer allows us to spawn a thread and we have to bail. - self._running = False - return False - - return True - - def _flush_loop(self): - # type: (...) -> None - _in_metrics.set(True) - while self._running or self._force_flush: - if self._running: - self._flush_event.wait(self.FLUSHER_SLEEP_TIME) - self._flush() - - def _flush(self): - # type: (...) -> None - self._emit(self._flushable_buckets(), self._flushable_locations()) - - def _flushable_buckets(self): - # type: (...) -> (Iterable[Tuple[int, Dict[BucketKey, Metric]]]) - with self._lock: - force_flush = self._force_flush - cutoff = time.time() - self.ROLLUP_IN_SECONDS - self._flush_shift - flushable_buckets = () # type: Iterable[Tuple[int, Dict[BucketKey, Metric]]] - weight_to_remove = 0 - - if force_flush: - flushable_buckets = self.buckets.items() - self.buckets = {} - self._buckets_total_weight = 0 - self._force_flush = False - else: - flushable_buckets = [] - for buckets_timestamp, buckets in self.buckets.items(): - # If the timestamp of the bucket is newer that the rollup we want to skip it. - if buckets_timestamp <= cutoff: - flushable_buckets.append((buckets_timestamp, buckets)) - - # We will clear the elements while holding the lock, in order to avoid requesting it downstream again. - for buckets_timestamp, buckets in flushable_buckets: - for metric in buckets.values(): - weight_to_remove += metric.weight - del self.buckets[buckets_timestamp] - - self._buckets_total_weight -= weight_to_remove - - return flushable_buckets - - def _flushable_locations(self): - # type: (...) -> Dict[int, List[Tuple[MetricMetaKey, Dict[str, Any]]]] - with self._lock: - locations = self._pending_locations - self._pending_locations = {} - return locations - - @metrics_noop - def add( - self, - ty, # type: MetricType - key, # type: str - value, # type: MetricValue - unit, # type: MeasurementUnit - tags, # type: Optional[MetricTags] - timestamp=None, # type: Optional[Union[float, datetime]] - local_aggregator=None, # type: Optional[LocalAggregator] - stacklevel=0, # type: Optional[int] - ): - # type: (...) -> None - if not self._ensure_thread() or self._flusher is None: - return None - - if timestamp is None: - timestamp = time.time() - elif isinstance(timestamp, datetime): - timestamp = to_timestamp(timestamp) - - bucket_timestamp = int( - (timestamp // self.ROLLUP_IN_SECONDS) * self.ROLLUP_IN_SECONDS - ) - serialized_tags = _serialize_tags(tags) - bucket_key = ( - ty, - key, - unit, - serialized_tags, - ) - - with self._lock: - local_buckets = self.buckets.setdefault(bucket_timestamp, {}) - metric = local_buckets.get(bucket_key) - if metric is not None: - previous_weight = metric.weight - metric.add(value) - else: - metric = local_buckets[bucket_key] = METRIC_TYPES[ty](value) - previous_weight = 0 - - added = metric.weight - previous_weight - - if stacklevel is not None: - self.record_code_location(ty, key, unit, stacklevel + 2, timestamp) - - # Given the new weight we consider whether we want to force flush. - self._consider_force_flush() - - # For sets, we only record that a value has been added to the set but not which one. - # See develop docs: https://develop.sentry.dev/sdk/metrics/#sets - if local_aggregator is not None: - local_value = float(added if ty == "s" else value) - local_aggregator.add(ty, key, local_value, unit, serialized_tags) - - def record_code_location( - self, - ty, # type: MetricType - key, # type: str - unit, # type: MeasurementUnit - stacklevel, # type: int - timestamp=None, # type: Optional[float] - ): - # type: (...) -> None - if not self._enable_code_locations: - return - if timestamp is None: - timestamp = time.time() - meta_key = (ty, key, unit) - start_of_day = datetime.fromtimestamp(timestamp, timezone.utc).replace( - hour=0, minute=0, second=0, microsecond=0, tzinfo=None - ) - start_of_day = int(to_timestamp(start_of_day)) - - if (start_of_day, meta_key) not in self._seen_locations: - self._seen_locations.add((start_of_day, meta_key)) - loc = get_code_location(stacklevel + 3) - if loc is not None: - # Group metadata by day to make flushing more efficient. - # There needs to be one envelope item per timestamp. - self._pending_locations.setdefault(start_of_day, []).append( - (meta_key, loc) - ) - - @metrics_noop - def need_code_location( - self, - ty, # type: MetricType - key, # type: str - unit, # type: MeasurementUnit - timestamp, # type: float - ): - # type: (...) -> bool - if self._enable_code_locations: - return False - meta_key = (ty, key, unit) - start_of_day = datetime.fromtimestamp(timestamp, timezone.utc).replace( - hour=0, minute=0, second=0, microsecond=0, tzinfo=None - ) - start_of_day = int(to_timestamp(start_of_day)) - return (start_of_day, meta_key) not in self._seen_locations - - def kill(self): - # type: (...) -> None - if self._flusher is None: - return - - self._running = False - self._flush_event.set() - self._flusher = None - - @metrics_noop - def flush(self): - # type: (...) -> None - self._force_flush = True - self._flush() - - def _consider_force_flush(self): - # type: (...) -> None - # It's important to acquire a lock around this method, since it will touch shared data structures. - total_weight = len(self.buckets) + self._buckets_total_weight - if total_weight >= self.MAX_WEIGHT: - self._force_flush = True - self._flush_event.set() - - def _emit( - self, - flushable_buckets, # type: (Iterable[Tuple[int, Dict[BucketKey, Metric]]]) - code_locations, # type: Dict[int, List[Tuple[MetricMetaKey, Dict[str, Any]]]] - ): - # type: (...) -> Optional[Envelope] - envelope = Envelope() - - if flushable_buckets: - encoded_metrics = _encode_metrics(flushable_buckets) - envelope.add_item(Item(payload=encoded_metrics, type="statsd")) - - for timestamp, locations in code_locations.items(): - encoded_locations = _encode_locations(timestamp, locations) - envelope.add_item(Item(payload=encoded_locations, type="metric_meta")) - - if envelope.items: - self._capture_func(envelope) - return envelope - return None - - -def _serialize_tags( - tags, # type: Optional[MetricTags] -): - # type: (...) -> MetricTagsInternal - if not tags: - return () - - rv = [] - for key, value in tags.items(): - # If the value is a collection, we want to flatten it. - if isinstance(value, (list, tuple)): - for inner_value in value: - if inner_value is not None: - rv.append((key, str(inner_value))) - elif value is not None: - rv.append((key, str(value))) - - # It's very important to sort the tags in order to obtain the - # same bucket key. - return tuple(sorted(rv)) - - -def _tags_to_dict(tags): - # type: (MetricTagsInternal) -> Dict[str, Any] - rv = {} # type: Dict[str, Any] - for tag_name, tag_value in tags: - old_value = rv.get(tag_name) - if old_value is not None: - if isinstance(old_value, list): - old_value.append(tag_value) - else: - rv[tag_name] = [old_value, tag_value] - else: - rv[tag_name] = tag_value - return rv - - -def _get_aggregator(): - # type: () -> Optional[MetricsAggregator] - client = sentry_sdk.get_client() - return ( - client.metrics_aggregator - if client.is_active() and client.metrics_aggregator is not None - else None - ) - - -def _get_aggregator_and_update_tags(key, value, unit, tags): - # type: (str, Optional[MetricValue], MeasurementUnit, Optional[MetricTags]) -> Tuple[Optional[MetricsAggregator], Optional[LocalAggregator], Optional[MetricTags]] - client = sentry_sdk.get_client() - if not client.is_active() or client.metrics_aggregator is None: - return None, None, tags - - updated_tags = dict(tags or ()) # type: Dict[str, MetricTagValue] - updated_tags.setdefault("release", client.options["release"]) - updated_tags.setdefault("environment", client.options["environment"]) - - scope = sentry_sdk.get_current_scope() - local_aggregator = None - - # We go with the low-level API here to access transaction information as - # this one is the same between just errors and errors + performance - transaction_source = scope._transaction_info.get("source") - if transaction_source in GOOD_TRANSACTION_SOURCES: - transaction_name = scope._transaction - if transaction_name: - updated_tags.setdefault("transaction", transaction_name) - if scope._span is not None: - local_aggregator = scope._span._get_local_aggregator() - - experiments = client.options.get("_experiments", {}) - before_emit_callback = experiments.get("before_emit_metric") - if before_emit_callback is not None: - with recursion_protection() as in_metrics: - if not in_metrics: - if not before_emit_callback(key, value, unit, updated_tags): - return None, None, updated_tags - - return client.metrics_aggregator, local_aggregator, updated_tags - - -def increment( - key, # type: str - value=1.0, # type: float - unit="none", # type: MeasurementUnit - tags=None, # type: Optional[MetricTags] - timestamp=None, # type: Optional[Union[float, datetime]] - stacklevel=0, # type: int -): - # type: (...) -> None - """Increments a counter.""" - aggregator, local_aggregator, tags = _get_aggregator_and_update_tags( - key, value, unit, tags - ) - if aggregator is not None: - aggregator.add( - "c", key, value, unit, tags, timestamp, local_aggregator, stacklevel - ) - - -# alias as incr is relatively common in python -incr = increment - - -class _Timing: - def __init__( - self, - key, # type: str - tags, # type: Optional[MetricTags] - timestamp, # type: Optional[Union[float, datetime]] - value, # type: Optional[float] - unit, # type: DurationUnit - stacklevel, # type: int - ): - # type: (...) -> None - self.key = key - self.tags = tags - self.timestamp = timestamp - self.value = value - self.unit = unit - self.entered = None # type: Optional[float] - self._span = None # type: Optional[sentry_sdk.tracing.Span] - self.stacklevel = stacklevel - - def _validate_invocation(self, context): - # type: (str) -> None - if self.value is not None: - raise TypeError( - "cannot use timing as %s when a value is provided" % context - ) - - def __enter__(self): - # type: (...) -> _Timing - self.entered = TIMING_FUNCTIONS[self.unit]() - self._validate_invocation("context-manager") - self._span = sentry_sdk.start_span(op="metric.timing", name=self.key) - if self.tags: - for key, value in self.tags.items(): - if isinstance(value, (tuple, list)): - value = ",".join(sorted(map(str, value))) - self._span.set_tag(key, value) - self._span.__enter__() - - # report code locations here for better accuracy - aggregator = _get_aggregator() - if aggregator is not None: - aggregator.record_code_location("d", self.key, self.unit, self.stacklevel) - - return self - - def __exit__(self, exc_type, exc_value, tb): - # type: (Any, Any, Any) -> None - assert self._span, "did not enter" - aggregator, local_aggregator, tags = _get_aggregator_and_update_tags( - self.key, - self.value, - self.unit, - self.tags, - ) - if aggregator is not None: - elapsed = TIMING_FUNCTIONS[self.unit]() - self.entered # type: ignore - aggregator.add( - "d", - self.key, - elapsed, - self.unit, - tags, - self.timestamp, - local_aggregator, - None, # code locations are reported in __enter__ - ) - - self._span.__exit__(exc_type, exc_value, tb) - self._span = None - - def __call__(self, f): - # type: (Any) -> Any - self._validate_invocation("decorator") - - @wraps(f) - def timed_func(*args, **kwargs): - # type: (*Any, **Any) -> Any - with timing( - key=self.key, - tags=self.tags, - timestamp=self.timestamp, - unit=self.unit, - stacklevel=self.stacklevel + 1, - ): - return f(*args, **kwargs) - - return timed_func - - -def timing( - key, # type: str - value=None, # type: Optional[float] - unit="second", # type: DurationUnit - tags=None, # type: Optional[MetricTags] - timestamp=None, # type: Optional[Union[float, datetime]] - stacklevel=0, # type: int -): - # type: (...) -> _Timing - """Emits a distribution with the time it takes to run the given code block. - - This method supports three forms of invocation: - - - when a `value` is provided, it functions similar to `distribution` but with - - it can be used as a context manager - - it can be used as a decorator - """ - if value is not None: - aggregator, local_aggregator, tags = _get_aggregator_and_update_tags( - key, value, unit, tags - ) - if aggregator is not None: - aggregator.add( - "d", key, value, unit, tags, timestamp, local_aggregator, stacklevel - ) - return _Timing(key, tags, timestamp, value, unit, stacklevel) - - -def distribution( - key, # type: str - value, # type: float - unit="none", # type: MeasurementUnit - tags=None, # type: Optional[MetricTags] - timestamp=None, # type: Optional[Union[float, datetime]] - stacklevel=0, # type: int -): - # type: (...) -> None - """Emits a distribution.""" - aggregator, local_aggregator, tags = _get_aggregator_and_update_tags( - key, value, unit, tags - ) - if aggregator is not None: - aggregator.add( - "d", key, value, unit, tags, timestamp, local_aggregator, stacklevel - ) - - -def set( - key, # type: str - value, # type: Union[int, str] - unit="none", # type: MeasurementUnit - tags=None, # type: Optional[MetricTags] - timestamp=None, # type: Optional[Union[float, datetime]] - stacklevel=0, # type: int -): - # type: (...) -> None - """Emits a set.""" - aggregator, local_aggregator, tags = _get_aggregator_and_update_tags( - key, value, unit, tags - ) - if aggregator is not None: - aggregator.add( - "s", key, value, unit, tags, timestamp, local_aggregator, stacklevel - ) - - -def gauge( - key, # type: str - value, # type: float - unit="none", # type: MeasurementUnit - tags=None, # type: Optional[MetricTags] - timestamp=None, # type: Optional[Union[float, datetime]] - stacklevel=0, # type: int -): - # type: (...) -> None - """Emits a gauge.""" - aggregator, local_aggregator, tags = _get_aggregator_and_update_tags( - key, value, unit, tags - ) - if aggregator is not None: - aggregator.add( - "g", key, value, unit, tags, timestamp, local_aggregator, stacklevel - ) diff --git a/sentry_sdk/tracing.py b/sentry_sdk/tracing.py index 1697df1f22..0d652e490a 100644 --- a/sentry_sdk/tracing.py +++ b/sentry_sdk/tracing.py @@ -276,7 +276,6 @@ class Span: "hub", "_context_manager_state", "_containing_transaction", - "_local_aggregator", "scope", "origin", "name", @@ -345,7 +344,6 @@ def __init__( self.timestamp = None # type: Optional[datetime] self._span_recorder = None # type: Optional[_SpanRecorder] - self._local_aggregator = None # type: Optional[LocalAggregator] self.update_active_thread() self.set_profiler_id(get_profiler_id()) @@ -383,13 +381,6 @@ def span_id(self, value): # type: (str) -> None self._span_id = value - def _get_local_aggregator(self): - # type: (...) -> LocalAggregator - rv = self._local_aggregator - if rv is None: - rv = self._local_aggregator = LocalAggregator() - return rv - def __repr__(self): # type: () -> str return ( @@ -741,11 +732,6 @@ def to_json(self): if self.status: self._tags["status"] = self.status - if self._local_aggregator is not None: - metrics_summary = self._local_aggregator.to_json() - if metrics_summary: - rv["_metrics_summary"] = metrics_summary - if len(self._measurements) > 0: rv["measurements"] = self._measurements @@ -1122,13 +1108,6 @@ def finish( event["measurements"] = self._measurements - # This is here since `to_json` is not invoked. This really should - # be gone when we switch to onlyspans. - if self._local_aggregator is not None: - metrics_summary = self._local_aggregator.to_json() - if metrics_summary: - event["_metrics_summary"] = metrics_summary - return scope.capture_event(event) def set_measurement(self, name, value, unit=""): @@ -1505,8 +1484,3 @@ def calculate_interest_rate(amount, rate, years): has_tracing_enabled, maybe_create_breadcrumbs_from_span, ) - -with warnings.catch_warnings(): - # The code in this file which uses `LocalAggregator` is only called from the deprecated `metrics` module. - warnings.simplefilter("ignore", DeprecationWarning) - from sentry_sdk.metrics import LocalAggregator diff --git a/sentry_sdk/transport.py b/sentry_sdk/transport.py index 75384519e9..645bfead19 100644 --- a/sentry_sdk/transport.py +++ b/sentry_sdk/transport.py @@ -171,17 +171,7 @@ def _parse_rate_limits(header, now=None): retry_after = now + timedelta(seconds=int(retry_after_val)) for category in categories and categories.split(";") or (None,): - if category == "metric_bucket": - try: - namespaces = parameters[4].split(";") - except IndexError: - namespaces = [] - - if not namespaces or "custom" in namespaces: - yield category, retry_after # type: ignore - - else: - yield category, retry_after # type: ignore + yield category, retry_after # type: ignore except (LookupError, ValueError): continue @@ -417,12 +407,6 @@ def _check_disabled(self, category): # type: (str) -> bool def _disabled(bucket): # type: (Any) -> bool - - # The envelope item type used for metrics is statsd - # whereas the rate limit category is metric_bucket - if bucket == "statsd": - bucket = "metric_bucket" - ts = self._disabled_until.get(bucket) return ts is not None and ts > datetime.now(timezone.utc) diff --git a/tests/test_envelope.py b/tests/test_envelope.py index 06f8971dc3..d66cd9460a 100644 --- a/tests/test_envelope.py +++ b/tests/test_envelope.py @@ -252,7 +252,6 @@ def test_envelope_item_data_category_mapping(): ("client_report", "internal"), ("profile", "profile"), ("profile_chunk", "profile_chunk"), - ("statsd", "metric_bucket"), ("check_in", "monitor"), ("unknown_type", "default"), ] diff --git a/tests/test_metrics.py b/tests/test_metrics.py deleted file mode 100644 index c02f075288..0000000000 --- a/tests/test_metrics.py +++ /dev/null @@ -1,971 +0,0 @@ -import sys -import time -import linecache -from unittest import mock - -import pytest - -import sentry_sdk -from sentry_sdk import metrics -from sentry_sdk.tracing import TransactionSource -from sentry_sdk.envelope import parse_json - -try: - import gevent -except ImportError: - gevent = None - - -minimum_python_37_with_gevent = pytest.mark.skipif( - gevent and sys.version_info < (3, 7), - reason="Require Python 3.7 or higher with gevent", -) - - -def parse_metrics(bytes): - rv = [] - for line in bytes.splitlines(): - pieces = line.decode("utf-8").split("|") - payload = pieces[0].split(":") - name = payload[0] - values = payload[1:] - ty = pieces[1] - ts = None - tags = {} - for piece in pieces[2:]: - if piece[0] == "#": - for pair in piece[1:].split(","): - k, v = pair.split(":", 1) - old = tags.get(k) - if old is not None: - if isinstance(old, list): - old.append(v) - else: - tags[k] = [old, v] - else: - tags[k] = v - elif piece[0] == "T": - ts = int(piece[1:]) - else: - raise ValueError("unknown piece %r" % (piece,)) - rv.append((ts, name, ty, values, tags)) - rv.sort(key=lambda x: (x[0], x[1], tuple(sorted(tags.items())))) - return rv - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_increment(sentry_init, capture_envelopes, maybe_monkeypatched_threading): - sentry_init( - release="fun-release", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": True}, - ) - ts = time.time() - envelopes = capture_envelopes() - - metrics.increment("foobar", 1.0, tags={"foo": "bar", "blub": "blah"}, timestamp=ts) - # python specific alias - metrics.incr("foobar", 2.0, tags={"foo": "bar", "blub": "blah"}, timestamp=ts) - sentry_sdk.flush() - - (envelope,) = envelopes - statsd_item, meta_item = envelope.items - - assert statsd_item.headers["type"] == "statsd" - m = parse_metrics(statsd_item.payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "foobar@none" - assert m[0][2] == "c" - assert m[0][3] == ["3.0"] - assert m[0][4] == { - "blub": "blah", - "foo": "bar", - "release": "fun-release", - "environment": "not-fun-env", - } - - assert meta_item.headers["type"] == "metric_meta" - assert parse_json(meta_item.payload.get_bytes()) == { - "timestamp": mock.ANY, - "mapping": { - "c:foobar@none": [ - { - "type": "location", - "filename": "tests/test_metrics.py", - "abs_path": __file__, - "function": sys._getframe().f_code.co_name, - "module": __name__, - "lineno": mock.ANY, - "pre_context": mock.ANY, - "context_line": mock.ANY, - "post_context": mock.ANY, - } - ] - }, - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_timing(sentry_init, capture_envelopes, maybe_monkeypatched_threading): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": True}, - ) - ts = time.time() - envelopes = capture_envelopes() - - with metrics.timing("whatever", tags={"blub": "blah"}, timestamp=ts): - time.sleep(0.1) - sentry_sdk.flush() - - (envelope,) = envelopes - statsd_item, meta_item = envelope.items - - assert statsd_item.headers["type"] == "statsd" - m = parse_metrics(statsd_item.payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "whatever@second" - assert m[0][2] == "d" - assert len(m[0][3]) == 1 - assert float(m[0][3][0]) >= 0.1 - assert m[0][4] == { - "blub": "blah", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert meta_item.headers["type"] == "metric_meta" - json = parse_json(meta_item.payload.get_bytes()) - assert json == { - "timestamp": mock.ANY, - "mapping": { - "d:whatever@second": [ - { - "type": "location", - "filename": "tests/test_metrics.py", - "abs_path": __file__, - "function": sys._getframe().f_code.co_name, - "module": __name__, - "lineno": mock.ANY, - "pre_context": mock.ANY, - "context_line": mock.ANY, - "post_context": mock.ANY, - } - ] - }, - } - - loc = json["mapping"]["d:whatever@second"][0] - line = linecache.getline(loc["abs_path"], loc["lineno"]) - assert ( - line.strip() - == 'with metrics.timing("whatever", tags={"blub": "blah"}, timestamp=ts):' - ) - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_timing_decorator( - sentry_init, capture_envelopes, maybe_monkeypatched_threading -): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": True}, - ) - envelopes = capture_envelopes() - - @metrics.timing("whatever-1", tags={"x": "y"}) - def amazing(): - time.sleep(0.1) - return 42 - - @metrics.timing("whatever-2", tags={"x": "y"}, unit="nanosecond") - def amazing_nano(): - time.sleep(0.01) - return 23 - - assert amazing() == 42 - assert amazing_nano() == 23 - sentry_sdk.flush() - - (envelope,) = envelopes - statsd_item, meta_item = envelope.items - - assert statsd_item.headers["type"] == "statsd" - m = parse_metrics(statsd_item.payload.get_bytes()) - - assert len(m) == 2 - assert m[0][1] == "whatever-1@second" - assert m[0][2] == "d" - assert len(m[0][3]) == 1 - assert float(m[0][3][0]) >= 0.1 - assert m[0][4] == { - "x": "y", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert m[1][1] == "whatever-2@nanosecond" - assert m[1][2] == "d" - assert len(m[1][3]) == 1 - assert float(m[1][3][0]) >= 10000000.0 - assert m[1][4] == { - "x": "y", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert meta_item.headers["type"] == "metric_meta" - json = parse_json(meta_item.payload.get_bytes()) - assert json == { - "timestamp": mock.ANY, - "mapping": { - "d:whatever-1@second": [ - { - "type": "location", - "filename": "tests/test_metrics.py", - "abs_path": __file__, - "function": sys._getframe().f_code.co_name, - "module": __name__, - "lineno": mock.ANY, - "pre_context": mock.ANY, - "context_line": mock.ANY, - "post_context": mock.ANY, - } - ], - "d:whatever-2@nanosecond": [ - { - "type": "location", - "filename": "tests/test_metrics.py", - "abs_path": __file__, - "function": sys._getframe().f_code.co_name, - "module": __name__, - "lineno": mock.ANY, - "pre_context": mock.ANY, - "context_line": mock.ANY, - "post_context": mock.ANY, - } - ], - }, - } - - # XXX: this is not the best location. It would probably be better to - # report the location in the function, however that is quite a bit - # tricker to do since we report from outside the function so we really - # only see the callsite. - loc = json["mapping"]["d:whatever-1@second"][0] - line = linecache.getline(loc["abs_path"], loc["lineno"]) - assert line.strip() == "assert amazing() == 42" - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_timing_basic(sentry_init, capture_envelopes, maybe_monkeypatched_threading): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": True}, - ) - ts = time.time() - envelopes = capture_envelopes() - - metrics.timing("timing", 1.0, tags={"a": "b"}, timestamp=ts) - metrics.timing("timing", 2.0, tags={"a": "b"}, timestamp=ts) - metrics.timing("timing", 2.0, tags={"a": "b"}, timestamp=ts) - metrics.timing("timing", 3.0, tags={"a": "b"}, timestamp=ts) - sentry_sdk.flush() - - (envelope,) = envelopes - statsd_item, meta_item = envelope.items - - assert statsd_item.headers["type"] == "statsd" - m = parse_metrics(statsd_item.payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "timing@second" - assert m[0][2] == "d" - assert len(m[0][3]) == 4 - assert sorted(map(float, m[0][3])) == [1.0, 2.0, 2.0, 3.0] - assert m[0][4] == { - "a": "b", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert meta_item.headers["type"] == "metric_meta" - assert parse_json(meta_item.payload.get_bytes()) == { - "timestamp": mock.ANY, - "mapping": { - "d:timing@second": [ - { - "type": "location", - "filename": "tests/test_metrics.py", - "abs_path": __file__, - "function": sys._getframe().f_code.co_name, - "module": __name__, - "lineno": mock.ANY, - "pre_context": mock.ANY, - "context_line": mock.ANY, - "post_context": mock.ANY, - } - ] - }, - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_distribution(sentry_init, capture_envelopes, maybe_monkeypatched_threading): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": True}, - ) - ts = time.time() - envelopes = capture_envelopes() - - metrics.distribution("dist", 1.0, tags={"a": "b"}, timestamp=ts) - metrics.distribution("dist", 2.0, tags={"a": "b"}, timestamp=ts) - metrics.distribution("dist", 2.0, tags={"a": "b"}, timestamp=ts) - metrics.distribution("dist", 3.0, tags={"a": "b"}, timestamp=ts) - sentry_sdk.flush() - - (envelope,) = envelopes - statsd_item, meta_item = envelope.items - - assert statsd_item.headers["type"] == "statsd" - m = parse_metrics(statsd_item.payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "dist@none" - assert m[0][2] == "d" - assert len(m[0][3]) == 4 - assert sorted(map(float, m[0][3])) == [1.0, 2.0, 2.0, 3.0] - assert m[0][4] == { - "a": "b", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert meta_item.headers["type"] == "metric_meta" - json = parse_json(meta_item.payload.get_bytes()) - assert json == { - "timestamp": mock.ANY, - "mapping": { - "d:dist@none": [ - { - "type": "location", - "filename": "tests/test_metrics.py", - "abs_path": __file__, - "function": sys._getframe().f_code.co_name, - "module": __name__, - "lineno": mock.ANY, - "pre_context": mock.ANY, - "context_line": mock.ANY, - "post_context": mock.ANY, - } - ] - }, - } - - loc = json["mapping"]["d:dist@none"][0] - line = linecache.getline(loc["abs_path"], loc["lineno"]) - assert ( - line.strip() - == 'metrics.distribution("dist", 1.0, tags={"a": "b"}, timestamp=ts)' - ) - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_set(sentry_init, capture_envelopes, maybe_monkeypatched_threading): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": True}, - ) - ts = time.time() - envelopes = capture_envelopes() - - metrics.set("my-set", "peter", tags={"magic": "puff"}, timestamp=ts) - metrics.set("my-set", "paul", tags={"magic": "puff"}, timestamp=ts) - metrics.set("my-set", "mary", tags={"magic": "puff"}, timestamp=ts) - sentry_sdk.flush() - - (envelope,) = envelopes - statsd_item, meta_item = envelope.items - - assert statsd_item.headers["type"] == "statsd" - m = parse_metrics(statsd_item.payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "my-set@none" - assert m[0][2] == "s" - assert len(m[0][3]) == 3 - assert sorted(map(int, m[0][3])) == [354582103, 2513273657, 3329318813] - assert m[0][4] == { - "magic": "puff", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert meta_item.headers["type"] == "metric_meta" - assert parse_json(meta_item.payload.get_bytes()) == { - "timestamp": mock.ANY, - "mapping": { - "s:my-set@none": [ - { - "type": "location", - "filename": "tests/test_metrics.py", - "abs_path": __file__, - "function": sys._getframe().f_code.co_name, - "module": __name__, - "lineno": mock.ANY, - "pre_context": mock.ANY, - "context_line": mock.ANY, - "post_context": mock.ANY, - } - ] - }, - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_gauge(sentry_init, capture_envelopes, maybe_monkeypatched_threading): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": False}, - ) - ts = time.time() - envelopes = capture_envelopes() - - metrics.gauge("my-gauge", 10.0, tags={"x": "y"}, timestamp=ts) - metrics.gauge("my-gauge", 20.0, tags={"x": "y"}, timestamp=ts) - metrics.gauge("my-gauge", 30.0, tags={"x": "y"}, timestamp=ts) - sentry_sdk.flush() - - (envelope,) = envelopes - - assert len(envelope.items) == 1 - assert envelope.items[0].headers["type"] == "statsd" - m = parse_metrics(envelope.items[0].payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "my-gauge@none" - assert m[0][2] == "g" - assert len(m[0][3]) == 5 - assert list(map(float, m[0][3])) == [30.0, 10.0, 30.0, 60.0, 3.0] - assert m[0][4] == { - "x": "y", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_multiple(sentry_init, capture_envelopes): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": False}, - ) - ts = time.time() - envelopes = capture_envelopes() - - metrics.gauge("my-gauge", 10.0, tags={"x": "y"}, timestamp=ts) - metrics.gauge("my-gauge", 20.0, tags={"x": "y"}, timestamp=ts) - metrics.gauge("my-gauge", 30.0, tags={"x": "y"}, timestamp=ts) - for _ in range(10): - metrics.increment("counter-1", 1.0, timestamp=ts) - metrics.increment("counter-2", 1.0, timestamp=ts) - - sentry_sdk.flush() - - (envelope,) = envelopes - - assert len(envelope.items) == 1 - assert envelope.items[0].headers["type"] == "statsd" - m = parse_metrics(envelope.items[0].payload.get_bytes()) - - assert len(m) == 3 - - assert m[0][1] == "counter-1@none" - assert m[0][2] == "c" - assert list(map(float, m[0][3])) == [10.0] - assert m[0][4] == { - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert m[1][1] == "counter-2@none" - assert m[1][2] == "c" - assert list(map(float, m[1][3])) == [1.0] - assert m[1][4] == { - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert m[2][1] == "my-gauge@none" - assert m[2][2] == "g" - assert len(m[2][3]) == 5 - assert list(map(float, m[2][3])) == [30.0, 10.0, 30.0, 60.0, 3.0] - assert m[2][4] == { - "x": "y", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_transaction_name( - sentry_init, capture_envelopes, maybe_monkeypatched_threading -): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": False}, - ) - ts = time.time() - envelopes = capture_envelopes() - - sentry_sdk.get_current_scope().set_transaction_name( - "/user/{user_id}", source=TransactionSource.ROUTE - ) - metrics.distribution("dist", 1.0, tags={"a": "b"}, timestamp=ts) - metrics.distribution("dist", 2.0, tags={"a": "b"}, timestamp=ts) - metrics.distribution("dist", 2.0, tags={"a": "b"}, timestamp=ts) - metrics.distribution("dist", 3.0, tags={"a": "b"}, timestamp=ts) - - sentry_sdk.flush() - - (envelope,) = envelopes - - assert len(envelope.items) == 1 - assert envelope.items[0].headers["type"] == "statsd" - m = parse_metrics(envelope.items[0].payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "dist@none" - assert m[0][2] == "d" - assert len(m[0][3]) == 4 - assert sorted(map(float, m[0][3])) == [1.0, 2.0, 2.0, 3.0] - assert m[0][4] == { - "a": "b", - "transaction": "/user/{user_id}", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_metric_summaries( - sentry_init, capture_envelopes, maybe_monkeypatched_threading -): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - enable_tracing=True, - ) - ts = time.time() - envelopes = capture_envelopes() - - with sentry_sdk.start_transaction( - op="stuff", name="/foo", source=TransactionSource.ROUTE - ) as transaction: - metrics.increment("root-counter", timestamp=ts) - with metrics.timing("my-timer-metric", tags={"a": "b"}, timestamp=ts): - for x in range(10): - metrics.distribution("my-dist", float(x), timestamp=ts) - - sentry_sdk.flush() - - (transaction, envelope) = envelopes - - # Metrics Emission - assert envelope.items[0].headers["type"] == "statsd" - m = parse_metrics(envelope.items[0].payload.get_bytes()) - - assert len(m) == 3 - - assert m[0][1] == "my-dist@none" - assert m[0][2] == "d" - assert len(m[0][3]) == 10 - assert sorted(m[0][3]) == list(map(str, map(float, range(10)))) - assert m[0][4] == { - "transaction": "/foo", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert m[1][1] == "my-timer-metric@second" - assert m[1][2] == "d" - assert len(m[1][3]) == 1 - assert m[1][4] == { - "a": "b", - "transaction": "/foo", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - assert m[2][1] == "root-counter@none" - assert m[2][2] == "c" - assert m[2][3] == ["1.0"] - assert m[2][4] == { - "transaction": "/foo", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - } - - # Measurement Attachment - t = transaction.items[0].get_transaction_event() - - assert t["_metrics_summary"] == { - "c:root-counter@none": [ - { - "count": 1, - "min": 1.0, - "max": 1.0, - "sum": 1.0, - "tags": { - "transaction": "/foo", - "release": "fun-release@1.0.0", - "environment": "not-fun-env", - }, - } - ] - } - - assert t["spans"][0]["_metrics_summary"]["d:my-dist@none"] == [ - { - "count": 10, - "min": 0.0, - "max": 9.0, - "sum": 45.0, - "tags": { - "environment": "not-fun-env", - "release": "fun-release@1.0.0", - "transaction": "/foo", - }, - } - ] - - assert t["spans"][0]["tags"] == {"a": "b"} - (timer,) = t["spans"][0]["_metrics_summary"]["d:my-timer-metric@second"] - assert timer["count"] == 1 - assert timer["max"] == timer["min"] == timer["sum"] - assert timer["sum"] > 0 - assert timer["tags"] == { - "a": "b", - "environment": "not-fun-env", - "release": "fun-release@1.0.0", - "transaction": "/foo", - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -@pytest.mark.parametrize( - "metric_name,metric_unit,expected_name", - [ - ("first-metric", "nano-second", "first-metric@nanosecond"), - ("another_metric?", "nano second", "another_metric_@nanosecond"), - ( - "metric", - "nanosecond", - "metric@nanosecond", - ), - ( - "my.amaze.metric I guess", - "nano|\nsecond", - "my.amaze.metric_I_guess@nanosecond", - ), - ("métríc", "nanöseconď", "m_tr_c@nansecon"), - ], -) -def test_metric_name_normalization( - sentry_init, - capture_envelopes, - metric_name, - metric_unit, - expected_name, - maybe_monkeypatched_threading, -): - sentry_init( - _experiments={"enable_metrics": True, "metric_code_locations": False}, - ) - envelopes = capture_envelopes() - - metrics.distribution(metric_name, 1.0, unit=metric_unit) - - sentry_sdk.flush() - - (envelope,) = envelopes - - assert len(envelope.items) == 1 - assert envelope.items[0].headers["type"] == "statsd" - - parsed_metrics = parse_metrics(envelope.items[0].payload.get_bytes()) - assert len(parsed_metrics) == 1 - - name = parsed_metrics[0][1] - assert name == expected_name - - -@minimum_python_37_with_gevent -@pytest.mark.forked -@pytest.mark.parametrize( - "metric_tag,expected_tag", - [ - ({"f-oo|bar": "%$foo/"}, {"f-oobar": "%$foo/"}), - ({"foo$.$.$bar": "blah{}"}, {"foo..bar": "blah{}"}), - ( - {"foö-bar": "snöwmän"}, - {"fo-bar": "snöwmän"}, - ), - ({"route": "GET /foo"}, {"route": "GET /foo"}), - ({"__bar__": "this | or , that"}, {"__bar__": "this \\u{7c} or \\u{2c} that"}), - ({"foo/": "hello!\n\r\t\\"}, {"foo/": "hello!\\n\\r\\t\\\\"}), - ], -) -def test_metric_tag_normalization( - sentry_init, - capture_envelopes, - metric_tag, - expected_tag, - maybe_monkeypatched_threading, -): - sentry_init( - _experiments={"enable_metrics": True, "metric_code_locations": False}, - ) - envelopes = capture_envelopes() - - metrics.distribution("a", 1.0, tags=metric_tag) - - sentry_sdk.flush() - - (envelope,) = envelopes - - assert len(envelope.items) == 1 - assert envelope.items[0].headers["type"] == "statsd" - - parsed_metrics = parse_metrics(envelope.items[0].payload.get_bytes()) - assert len(parsed_metrics) == 1 - - tags = parsed_metrics[0][4] - - expected_tag_key, expected_tag_value = expected_tag.popitem() - assert expected_tag_key in tags - assert tags[expected_tag_key] == expected_tag_value - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_before_emit_metric( - sentry_init, capture_envelopes, maybe_monkeypatched_threading -): - def before_emit(key, value, unit, tags): - if key == "removed-metric" or value == 47 or unit == "unsupported": - return False - - tags["extra"] = "foo" - del tags["release"] - # this better be a noop! - metrics.increment("shitty-recursion") - return True - - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={ - "enable_metrics": True, - "metric_code_locations": False, - "before_emit_metric": before_emit, - }, - ) - envelopes = capture_envelopes() - - metrics.increment("removed-metric", 1.0) - metrics.increment("another-removed-metric", 47) - metrics.increment("yet-another-removed-metric", 1.0, unit="unsupported") - metrics.increment("actual-metric", 1.0) - sentry_sdk.flush() - - (envelope,) = envelopes - - assert len(envelope.items) == 1 - assert envelope.items[0].headers["type"] == "statsd" - m = parse_metrics(envelope.items[0].payload.get_bytes()) - - assert len(m) == 1 - assert m[0][1] == "actual-metric@none" - assert m[0][3] == ["1.0"] - assert m[0][4] == { - "extra": "foo", - "environment": "not-fun-env", - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_aggregator_flush( - sentry_init, capture_envelopes, maybe_monkeypatched_threading -): - sentry_init( - release="fun-release@1.0.0", - environment="not-fun-env", - _experiments={ - "enable_metrics": True, - }, - ) - envelopes = capture_envelopes() - - metrics.increment("a-metric", 1.0) - sentry_sdk.flush() - - assert len(envelopes) == 1 - assert sentry_sdk.get_client().metrics_aggregator.buckets == {} - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_tag_serialization( - sentry_init, capture_envelopes, maybe_monkeypatched_threading -): - sentry_init( - release="fun-release", - environment="not-fun-env", - _experiments={"enable_metrics": True, "metric_code_locations": False}, - ) - envelopes = capture_envelopes() - - metrics.increment( - "counter", - tags={ - "no-value": None, - "an-int": 42, - "a-float": 23.0, - "a-string": "blah", - "more-than-one": [1, "zwei", "3.0", None], - }, - ) - sentry_sdk.flush() - - (envelope,) = envelopes - - assert len(envelope.items) == 1 - assert envelope.items[0].headers["type"] == "statsd" - m = parse_metrics(envelope.items[0].payload.get_bytes()) - - assert len(m) == 1 - assert m[0][4] == { - "an-int": "42", - "a-float": "23.0", - "a-string": "blah", - "more-than-one": ["1", "3.0", "zwei"], - "release": "fun-release", - "environment": "not-fun-env", - } - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_flush_recursion_protection( - sentry_init, capture_envelopes, monkeypatch, maybe_monkeypatched_threading -): - sentry_init( - release="fun-release", - environment="not-fun-env", - _experiments={"enable_metrics": True}, - ) - envelopes = capture_envelopes() - test_client = sentry_sdk.get_client() - - real_capture_envelope = test_client.transport.capture_envelope - - def bad_capture_envelope(*args, **kwargs): - metrics.increment("bad-metric") - return real_capture_envelope(*args, **kwargs) - - monkeypatch.setattr(test_client.transport, "capture_envelope", bad_capture_envelope) - - metrics.increment("counter") - - # flush twice to see the inner metric - sentry_sdk.flush() - sentry_sdk.flush() - - (envelope,) = envelopes - m = parse_metrics(envelope.items[0].payload.get_bytes()) - assert len(m) == 1 - assert m[0][1] == "counter@none" - - -@minimum_python_37_with_gevent -@pytest.mark.forked -def test_flush_recursion_protection_background_flush( - sentry_init, capture_envelopes, monkeypatch, maybe_monkeypatched_threading -): - monkeypatch.setattr(metrics.MetricsAggregator, "FLUSHER_SLEEP_TIME", 0.01) - sentry_init( - release="fun-release", - environment="not-fun-env", - _experiments={"enable_metrics": True}, - ) - envelopes = capture_envelopes() - test_client = sentry_sdk.get_client() - - real_capture_envelope = test_client.transport.capture_envelope - - def bad_capture_envelope(*args, **kwargs): - metrics.increment("bad-metric") - return real_capture_envelope(*args, **kwargs) - - monkeypatch.setattr(test_client.transport, "capture_envelope", bad_capture_envelope) - - metrics.increment("counter") - - # flush via sleep and flag - sentry_sdk.get_client().metrics_aggregator._force_flush = True - time.sleep(0.5) - - (envelope,) = envelopes - m = parse_metrics(envelope.items[0].payload.get_bytes()) - assert len(m) == 1 - assert m[0][1] == "counter@none" - - -@pytest.mark.skipif( - not gevent or sys.version_info >= (3, 7), - reason="Python 3.6 or lower and gevent required", -) -@pytest.mark.forked -def test_disable_metrics_for_old_python_with_gevent( - sentry_init, capture_envelopes, maybe_monkeypatched_threading -): - if maybe_monkeypatched_threading != "greenlet": - pytest.skip("Test specifically for gevent/greenlet") - - sentry_init( - release="fun-release", - environment="not-fun-env", - _experiments={"enable_metrics": True}, - ) - envelopes = capture_envelopes() - - metrics.incr("counter") - - sentry_sdk.flush() - - assert sentry_sdk.get_client().metrics_aggregator is None - assert not envelopes diff --git a/tests/test_transport.py b/tests/test_transport.py index 68669fa24d..804105b010 100644 --- a/tests/test_transport.py +++ b/tests/test_transport.py @@ -590,43 +590,6 @@ def test_complex_limits_without_data_category( assert len(capturing_server.captured) == 0 -@pytest.mark.parametrize("response_code", [200, 429]) -def test_metric_bucket_limits(capturing_server, response_code, make_client): - client = make_client() - capturing_server.respond_with( - code=response_code, - headers={ - "X-Sentry-Rate-Limits": "4711:metric_bucket:organization:quota_exceeded:custom" - }, - ) - - envelope = Envelope() - envelope.add_item(Item(payload=b"{}", type="statsd")) - client.transport.capture_envelope(envelope) - client.flush() - - assert len(capturing_server.captured) == 1 - assert capturing_server.captured[0].path == "/api/132/envelope/" - capturing_server.clear_captured() - - assert set(client.transport._disabled_until) == {"metric_bucket"} - - client.transport.capture_envelope(envelope) - client.capture_event({"type": "transaction"}) - client.flush() - - assert len(capturing_server.captured) == 2 - - envelope = capturing_server.captured[0].envelope - assert envelope.items[0].type == "transaction" - envelope = capturing_server.captured[1].envelope - assert envelope.items[0].type == "client_report" - report = parse_json(envelope.items[0].get_bytes()) - assert report["discarded_events"] == [ - {"category": "metric_bucket", "reason": "ratelimit_backoff", "quantity": 1}, - ] - - @pytest.mark.parametrize("response_code", [200, 429]) def test_log_item_limits(capturing_server, response_code, make_client): client = make_client() @@ -664,80 +627,6 @@ def test_log_item_limits(capturing_server, response_code, make_client): ] -@pytest.mark.parametrize("response_code", [200, 429]) -def test_metric_bucket_limits_with_namespace( - capturing_server, response_code, make_client -): - client = make_client() - capturing_server.respond_with( - code=response_code, - headers={ - "X-Sentry-Rate-Limits": "4711:metric_bucket:organization:quota_exceeded:foo" - }, - ) - - envelope = Envelope() - envelope.add_item(Item(payload=b"{}", type="statsd")) - client.transport.capture_envelope(envelope) - client.flush() - - assert len(capturing_server.captured) == 1 - assert capturing_server.captured[0].path == "/api/132/envelope/" - capturing_server.clear_captured() - - assert set(client.transport._disabled_until) == set([]) - - client.transport.capture_envelope(envelope) - client.capture_event({"type": "transaction"}) - client.flush() - - assert len(capturing_server.captured) == 2 - - envelope = capturing_server.captured[0].envelope - assert envelope.items[0].type == "statsd" - envelope = capturing_server.captured[1].envelope - assert envelope.items[0].type == "transaction" - - -@pytest.mark.parametrize("response_code", [200, 429]) -def test_metric_bucket_limits_with_all_namespaces( - capturing_server, response_code, make_client -): - client = make_client() - capturing_server.respond_with( - code=response_code, - headers={ - "X-Sentry-Rate-Limits": "4711:metric_bucket:organization:quota_exceeded" - }, - ) - - envelope = Envelope() - envelope.add_item(Item(payload=b"{}", type="statsd")) - client.transport.capture_envelope(envelope) - client.flush() - - assert len(capturing_server.captured) == 1 - assert capturing_server.captured[0].path == "/api/132/envelope/" - capturing_server.clear_captured() - - assert set(client.transport._disabled_until) == set(["metric_bucket"]) - - client.transport.capture_envelope(envelope) - client.capture_event({"type": "transaction"}) - client.flush() - - assert len(capturing_server.captured) == 2 - - envelope = capturing_server.captured[0].envelope - assert envelope.items[0].type == "transaction" - envelope = capturing_server.captured[1].envelope - assert envelope.items[0].type == "client_report" - report = parse_json(envelope.items[0].get_bytes()) - assert report["discarded_events"] == [ - {"category": "metric_bucket", "reason": "ratelimit_backoff", "quantity": 1}, - ] - - def test_hub_cls_backwards_compat(): class TestCustomHubClass(Hub): pass From a04974744e54c333733648152b530d19c457205f Mon Sep 17 00:00:00 2001 From: Ivana Kellyer Date: Thu, 9 Oct 2025 13:34:35 +0200 Subject: [PATCH 6/9] ref: Remove "experimental" from log func name (#4901) ### Description Logs are not experimental anymore, but one of the internal log-related functions still had "experimental" in the name. #### Issues #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --- sentry_sdk/client.py | 4 ++-- sentry_sdk/integrations/logging.py | 2 +- sentry_sdk/integrations/loguru.py | 2 +- sentry_sdk/logger.py | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/sentry_sdk/client.py b/sentry_sdk/client.py index 59a5013783..9401c3f0b0 100644 --- a/sentry_sdk/client.py +++ b/sentry_sdk/client.py @@ -213,7 +213,7 @@ def capture_event(self, *args, **kwargs): # type: (*Any, **Any) -> Optional[str] return None - def _capture_experimental_log(self, log): + def _capture_log(self, log): # type: (Log) -> None pass @@ -877,7 +877,7 @@ def capture_event( return return_value - def _capture_experimental_log(self, log): + def _capture_log(self, log): # type: (Optional[Log]) -> None if not has_logs_enabled(self.options) or log is None: return diff --git a/sentry_sdk/integrations/logging.py b/sentry_sdk/integrations/logging.py index bfb30fc67b..7e16943b28 100644 --- a/sentry_sdk/integrations/logging.py +++ b/sentry_sdk/integrations/logging.py @@ -409,7 +409,7 @@ def _capture_log_from_record(self, client, record): attrs["logger.name"] = record.name # noinspection PyProtectedMember - client._capture_experimental_log( + client._capture_log( { "severity_text": otel_severity_text, "severity_number": otel_severity_number, diff --git a/sentry_sdk/integrations/loguru.py b/sentry_sdk/integrations/loguru.py index b910b9a407..2c0279d0ce 100644 --- a/sentry_sdk/integrations/loguru.py +++ b/sentry_sdk/integrations/loguru.py @@ -193,7 +193,7 @@ def loguru_sentry_logs_handler(message): if record.get("name"): attrs["logger.name"] = record["name"] - client._capture_experimental_log( + client._capture_log( { "severity_text": otel_severity_text, "severity_number": otel_severity_number, diff --git a/sentry_sdk/logger.py b/sentry_sdk/logger.py index bc98f35155..0ea7218e01 100644 --- a/sentry_sdk/logger.py +++ b/sentry_sdk/logger.py @@ -46,7 +46,7 @@ def _capture_log(severity_text, severity_number, template, **kwargs): } # noinspection PyProtectedMember - client._capture_experimental_log( + client._capture_log( { "severity_text": severity_text, "severity_number": severity_number, From 1f8c008e9368874d6cea289283da61e236062b34 Mon Sep 17 00:00:00 2001 From: Kev <6111995+k-fish@users.noreply.github.com> Date: Thu, 9 Oct 2025 09:58:52 -0400 Subject: [PATCH 7/9] feat(metrics): Add trace metrics behind an experiments flag (#4898) ### Summary Similar to https://github.com/getsentry/sentry-javascript/pull/17883, this allows the py sdk to send in new trace metric protocol items, although this code is experimental since the schema may still change. Most of this code has been copied from logs (eg. log batcher -> metrics batcher) in order to dogfood, once we're more sure of our approach we can refactor. Closes LOGS-367 --------- Co-authored-by: Ivana Kellyer --- sentry_sdk/_metrics.py | 81 +++++++++++++ sentry_sdk/_metrics_batcher.py | 156 +++++++++++++++++++++++++ sentry_sdk/_types.py | 27 +++++ sentry_sdk/client.py | 80 ++++++++++++- sentry_sdk/consts.py | 3 + sentry_sdk/envelope.py | 2 + sentry_sdk/types.py | 3 + sentry_sdk/utils.py | 18 ++- tests/test_metrics.py | 208 +++++++++++++++++++++++++++++++++ 9 files changed, 576 insertions(+), 2 deletions(-) create mode 100644 sentry_sdk/_metrics.py create mode 100644 sentry_sdk/_metrics_batcher.py create mode 100644 tests/test_metrics.py diff --git a/sentry_sdk/_metrics.py b/sentry_sdk/_metrics.py new file mode 100644 index 0000000000..03bde137bd --- /dev/null +++ b/sentry_sdk/_metrics.py @@ -0,0 +1,81 @@ +""" +NOTE: This file contains experimental code that may be changed or removed at any +time without prior notice. +""" + +import time +from typing import Any, Optional, TYPE_CHECKING, Union + +import sentry_sdk +from sentry_sdk.utils import safe_repr + +if TYPE_CHECKING: + from sentry_sdk._types import Metric, MetricType + + +def _capture_metric( + name, # type: str + metric_type, # type: MetricType + value, # type: float + unit=None, # type: Optional[str] + attributes=None, # type: Optional[dict[str, Any]] +): + # type: (...) -> None + client = sentry_sdk.get_client() + + attrs = {} # type: dict[str, Union[str, bool, float, int]] + if attributes: + for k, v in attributes.items(): + attrs[k] = ( + v + if ( + isinstance(v, str) + or isinstance(v, int) + or isinstance(v, bool) + or isinstance(v, float) + ) + else safe_repr(v) + ) + + metric = { + "timestamp": time.time(), + "trace_id": None, + "span_id": None, + "name": name, + "type": metric_type, + "value": float(value), + "unit": unit, + "attributes": attrs, + } # type: Metric + + client._capture_metric(metric) + + +def count( + name, # type: str + value, # type: float + unit=None, # type: Optional[str] + attributes=None, # type: Optional[dict[str, Any]] +): + # type: (...) -> None + _capture_metric(name, "counter", value, unit, attributes) + + +def gauge( + name, # type: str + value, # type: float + unit=None, # type: Optional[str] + attributes=None, # type: Optional[dict[str, Any]] +): + # type: (...) -> None + _capture_metric(name, "gauge", value, unit, attributes) + + +def distribution( + name, # type: str + value, # type: float + unit=None, # type: Optional[str] + attributes=None, # type: Optional[dict[str, Any]] +): + # type: (...) -> None + _capture_metric(name, "distribution", value, unit, attributes) diff --git a/sentry_sdk/_metrics_batcher.py b/sentry_sdk/_metrics_batcher.py new file mode 100644 index 0000000000..fd9a5d732b --- /dev/null +++ b/sentry_sdk/_metrics_batcher.py @@ -0,0 +1,156 @@ +import os +import random +import threading +from datetime import datetime, timezone +from typing import Optional, List, Callable, TYPE_CHECKING, Any, Union + +from sentry_sdk.utils import format_timestamp, safe_repr +from sentry_sdk.envelope import Envelope, Item, PayloadRef + +if TYPE_CHECKING: + from sentry_sdk._types import Metric + + +class MetricsBatcher: + MAX_METRICS_BEFORE_FLUSH = 100 + FLUSH_WAIT_TIME = 5.0 + + def __init__( + self, + capture_func, # type: Callable[[Envelope], None] + ): + # type: (...) -> None + self._metric_buffer = [] # type: List[Metric] + self._capture_func = capture_func + self._running = True + self._lock = threading.Lock() + + self._flush_event = threading.Event() # type: threading.Event + + self._flusher = None # type: Optional[threading.Thread] + self._flusher_pid = None # type: Optional[int] + + def _ensure_thread(self): + # type: (...) -> bool + if not self._running: + return False + + pid = os.getpid() + if self._flusher_pid == pid: + return True + + with self._lock: + if self._flusher_pid == pid: + return True + + self._flusher_pid = pid + + self._flusher = threading.Thread(target=self._flush_loop) + self._flusher.daemon = True + + try: + self._flusher.start() + except RuntimeError: + self._running = False + return False + + return True + + def _flush_loop(self): + # type: (...) -> None + while self._running: + self._flush_event.wait(self.FLUSH_WAIT_TIME + random.random()) + self._flush_event.clear() + self._flush() + + def add( + self, + metric, # type: Metric + ): + # type: (...) -> None + if not self._ensure_thread() or self._flusher is None: + return None + + with self._lock: + self._metric_buffer.append(metric) + if len(self._metric_buffer) >= self.MAX_METRICS_BEFORE_FLUSH: + self._flush_event.set() + + def kill(self): + # type: (...) -> None + if self._flusher is None: + return + + self._running = False + self._flush_event.set() + self._flusher = None + + def flush(self): + # type: (...) -> None + self._flush() + + @staticmethod + def _metric_to_transport_format(metric): + # type: (Metric) -> Any + def format_attribute(val): + # type: (Union[int, float, str, bool]) -> Any + if isinstance(val, bool): + return {"value": val, "type": "boolean"} + if isinstance(val, int): + return {"value": val, "type": "integer"} + if isinstance(val, float): + return {"value": val, "type": "double"} + if isinstance(val, str): + return {"value": val, "type": "string"} + return {"value": safe_repr(val), "type": "string"} + + res = { + "timestamp": metric["timestamp"], + "trace_id": metric["trace_id"], + "name": metric["name"], + "type": metric["type"], + "value": metric["value"], + "attributes": { + k: format_attribute(v) for (k, v) in metric["attributes"].items() + }, + } + + if metric.get("span_id") is not None: + res["span_id"] = metric["span_id"] + + if metric.get("unit") is not None: + res["unit"] = metric["unit"] + + return res + + def _flush(self): + # type: (...) -> Optional[Envelope] + + envelope = Envelope( + headers={"sent_at": format_timestamp(datetime.now(timezone.utc))} + ) + with self._lock: + if len(self._metric_buffer) == 0: + return None + + envelope.add_item( + Item( + type="trace_metric", + content_type="application/vnd.sentry.items.trace-metric+json", + headers={ + "item_count": len(self._metric_buffer), + }, + payload=PayloadRef( + json={ + "items": [ + self._metric_to_transport_format(metric) + for metric in self._metric_buffer + ] + } + ), + ) + ) + self._metric_buffer.clear() + + self._capture_func(envelope) + return envelope diff --git a/sentry_sdk/_types.py b/sentry_sdk/_types.py index d057f215e4..66ed7df4f7 100644 --- a/sentry_sdk/_types.py +++ b/sentry_sdk/_types.py @@ -234,6 +234,32 @@ class SDKInfo(TypedDict): }, ) + MetricType = Literal["counter", "gauge", "distribution"] + + MetricAttributeValue = TypedDict( + "MetricAttributeValue", + { + "value": Union[str, bool, float, int], + "type": Literal["string", "boolean", "double", "integer"], + }, + ) + + Metric = TypedDict( + "Metric", + { + "timestamp": float, + "trace_id": Optional[str], + "span_id": Optional[str], + "name": str, + "type": MetricType, + "value": float, + "unit": Optional[str], + "attributes": dict[str, str | bool | float | int], + }, + ) + + MetricProcessor = Callable[[Metric, Hint], Optional[Metric]] + # TODO: Make a proper type definition for this (PRs welcome!) Breadcrumb = Dict[str, Any] @@ -268,6 +294,7 @@ class SDKInfo(TypedDict): "monitor", "span", "log_item", + "trace_metric", ] SessionStatus = Literal["ok", "exited", "crashed", "abnormal"] diff --git a/sentry_sdk/client.py b/sentry_sdk/client.py index 9401c3f0b0..d17f922642 100644 --- a/sentry_sdk/client.py +++ b/sentry_sdk/client.py @@ -24,7 +24,9 @@ is_gevent, logger, get_before_send_log, + get_before_send_metric, has_logs_enabled, + has_metrics_enabled, ) from sentry_sdk.serializer import serialize from sentry_sdk.tracing import trace @@ -59,13 +61,14 @@ from typing import Union from typing import TypeVar - from sentry_sdk._types import Event, Hint, SDKInfo, Log + from sentry_sdk._types import Event, Hint, SDKInfo, Log, Metric from sentry_sdk.integrations import Integration from sentry_sdk.scope import Scope from sentry_sdk.session import Session from sentry_sdk.spotlight import SpotlightClient from sentry_sdk.transport import Transport from sentry_sdk._log_batcher import LogBatcher + from sentry_sdk._metrics_batcher import MetricsBatcher I = TypeVar("I", bound=Integration) # noqa: E741 @@ -182,6 +185,7 @@ def __init__(self, options=None): self.transport = None # type: Optional[Transport] self.monitor = None # type: Optional[Monitor] self.log_batcher = None # type: Optional[LogBatcher] + self.metrics_batcher = None # type: Optional[MetricsBatcher] def __getstate__(self, *args, **kwargs): # type: (*Any, **Any) -> Any @@ -217,6 +221,10 @@ def _capture_log(self, log): # type: (Log) -> None pass + def _capture_metric(self, metric): + # type: (Metric) -> None + pass + def capture_session(self, *args, **kwargs): # type: (*Any, **Any) -> None return None @@ -366,6 +374,13 @@ def _capture_envelope(envelope): self.log_batcher = LogBatcher(capture_func=_capture_envelope) + self.metrics_batcher = None + + if has_metrics_enabled(self.options): + from sentry_sdk._metrics_batcher import MetricsBatcher + + self.metrics_batcher = MetricsBatcher(capture_func=_capture_envelope) + max_request_body_size = ("always", "never", "small", "medium") if self.options["max_request_body_size"] not in max_request_body_size: raise ValueError( @@ -944,6 +959,65 @@ def _capture_log(self, log): if self.log_batcher: self.log_batcher.add(log) + def _capture_metric(self, metric): + # type: (Optional[Metric]) -> None + if not has_metrics_enabled(self.options) or metric is None: + return + + isolation_scope = sentry_sdk.get_isolation_scope() + + metric["attributes"]["sentry.sdk.name"] = SDK_INFO["name"] + metric["attributes"]["sentry.sdk.version"] = SDK_INFO["version"] + + environment = self.options.get("environment") + if environment is not None and "sentry.environment" not in metric["attributes"]: + metric["attributes"]["sentry.environment"] = environment + + release = self.options.get("release") + if release is not None and "sentry.release" not in metric["attributes"]: + metric["attributes"]["sentry.release"] = release + + span = sentry_sdk.get_current_span() + metric["trace_id"] = "00000000-0000-0000-0000-000000000000" + + if span: + metric["trace_id"] = span.trace_id + metric["span_id"] = span.span_id + else: + propagation_context = isolation_scope.get_active_propagation_context() + if propagation_context and propagation_context.trace_id: + metric["trace_id"] = propagation_context.trace_id + + if isolation_scope._user is not None: + for metric_attribute, user_attribute in ( + ("user.id", "id"), + ("user.name", "username"), + ("user.email", "email"), + ): + if ( + user_attribute in isolation_scope._user + and metric_attribute not in metric["attributes"] + ): + metric["attributes"][metric_attribute] = isolation_scope._user[ + user_attribute + ] + + debug = self.options.get("debug", False) + if debug: + logger.debug( + f"[Sentry Metrics] [{metric.get('type')}] {metric.get('name')}: {metric.get('value')}" + ) + + before_send_metric = get_before_send_metric(self.options) + if before_send_metric is not None: + metric = before_send_metric(metric, {}) + + if metric is None: + return + + if self.metrics_batcher: + self.metrics_batcher.add(metric) + def capture_session( self, session, # type: Session @@ -998,6 +1072,8 @@ def close( self.session_flusher.kill() if self.log_batcher is not None: self.log_batcher.kill() + if self.metrics_batcher is not None: + self.metrics_batcher.kill() if self.monitor: self.monitor.kill() self.transport.kill() @@ -1022,6 +1098,8 @@ def flush( self.session_flusher.flush() if self.log_batcher is not None: self.log_batcher.flush() + if self.metrics_batcher is not None: + self.metrics_batcher.flush() self.transport.flush(timeout=timeout, callback=callback) def __enter__(self): diff --git a/sentry_sdk/consts.py b/sentry_sdk/consts.py index 0f71a0d460..12654cc76d 100644 --- a/sentry_sdk/consts.py +++ b/sentry_sdk/consts.py @@ -52,6 +52,7 @@ class CompressionAlgo(Enum): Hint, Log, MeasurementUnit, + Metric, ProfilerMode, TracesSampler, TransactionProcessor, @@ -77,6 +78,8 @@ class CompressionAlgo(Enum): "transport_http2": Optional[bool], "enable_logs": Optional[bool], "before_send_log": Optional[Callable[[Log, Hint], Optional[Log]]], + "enable_metrics": Optional[bool], + "before_send_metric": Optional[Callable[[Metric, Hint], Optional[Metric]]], }, total=False, ) diff --git a/sentry_sdk/envelope.py b/sentry_sdk/envelope.py index b26c458d41..56bb5fde73 100644 --- a/sentry_sdk/envelope.py +++ b/sentry_sdk/envelope.py @@ -285,6 +285,8 @@ def data_category(self): return "error" elif ty == "log": return "log_item" + elif ty == "trace_metric": + return "trace_metric" elif ty == "client_report": return "internal" elif ty == "profile": diff --git a/sentry_sdk/types.py b/sentry_sdk/types.py index 1a65247584..8b28166462 100644 --- a/sentry_sdk/types.py +++ b/sentry_sdk/types.py @@ -21,6 +21,7 @@ Log, MonitorConfig, SamplingContext, + Metric, ) else: from typing import Any @@ -35,6 +36,7 @@ Log = Any MonitorConfig = Any SamplingContext = Any + Metric = Any __all__ = ( @@ -46,4 +48,5 @@ "Log", "MonitorConfig", "SamplingContext", + "Metric", ) diff --git a/sentry_sdk/utils.py b/sentry_sdk/utils.py index 2083fd296c..cd825b29e2 100644 --- a/sentry_sdk/utils.py +++ b/sentry_sdk/utils.py @@ -59,7 +59,7 @@ from gevent.hub import Hub - from sentry_sdk._types import Event, ExcInfo, Log, Hint + from sentry_sdk._types import Event, ExcInfo, Log, Hint, Metric P = ParamSpec("P") R = TypeVar("R") @@ -2013,3 +2013,19 @@ def get_before_send_log(options): return options.get("before_send_log") or options["_experiments"].get( "before_send_log" ) + + +def has_metrics_enabled(options): + # type: (Optional[dict[str, Any]]) -> bool + if options is None: + return False + + return bool(options["_experiments"].get("enable_metrics", False)) + + +def get_before_send_metric(options): + # type: (Optional[dict[str, Any]]) -> Optional[Callable[[Metric, Hint], Optional[Metric]]] + if options is None: + return None + + return options["_experiments"].get("before_send_metric") diff --git a/tests/test_metrics.py b/tests/test_metrics.py new file mode 100644 index 0000000000..5e774227fd --- /dev/null +++ b/tests/test_metrics.py @@ -0,0 +1,208 @@ +import json +import sys +from typing import List, Any, Mapping +import pytest + +import sentry_sdk +from sentry_sdk import _metrics +from sentry_sdk import get_client +from sentry_sdk.envelope import Envelope +from sentry_sdk.types import Metric + + +def envelopes_to_metrics(envelopes): + # type: (List[Envelope]) -> List[Metric] + res = [] # type: List[Metric] + for envelope in envelopes: + for item in envelope.items: + if item.type == "trace_metric": + for metric_json in item.payload.json["items"]: + metric = { + "timestamp": metric_json["timestamp"], + "trace_id": metric_json["trace_id"], + "span_id": metric_json.get("span_id"), + "name": metric_json["name"], + "type": metric_json["type"], + "value": metric_json["value"], + "unit": metric_json.get("unit"), + "attributes": { + k: v["value"] + for (k, v) in metric_json["attributes"].items() + }, + } # type: Metric + res.append(metric) + return res + + +def test_metrics_disabled_by_default(sentry_init, capture_envelopes): + sentry_init() + + envelopes = capture_envelopes() + + _metrics.count("test.counter", 1) + _metrics.gauge("test.gauge", 42) + _metrics.distribution("test.distribution", 200) + + assert len(envelopes) == 0 + + +def test_metrics_basics(sentry_init, capture_envelopes): + sentry_init(_experiments={"enable_metrics": True}) + envelopes = capture_envelopes() + + _metrics.count("test.counter", 1) + _metrics.gauge("test.gauge", 42, unit="millisecond") + _metrics.distribution("test.distribution", 200, unit="second") + + get_client().flush() + metrics = envelopes_to_metrics(envelopes) + + assert len(metrics) == 3 + + assert metrics[0]["name"] == "test.counter" + assert metrics[0]["type"] == "counter" + assert metrics[0]["value"] == 1.0 + assert metrics[0]["unit"] is None + assert "sentry.sdk.name" in metrics[0]["attributes"] + assert "sentry.sdk.version" in metrics[0]["attributes"] + + assert metrics[1]["name"] == "test.gauge" + assert metrics[1]["type"] == "gauge" + assert metrics[1]["value"] == 42.0 + assert metrics[1]["unit"] == "millisecond" + + assert metrics[2]["name"] == "test.distribution" + assert metrics[2]["type"] == "distribution" + assert metrics[2]["value"] == 200.0 + assert metrics[2]["unit"] == "second" + + +def test_metrics_experimental_option(sentry_init, capture_envelopes): + sentry_init(_experiments={"enable_metrics": True}) + envelopes = capture_envelopes() + + _metrics.count("test.counter", 5) + + get_client().flush() + + metrics = envelopes_to_metrics(envelopes) + assert len(metrics) == 1 + + assert metrics[0]["name"] == "test.counter" + assert metrics[0]["type"] == "counter" + assert metrics[0]["value"] == 5.0 + + +def test_metrics_with_attributes(sentry_init, capture_envelopes): + sentry_init( + _experiments={"enable_metrics": True}, release="1.0.0", environment="test" + ) + envelopes = capture_envelopes() + + _metrics.count( + "test.counter", 1, attributes={"endpoint": "/api/test", "status": "success"} + ) + + get_client().flush() + + metrics = envelopes_to_metrics(envelopes) + assert len(metrics) == 1 + + assert metrics[0]["attributes"]["endpoint"] == "/api/test" + assert metrics[0]["attributes"]["status"] == "success" + assert metrics[0]["attributes"]["sentry.release"] == "1.0.0" + assert metrics[0]["attributes"]["sentry.environment"] == "test" + + +def test_metrics_with_user(sentry_init, capture_envelopes): + sentry_init(_experiments={"enable_metrics": True}) + envelopes = capture_envelopes() + + sentry_sdk.set_user( + {"id": "user-123", "email": "test@example.com", "username": "testuser"} + ) + _metrics.count("test.user.counter", 1) + + get_client().flush() + + metrics = envelopes_to_metrics(envelopes) + assert len(metrics) == 1 + + assert metrics[0]["attributes"]["user.id"] == "user-123" + assert metrics[0]["attributes"]["user.email"] == "test@example.com" + assert metrics[0]["attributes"]["user.name"] == "testuser" + + +def test_metrics_with_span(sentry_init, capture_envelopes): + sentry_init(_experiments={"enable_metrics": True}, traces_sample_rate=1.0) + envelopes = capture_envelopes() + + with sentry_sdk.start_transaction(op="test", name="test-span"): + _metrics.count("test.span.counter", 1) + + get_client().flush() + + metrics = envelopes_to_metrics(envelopes) + assert len(metrics) == 1 + + assert metrics[0]["trace_id"] is not None + assert metrics[0]["trace_id"] != "00000000-0000-0000-0000-000000000000" + assert metrics[0]["span_id"] is not None + + +def test_metrics_tracing_without_performance(sentry_init, capture_envelopes): + sentry_init(_experiments={"enable_metrics": True}) + envelopes = capture_envelopes() + + _metrics.count("test.span.counter", 1) + + get_client().flush() + + metrics = envelopes_to_metrics(envelopes) + assert len(metrics) == 1 + + assert metrics[0]["trace_id"] is not None + assert metrics[0]["trace_id"] != "00000000-0000-0000-0000-000000000000" + assert metrics[0]["span_id"] is None + + +def test_metrics_before_send(sentry_init, capture_envelopes): + before_metric_called = False + + def _before_metric(record, hint): + nonlocal before_metric_called + + assert set(record.keys()) == { + "timestamp", + "trace_id", + "span_id", + "name", + "type", + "value", + "unit", + "attributes", + } + + if record["name"] == "test.skip": + return None + + before_metric_called = True + return record + + sentry_init( + _experiments={ + "enable_metrics": True, + "before_send_metric": _before_metric, + }, + ) + envelopes = capture_envelopes() + + _metrics.count("test.skip", 1) + _metrics.count("test.keep", 1) + + get_client().flush() + + metrics = envelopes_to_metrics(envelopes) + assert len(metrics) == 1 + assert metrics[0]["name"] == "test.keep" + assert before_metric_called From 272af1b1380db38fbf70087e561555c558bf8308 Mon Sep 17 00:00:00 2001 From: getsentry-bot Date: Thu, 9 Oct 2025 14:00:37 +0000 Subject: [PATCH 8/9] release: 2.41.0 --- CHANGELOG.md | 12 ++++++++++++ docs/conf.py | 2 +- sentry_sdk/consts.py | 2 +- setup.py | 2 +- 4 files changed, 15 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0bcd623611..63efefd543 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,17 @@ # Changelog +## 2.41.0 + +### Various fixes & improvements + +- feat(metrics): Add trace metrics behind an experiments flag (#4898) by @k-fish +- ref: Remove "experimental" from log func name (#4901) by @sentrivana +- chore: Remove old metrics code (#4899) by @sentrivana +- ci: Bump Python version for linting (#4897) by @sentrivana +- feat: Add concurrent.futures patch to threading integration (#4770) by @alexander-alderman-webb +- fix(ai): add mapping for gen_ai message roles (#4884) by @shellmayr +- ci: Remove toxgen check (#4892) by @sentrivana + ## 2.40.0 ### Various fixes & improvements diff --git a/docs/conf.py b/docs/conf.py index 2f630c382b..b3522a913e 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -31,7 +31,7 @@ copyright = "2019-{}, Sentry Team and Contributors".format(datetime.now().year) author = "Sentry Team and Contributors" -release = "2.40.0" +release = "2.41.0" version = ".".join(release.split(".")[:2]) # The short X.Y version. diff --git a/sentry_sdk/consts.py b/sentry_sdk/consts.py index 12654cc76d..2158276f9b 100644 --- a/sentry_sdk/consts.py +++ b/sentry_sdk/consts.py @@ -1339,4 +1339,4 @@ def _get_default_options(): del _get_default_options -VERSION = "2.40.0" +VERSION = "2.41.0" diff --git a/setup.py b/setup.py index fbb8694e5e..274e343be7 100644 --- a/setup.py +++ b/setup.py @@ -21,7 +21,7 @@ def get_file_text(file_name): setup( name="sentry-sdk", - version="2.40.0", + version="2.41.0", author="Sentry Team and Contributors", author_email="hello@sentry.io", url="https://github.com/getsentry/sentry-python", From 685287d36da8f9423d2cfb03dc7c13bb7d6c57dd Mon Sep 17 00:00:00 2001 From: Ivana Kellyer Date: Thu, 9 Oct 2025 16:05:08 +0200 Subject: [PATCH 9/9] Update CHANGELOG.md --- CHANGELOG.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 63efefd543..8f59545d6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,13 +4,15 @@ ### Various fixes & improvements -- feat(metrics): Add trace metrics behind an experiments flag (#4898) by @k-fish -- ref: Remove "experimental" from log func name (#4901) by @sentrivana +- feat: Add `concurrent.futures` patch to threading integration (#4770) by @alexander-alderman-webb + + The SDK now makes sure to automatically preserve span relationships when using `ThreadPoolExecutor`. - chore: Remove old metrics code (#4899) by @sentrivana -- ci: Bump Python version for linting (#4897) by @sentrivana -- feat: Add concurrent.futures patch to threading integration (#4770) by @alexander-alderman-webb -- fix(ai): add mapping for gen_ai message roles (#4884) by @shellmayr -- ci: Remove toxgen check (#4892) by @sentrivana + + Removed all code related to the deprecated experimental metrics feature (`sentry_sdk.metrics`). +- ref: Remove "experimental" from log function name (#4901) by @sentrivana +- fix(ai): Add mapping for gen_ai message roles (#4884) by @shellmayr +- feat(metrics): Add trace metrics behind an experiments flag (#4898) by @k-fish ## 2.40.0