Skip to content

[BUG] Misleading metric type of LightevalTaskConfig #710

@wangtsuh

Description

@wangtsuh

Describe the bug

In the documentation and function LightevalTaskConfig, the metric argument has denoted list[str] as the type annotation. However, in the actual code, we can see that the expected type should be ListLike[Metric | Metrics]. The incorrect type annotation is misleading and causes users hard to know what list of string should be passed.

Version info

0.9.2

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions