[ENH] V1 → V2 API Migration - evaluations

[EmanAbdelhaleem]

Fixes #1623
Depends on #1576
Related to #1575

Details

This PR implements Evaluations resource, and refactor its existing functions

[fkiraly]

looks like circular imports

[EmanAbdelhaleem]

@geetu040 We have circular imports here cuz of importing OpenMLEvaluation

Error message:

  File "D:\Work\ESoC\Developing\openML\openml-python\scripts\reproduce_issues.py", line 1, in <module>
    import openml
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\__init__.py", line 21, in <module>
    from . import (
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\evaluations\__init__.py", line 3, in <module>
    from .evaluation import OpenMLEvaluation
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\evaluations\evaluation.py", line 7, in <module>
    import openml.runs
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\runs\__init__.py", line 3, in <module>
    from .functions import (
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\runs\functions.py", line 32, in <module>
    from openml.tasks import (
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\tasks\__init__.py", line 3, in <module>
    from .functions import (
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\tasks\functions.py", line 15, in <module>
    from openml._api import api_context
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\_api\__init__.py", line 1, in <module>
    from openml._api.runtime.core import APIContext
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\_api\runtime\core.py", line 10, in <module>
    from openml._api.resources import (
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\_api\resources\__init__.py", line 2, in <module>
    from openml._api.resources.evaluations import EvaluationsV1, EvaluationsV2
  File "D:\Work\ESoC\Developing\openML\openml-python\openml\_api\resources\evaluations.py", line 8, in <module>        
    from openml.evaluations.evaluation import OpenMLEvaluation
ImportError: cannot import name 'OpenMLEvaluation' from partially initialized module 'openml.evaluations.evaluation' (most likely due to a circular import) (D:\Work\ESoC\Developing\openML\openml-python\openml\evaluations\evaluation.py)```

[geetu040]

Please update the listing functionality with the suggested approach in #1575 (comment). And also sync this PR with my base PR.

[geetu040]

@geetu040 We have circular imports here cuz of importing OpenMLEvaluation

I don't really have an explanation for why this could be happening, can you try to sync with my branch and see if it persists and we can have a look, otherwise I am going to refactor the config part so it may automatically fix this.

[EmanAbdelhaleem]

Please update the listing functionality with the suggested approach in #1575 (comment). And also sync this PR with my base PR.

Updated, please check

[geetu040]

Looks good overall, should be fine with few changes as suggested. And some comments are open for discussion.

[geetu040]

I suspect you didn't completely remove this and replace with api_context.backend.evaluations.list in partial because of the circular import error? As discussed in the standup, I think the api_context.backend.evaluations.list should return OpenMLEvaluation object, so best comply with both v1/v2 interface.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 62.87193% with 287 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.90%. Comparing base (d421b9e) to head (54bbb05).

Files with missing lines	Patch %	Lines
openml/_api/clients/http.py	39.47%	115 Missing ⚠️
openml/_api/resources/base/versions.py	24.71%	67 Missing ⚠️
openml/_api/setup/backend.py	65.16%	31 Missing ⚠️
openml/_api/resources/base/fallback.py	26.31%	28 Missing ⚠️
openml/testing.py	51.16%	21 Missing ⚠️
openml/_api/setup/_utils.py	56.00%	11 Missing ⚠️
openml/_api/setup/builder.py	81.08%	7 Missing ⚠️
openml/_api/resources/base/base.py	78.57%	3 Missing ⚠️
openml/_api/resources/evaluation.py	95.16%	3 Missing ⚠️
openml/config.py	93.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1606      +/-   ##
==========================================
+ Coverage   52.04%   53.90%   +1.86%     
==========================================
  Files          36       63      +27     
  Lines        4333     5059     +726     
==========================================
+ Hits         2255     2727     +472     
- Misses       2078     2332     +254     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[EmanAbdelhaleem]

@geetu040 Ready for review

[EmanAbdelhaleem]

@geetu040 I think you need to look into this:
https://github.com/openml/openml-python/actions/runs/21646917069/job/62401178908?pr=1606#:~:text=openml/_api/clients/http.py%3A310,is%20not%20associated%20with%20a%20value

[geetu040]

@geetu040 I think you need to look into this:

Thanks for pointing this out, seems like a fix for the core PR