Skip to content

Conversation

@yakov-g
Copy link
Collaborator

@yakov-g yakov-g commented Sep 12, 2025

This repository is public. Do not put here any private DataRobot or customer's data: code, datasets, model artifacts, .etc.

Summary

Rationale


Note

Standardizes Gunicorn as the DRUM server across drop-in environments, bumps datarobot-drum to 1.17.10, updates env version IDs/tags, and adjusts tests accordingly.

  • Runtime/Environments:
    • Set MLOPS_RUNTIME_PARAM_DRUM_SERVER_TYPE to "gunicorn" in Dockerfiles for java_codegen, python311, python3_keras, python3_onnx, python3_pytorch, python3_sklearn, python3_xgboost, and r_lang.
    • Update env_info.json environmentVersionId and corresponding tags for all affected environments.
  • Dependencies:
    • Bump datarobot-drum to 1.17.10 across requirements (including python311_genai_agents); regenerate requirements.txt for Python 3.11 with updated pip-compile metadata and include gunicorn where applicable.
  • Tests:
    • Adjust tests/functional/test_drum_server_failures.py parametrization to avoid with_error_server/production for DOCKER_PYTHON_SKLEARN and add notes reflecting Gunicorn usage.

Written by Cursor Bugbot for commit 5383bbf. This will update automatically on new commits. Configure here.

@devexp-slackbot devexp-slackbot bot changed the title [YOLO] use gunicorn in sklearn env [RAPTOR-14627] use gunicorn in sklearn env Sep 12, 2025
@yakov-g yakov-g force-pushed the yakov/python-sklearn-gunicorn branch 3 times, most recently from 1f8d755 to 8117a45 Compare September 15, 2025 19:18
@yakov-g yakov-g force-pushed the yakov/python-sklearn-gunicorn branch from a099f48 to af8d305 Compare December 17, 2025 00:09
ENV WITH_ERROR_SERVER=0 \
PYTHONUNBUFFERED=1

# ENV MLOPS_RUNTIME_PARAM_DRUM_CLIENT_REQUEST_TIMEOUT='{"type": "numeric", "payload": 660}'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Commented-out configuration left in Dockerfile

A commented-out ENV MLOPS_RUNTIME_PARAM_DRUM_CLIENT_REQUEST_TIMEOUT line was added to the Dockerfile. This appears to be leftover testing or debugging code that was not removed before committing. Commented-out configuration in production Dockerfiles can confuse maintainers and suggests incomplete cleanup.

Fix in Cursor Fix in Web

@yakov-g yakov-g force-pushed the yakov/python-sklearn-gunicorn branch 5 times, most recently from d60d02e to c33186b Compare December 18, 2025 00:00
loglevel = temp_loglevel

bind = os.environ.get("ADDRESS", "0.0.0.0:8080")
bind = os.environ["ADDRESS"]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Removed ADDRESS default causes KeyError without fallback

The change from os.environ.get("ADDRESS", "0.0.0.0:8080") to os.environ["ADDRESS"] removes the fallback default. While the Dockerfiles set ENV ADDRESS=0.0.0.0:8080, running gunicorn outside of Docker without the --address CLI option and without the ADDRESS environment variable being set will cause a KeyError. The setup_options() function only sets ADDRESS if options.address is truthy, providing no guarantee the variable exists.

Fix in Cursor Fix in Web

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: setup_options called twice in non-gunicorn execution path

The run_drum_server() function calls setup_options() to determine if gunicorn should be used. When the condition is false (non-gunicorn path), it calls main(), which internally calls setup_options() again. This causes argument parsing, environment variable setup via RuntimeParametersLoader, lazy loading setup, and option verification to all run twice. The argcomplete missing warning would also print twice if the package is not installed. The previous implementation checked sys.argv[1] == "server" directly without requiring a full parse, avoiding this redundancy.

custom_model_runner/datarobot_drum/drum/entry_point.py#L9-L18

def run_drum_server():
options = setup_options()
if (
options.subparser_name == ArgumentsOptions.SERVER
and RuntimeParameters.has("DRUM_SERVER_TYPE")
and str(RuntimeParameters.get("DRUM_SERVER_TYPE")).lower() == "gunicorn"
):
main_gunicorn()
else:
main()

Fix in Cursor Fix in Web


@yakov-g yakov-g force-pushed the yakov/python-sklearn-gunicorn branch from 23016e2 to a86d8b7 Compare December 18, 2025 22:21

@pytest.mark.parametrize(
"with_error_server, production, docker",
[(False, False, None), (True, False, None), (True, True, DOCKER_PYTHON_SKLEARN)],
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Inconsistent test parameter updates for gunicorn compatibility

The PR updates test parameters for test_e2e_no_model_artifact and test_e2e_model_loading_fails from (True, True, DOCKER_PYTHON_SKLEARN) to (False, False, DOCKER_PYTHON_SKLEARN) with comments explaining that gunicorn doesn't support error server or production mode. However, test_ping_endpoints (line 108) and test_e2e_predict_fails (line 188) still use (True, True, DOCKER_PYTHON_SKLEARN). If gunicorn truly doesn't support these modes, these tests will fail when run against the updated sklearn docker image.

Additional Locations (2)

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants