Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
278a137
Add `macos` executor for unit tests
Pijukatel Jul 30, 2025
985d068
Update `uv.lock` to include latest `impit`
Pijukatel Jul 30, 2025
0ee161e
Increase test tolerance to pass the test on MacOS
Pijukatel Aug 12, 2025
e4e52ba
Merge remote-tracking branch 'origin/master' into run-uts-on-macos
Pijukatel Dec 3, 2025
3bec9bb
Revert changes to see status of the tests on MacOS
Pijukatel Dec 3, 2025
3730920
Add many debug logs for flaky test `test_timeout_in_handler`
Pijukatel Dec 4, 2025
0e6d87c
Add even more debug. Probably another bug in RQ
Pijukatel Dec 4, 2025
55eb4cb
Add more debug to RQ state
Pijukatel Dec 4, 2025
72f8451
Add debug to autoscaled pool. Is the problem there?
Pijukatel Dec 4, 2025
7771658
Some unimaginable execption is raised in autoscaled pool. What is it?…
Pijukatel Dec 4, 2025
1591984
Focus only onthe single flaky test
Pijukatel Dec 4, 2025
13325d7
Even more logs
Pijukatel Dec 4, 2025
8d79ada
Who is setting the result?
Pijukatel Dec 4, 2025
be6294e
Even more logs
Pijukatel Dec 4, 2025
1a71ab7
Is it `BaseException` ?
Pijukatel Dec 4, 2025
353e821
Add test tolerance. It is pytest stoping the test due to MacOS being …
Pijukatel Dec 4, 2025
02246f5
Pool deadlocked until timeout?
Pijukatel Dec 4, 2025
3befc3e
Seems like the pool is self-exhausting all the resources? Is it due t…
Pijukatel Dec 4, 2025
5d83c47
Try to add waits to avoid busy-waiting in cases where it makes no sense
Pijukatel Dec 5, 2025
2e85e3b
Merge remote-tracking branch 'origin/master' into run-uts-on-macos
Pijukatel Dec 5, 2025
4da8361
Add more CPU usage measurement logs
Pijukatel Dec 5, 2025
d3d5c03
See measurements problems with CPU usage
Pijukatel Dec 5, 2025
48830c3
See more detailed CPu measurements
Pijukatel Dec 5, 2025
25207b3
Try to disable spotlight on the mac executor
Pijukatel Dec 5, 2025
4435c8a
Try to cleanup mac executor even more
Pijukatel Dec 5, 2025
58c8140
Revert all but executor changes
Pijukatel Dec 5, 2025
fc0632f
Add test tolerance
Pijukatel Dec 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 35 additions & 6 deletions .github/workflows/run_code_checks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,41 @@ jobs:
python-versions: '["3.10", "3.11", "3.12", "3.13", "3.14"]'

unit_tests:
name: Unit tests
uses: apify/workflows/.github/workflows/python_unit_tests.yaml@main
secrets:
httpbin_url: ${{ secrets.APIFY_HTTPBIN_TOKEN && format('https://httpbin.apify.actor?token={0}', secrets.APIFY_HTTPBIN_TOKEN) || 'https://httpbin.org'}}
with:
python-versions: '["3.10", "3.11", "3.12", "3.13", "3.14"]'
strategy:
fail-fast: false
matrix:
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
runs-on: ${{ matrix.os }}
env:
HTTPBIN_URL: 'asd'

steps:
- name: macOS cleanup
if: runner.os == 'macOS'
# Disable Spotlight indexing and try to kill all useless processes that could drain CPU during tests
run: |
sudo mdutil -i off /
sudo killall Finder spindump ecosystemanalyticsd SystemUIServer NotificationCenter mds mds_stores mds_worker mdworker mdworker_shared || true

- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Set up uv package manager
uses: astral-sh/setup-uv@v6
with:
python-version: ${{ matrix.python-version }}

- name: Install Python dependencies
run: make install-dev

- name: Run unit tests
run: make unit-tests

docs_check:
name: Docs check
Expand Down
28 changes: 11 additions & 17 deletions tests/unit/events/test_local_event_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,24 @@

import asyncio
from datetime import timedelta
from functools import update_wrapper
from typing import Any
from unittest.mock import AsyncMock

import pytest

from crawlee.events import LocalEventManager
from crawlee.events._types import Event, EventSystemInfoData


@pytest.fixture
def listener() -> AsyncMock:
async def async_listener(payload: Any) -> None:
pass

al = AsyncMock()
update_wrapper(al, async_listener)
return al
async def test_emit_system_info_event() -> None:
mocked_listener = AsyncMock()

async def async_listener(payload: Any) -> None:
mocked_listener(payload)

async def test_emit_system_info_event(listener: AsyncMock) -> None:
async with LocalEventManager(system_info_interval=timedelta(milliseconds=50)) as event_manager:
event_manager.on(event=Event.SYSTEM_INFO, listener=listener)
await asyncio.sleep(0.2)
system_info_interval = timedelta(milliseconds=50)
test_tolerance_coefficient = 10
async with LocalEventManager(system_info_interval=system_info_interval) as event_manager:
event_manager.on(event=Event.SYSTEM_INFO, listener=async_listener)
await asyncio.sleep(system_info_interval.total_seconds() * test_tolerance_coefficient)

assert listener.call_count >= 1
assert isinstance(listener.call_args[0][0], EventSystemInfoData)
assert mocked_listener.call_count >= 1
assert isinstance(mocked_listener.call_args[0][0], EventSystemInfoData)
Loading