Skip to content

Commit 15837c9

Browse files
dhohirpradanadhohirpradana
committed
skip channel
1 parent 5782b06 commit 15837c9

File tree

3 files changed

+174
-5
lines changed

3 files changed

+174
-5
lines changed

‎geckodriver.log‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3232,3 +3232,166 @@ WebDriver BiDi listening on ws://127.0.0.1:51604
32323232
[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt
32333233
console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at C:\\Users\\dhohi\\AppData\\Local\\Temp\\rust_mozprofilenje9Nv\\search.json.mozlz4", (void 0)))
32343234
DevTools listening on ws://127.0.0.1:51604/devtools/browser/05d9e3ea-d872-4db7-b496-64e00ebd66fb
3235+
1675406291048 geckodriver INFO Listening on 127.0.0.1:51698
3236+
1675406294111 mozrunner::runner INFO Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "--marionette" "--headless" "--window-size=1920,1080" "--d ... 9" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\dhohi\\AppData\\Local\\Temp\\rust_mozprofileGeT2Hk"
3237+
*** You are running in headless mode.
3238+
1675406294742 Marionette INFO Marionette enabled
3239+
Dynamically enable window occlusion 0
3240+
1675406294750 Marionette INFO Listening on port 51706
3241+
Read port: 51706
3242+
WebDriver BiDi listening on ws://127.0.0.1:51699
3243+
1675406294994 RemoteAgent WARN TLS certificate errors will be ignored for this session
3244+
[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt
3245+
console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at C:\\Users\\dhohi\\AppData\\Local\\Temp\\rust_mozprofileGeT2Hk\\search.json.mozlz4", (void 0)))
3246+
DevTools listening on ws://127.0.0.1:51699/devtools/browser/efe4308a-5a97-4768-994c-73d7dd42ca69
3247+
1675406385637 Marionette WARN Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.
3248+
[2023-02-03T06:39:58Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3249+
[2023-02-03T06:39:58Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3250+
[2023-02-03T06:39:58Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3251+
[2023-02-03T06:39:58Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3252+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd23bfe00 state=DECODING_METADATA Decode metadata failed, shutting down decoder: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachine.cpp:372
3253+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd23bfe00 Decode error: NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006) - static MP4Metadata::ResultAndByteBuffer __cdecl mozilla::MP4Metadata::Metadata(mozilla::ByteStream *): Cannot parse metadata: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachineBase.cpp:164
3254+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd3d58700 state=DECODING_METADATA Decode metadata failed, shutting down decoder: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachine.cpp:372
3255+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd3d58700 Decode error: NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006) - static MP4Metadata::ResultAndByteBuffer __cdecl mozilla::MP4Metadata::Metadata(mozilla::ByteStream *): Cannot parse metadata: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachineBase.cpp:164
3256+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd3f10700 state=DECODING_METADATA Decode metadata failed, shutting down decoder: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachine.cpp:372
3257+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd3f10700 Decode error: NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006) - static MP4Metadata::ResultAndByteBuffer __cdecl mozilla::MP4Metadata::Metadata(mozilla::ByteStream *): Cannot parse metadata: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachineBase.cpp:164
3258+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd3f10d00 state=DECODING_METADATA Decode metadata failed, shutting down decoder: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachine.cpp:372
3259+
[Child 19448, MediaDecoderStateMachine #1] WARNING: Decoder=1cdd3f10d00 Decode error: NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006) - static MP4Metadata::ResultAndByteBuffer __cdecl mozilla::MP4Metadata::Metadata(mozilla::ByteStream *): Cannot parse metadata: file /builds/worker/checkouts/gecko/dom/media/MediaDecoderStateMachineBase.cpp:164
3260+
1675406429274 Marionette INFO Stopped listening on port 51706
3261+
Dynamically enable window occlusion 1
3262+
JavaScript error: https://www.youtube.com/results?search_query=berita%20terbaru&themeRefresh=1, line 13: TypeError: can't access dead object
3263+
1675407005423 geckodriver INFO Listening on 127.0.0.1:52126
3264+
1675407008486 mozrunner::runner INFO Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "--marionette" "--headless" "--window-size=1920,1080" "--d ... 7" "--remote-allow-hosts" "localhost" "-no-remote" "-profile" "C:\\Users\\dhohi\\AppData\\Local\\Temp\\rust_mozprofileURl7AD"
3265+
*** You are running in headless mode.
3266+
1675407009116 Marionette INFO Marionette enabled
3267+
Dynamically enable window occlusion 0
3268+
1675407009126 Marionette INFO Listening on port 52136
3269+
Read port: 52136
3270+
WebDriver BiDi listening on ws://127.0.0.1:52127
3271+
1675407009355 RemoteAgent WARN TLS certificate errors will be ignored for this session
3272+
[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt
3273+
console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at C:\\Users\\dhohi\\AppData\\Local\\Temp\\rust_mozprofileURl7AD\\search.json.mozlz4", (void 0)))
3274+
DevTools listening on ws://127.0.0.1:52127/devtools/browser/76f19889-47a3-4928-b15e-f1ac1d0d5425
3275+
1675407034672 Marionette WARN Ignoring event 'DOMContentLoaded' because document has an invalid readyState of 'complete'.
3276+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3277+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3278+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3279+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3280+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3281+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3282+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3283+
[2023-02-03T06:50:37Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
3284+
1675407042456 Marionette INFO Stopped listening on port 52136
3285+
Dynamically enable window occlusion 1
3286+
JavaScript error: https://www.youtube.com/results?search_query=babahaha&themeRefresh=1, line 12: TypeError: can't access dead object
3287+
WARNING: A blocker encountered an error while we were waiting.
3288+
Blocker: Waiting for ping task
3289+
Phase: TelemetryController: Waiting for pending ping activity
3290+
State: (none)
3291+
WARNING: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3292+
WARNING: addBlocker@resource://gre/modules/AsyncShutdown.jsm:718:15
3293+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:523:26
3294+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:462:15
3295+
setupShutdown@resource://gre/modules/osfile/osfile_async_front.jsm:1548:28
3296+
@resource://gre/modules/osfile/osfile_async_front.jsm:1568:16
3297+
@resource://gre/modules/osfile.jsm:12:30
3298+
@resource://gre/modules/TelemetryStorage.jsm:19:28
3299+
promiseArchivePing@resource://gre/modules/TelemetryArchive.jsm:103:5
3300+
promiseArchivePing@resource://gre/modules/TelemetryArchive.jsm:63:33
3301+
_submitPingLogic@resource://gre/modules/TelemetryControllerParent.jsm:540:48
3302+
send@resource://gre/modules/TelemetryControllerParent.jsm:628:24
3303+
submitExternalPing@resource://gre/modules/TelemetryControllerParent.jsm:182:17
3304+
sendPing@resource://gre/modules/EventPing.jsm:51:30
3305+
_submitPing@resource://gre/modules/EventPing.jsm:229:12
3306+
shutdown@resource://gre/modules/EventPing.jsm:112:10
3307+
_cleanupOnShutdown@resource://gre/modules/TelemetryControllerParent.jsm:940:31
3308+
shutdown/<@resource://gre/modules/TelemetryControllerParent.jsm:1010:24
3309+
observe@resource://gre/modules/AsyncShutdown.jsm:572:16
3310+
3311+
WARNING: A blocker encountered an error while we were waiting.
3312+
Blocker: Waiting for ping task
3313+
Phase: TelemetryController: Waiting for pending ping activity
3314+
State: (none)
3315+
WARNING: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3316+
WARNING: addBlocker@resource://gre/modules/AsyncShutdown.jsm:718:15
3317+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:523:26
3318+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:462:15
3319+
setupShutdown@resource://gre/modules/osfile/osfile_async_front.jsm:1548:28
3320+
@resource://gre/modules/osfile/osfile_async_front.jsm:1568:16
3321+
@resource://gre/modules/osfile.jsm:12:30
3322+
@resource://gre/modules/TelemetryStorage.jsm:19:28
3323+
promiseArchivePing@resource://gre/modules/TelemetryArchive.jsm:103:5
3324+
promiseArchivePing@resource://gre/modules/TelemetryArchive.jsm:63:33
3325+
_submitPingLogic@resource://gre/modules/TelemetryControllerParent.jsm:540:48
3326+
send@resource://gre/modules/TelemetryControllerParent.jsm:628:24
3327+
submitExternalPing@resource://gre/modules/TelemetryControllerParent.jsm:182:17
3328+
saveShutdownPings@resource://gre/modules/TelemetrySession.jsm:982:34
3329+
shutdownChromeProcess/cleanup/<@resource://gre/modules/TelemetrySession.jsm:1232:20
3330+
cleanup@resource://gre/modules/TelemetrySession.jsm:1239:9
3331+
shutdownChromeProcess@resource://gre/modules/TelemetrySession.jsm:1256:14
3332+
shutdown@resource://gre/modules/TelemetrySession.jsm:247:17
3333+
_cleanupOnShutdown@resource://gre/modules/TelemetryControllerParent.jsm:959:35
3334+
observe@resource://gre/modules/AsyncShutdown.jsm:572:16
3335+
3336+
WARNING: A blocker encountered an error while we were waiting.
3337+
Blocker: Waiting for ping task
3338+
Phase: TelemetryController: Waiting for pending ping activity
3339+
State: (none)
3340+
WARNING: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3341+
WARNING: addBlocker@resource://gre/modules/AsyncShutdown.jsm:718:15
3342+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:523:26
3343+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:462:15
3344+
setupShutdown@resource://gre/modules/osfile/osfile_async_front.jsm:1548:28
3345+
@resource://gre/modules/osfile/osfile_async_front.jsm:1568:16
3346+
@resource://gre/modules/osfile.jsm:12:30
3347+
@resource://gre/modules/TelemetryStorage.jsm:19:28
3348+
promiseArchivePing@resource://gre/modules/TelemetryArchive.jsm:103:5
3349+
promiseArchivePing@resource://gre/modules/TelemetryArchive.jsm:63:33
3350+
_submitPingLogic@resource://gre/modules/TelemetryControllerParent.jsm:540:48
3351+
send@resource://gre/modules/TelemetryControllerParent.jsm:628:24
3352+
submitExternalPing@resource://gre/modules/TelemetryControllerParent.jsm:182:17
3353+
saveShutdownPings@resource://gre/modules/TelemetrySession.jsm:1014:36
3354+
shutdownChromeProcess/cleanup/<@resource://gre/modules/TelemetrySession.jsm:1232:20
3355+
cleanup@resource://gre/modules/TelemetrySession.jsm:1239:9
3356+
shutdownChromeProcess@resource://gre/modules/TelemetrySession.jsm:1256:14
3357+
shutdown@resource://gre/modules/TelemetrySession.jsm:247:17
3358+
_cleanupOnShutdown@resource://gre/modules/TelemetryControllerParent.jsm:959:35
3359+
observe@resource://gre/modules/AsyncShutdown.jsm:572:16
3360+
3361+
WARNING: A blocker encountered an error while we were waiting.
3362+
Blocker: TelemetryController: shutting down
3363+
Phase: profile-before-change-telemetry
3364+
State: Error getting state: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange" at addBlocker@resource://gre/modules/AsyncShutdown.jsm:718:15
3365+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:523:26
3366+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:462:15
3367+
setupShutdown@resource://gre/modules/osfile/osfile_async_front.jsm:1548:28
3368+
@resource://gre/modules/osfile/osfile_async_front.jsm:1568:16
3369+
@resource://gre/modules/osfile.jsm:12:30
3370+
@resource://gre/modules/TelemetryStorage.jsm:19:28
3371+
getShutdownState@resource://gre/modules/TelemetrySend.jsm:1692:7
3372+
getShutdownState@resource://gre/modules/TelemetrySend.jsm:374:30
3373+
_getState@resource://gre/modules/TelemetryControllerParent.jsm:1058:38
3374+
setupTelemetry/<@resource://gre/modules/TelemetryControllerParent.jsm:917:18
3375+
safeGetState@resource://gre/modules/AsyncShutdown.jsm:254:17
3376+
Barrier/_internalAddBlocker/promise<@resource://gre/modules/AsyncShutdown.jsm:773:20
3377+
observe@resource://gre/modules/AsyncShutdown.jsm:572:16
3378+
3379+
WARNING: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3380+
WARNING: addBlocker@resource://gre/modules/AsyncShutdown.jsm:718:15
3381+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:523:26
3382+
addBlocker@resource://gre/modules/AsyncShutdown.jsm:462:15
3383+
setupShutdown@resource://gre/modules/osfile/osfile_async_front.jsm:1548:28
3384+
@resource://gre/modules/osfile/osfile_async_front.jsm:1568:16
3385+
@resource://gre/modules/osfile.jsm:12:30
3386+
@resource://gre/modules/TelemetryStorage.jsm:19:28
3387+
removeAbortedSessionPing@resource://gre/modules/TelemetryControllerParent.jsm:721:5
3388+
removeAbortedSessionPing@resource://gre/modules/TelemetryControllerParent.jsm:249:17
3389+
shutdownChromeProcess/cleanup/<@resource://gre/modules/TelemetrySession.jsm:1235:42
3390+
observe@resource://gre/modules/AsyncShutdown.jsm:572:16
3391+
3392+
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3393+
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3394+
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3395+
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3396+
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"
3397+
JavaScript error: resource://gre/modules/AsyncShutdown.jsm, line 718: Error: Phase "profile-before-change" is finished, it is too late to register completion condition "OS.File: flush I/O queued before profileBeforeChange"

‎results/babahaha_scroll-1_20230203_135030.txt‎

Whitespace-only changes.

‎yt_scraper_sroll/__init__.py‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -66,8 +66,14 @@ def handler(request, jsonify):
6666
# print('video_ids: ', video_ids)
6767

6868
for i, video_id in enumerate(video_ids):
69-
# check playlist or not
69+
# skip playlist
7070
if "list" in video_id.get_attribute("href"):
71+
print("playlist")
72+
continue
73+
74+
# skip channel
75+
if video_id.get_attribute("href").startswith("/@"):
76+
print("channel")
7177
continue
7278

7379
video_links.append(video_id.get_attribute("href"))
@@ -91,10 +97,10 @@ def handler(request, jsonify):
9197

9298
time.sleep(3)
9399

94-
# print("video_links:", len(video_links))
95-
# print("video_titles:", len(video_titles))
96-
# print("video_views:", len(video_views))
97-
# print("video_published_times:", len(video_published_times))
100+
print("video_links:", len(video_links))
101+
print("video_titles:", len(video_titles))
102+
print("video_views:", len(video_views))
103+
print("video_published_times:", len(video_published_times))
98104

99105
# write to file
100106
with open(f"{BASE_DIR}/results/{file_name}.txt", "a", encoding="utf-8") as f:

0 commit comments

Comments
 (0)