Please support dzen.ru quickly

[aflamrip]

Checklist

• I'm reporting a new site support request
• I've verified that I'm running youtube-dl version 2021.12.17
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that none of provided URLs violate any copyrights
• I've searched the bugtracker for similar site support requests including closed ones

Example URLs

• Single video: https://dzen.ru/video/watch/6471e50e06863726828b435c
• Single video: https://dzen.ru/embed/vnVEaPfaSym8?from_block=partner&from=zen&mute=0&autoplay=0&tv=0
• Playlist: nothing

Description

It is a very popular Russian site and needed to be supported

[dirkf]

Please review and complete the checklist (why is it there?).

The non-JS page seen by yt-dl contains a gigantic hydration JSON object as a parameter of a JS function that defines the page. Video links can be extracted using the traversal path 'data', ..., 0, 'videoViewer', 'items', ..., 0, 'rawStreams', 'SingleStream', ..., 'StreamInfo', ..., 'OutputStream'. 'Title' and 'Thumbnail' are available alongside 'StreamInfo'. I was able to watch the clip about the fake IKEA (why not ШведГаус?) using one of these links in mpv.

[aflamrip]

I have no experience in understanding these matters, but I hope that you will support downloading from this site, as it is a strong competitor to ok.ru and vk.com.

[dirkf]

I encourage someone who's interested to take this up. Otherwise it's at the end of a long queue.

[Revisto]

Hi, can I work on this issue as my first contribution to youtube-dl?

[dirkf]

By all means. Are you happy that you know what to do? See #29310 in addition to the manual and FAQ.

[3052]

note this uBlock Origin rule is needed for the web client:

dzen.ru dzeninfra.ru * noop

[abhimessi16]

Hi @dirkf, is @Revisto still working on the issue else i would like to work on it

[dirkf]

There's no PR yet. Unless we hear otherwise, carry on.

[Lil-Nugs]

Hi, I'm working on this right now but was wondering if there's a certain way we should handle redirects and how you got the page downloaded, @dirkf? I'm running into an issue where it hits these two URL's before the final webpage with the video and JSON object:

• https://sso.passport.yandex.ru/push?uuid=d738a747-82b1-432d-8536-30918ebb94aa&retpath=https%3A%2F%2Fdzen.ru%2Fvideo%2Fwatch%2F6471e50e06863726828b435c - sends back a 200
• https://sso.dzen.ru/install?uuid=604f2166-bd61-4d1a-bfed-715e331e51d6 - sends back a 200

So the _download_webpage_handle function is downloading the empty page at the first page it gets redirected to since they aren't sending back 302 for redirect afterwards.

[Lil-Nugs]

Hm looking at some other issues actually, seems like this could be bypassed by grabbing the cookies after visiting those URL's

[dirkf]

Exactly.

I found that I could just download the second problem video and find the media link. Looking at the first one, it seems to go to the SSO link that you mention, presumably because it needs a login. So the first thing to try is passing cookies from a logged-in browser session that can play that video.

Getting --username ... --password ... to work would be ideal but experience indicates that today's login procedures are either too hard to implement or too transient or both. But maybe passport.yandex.ru has been handled already -- I haven't checked.

[Debojit-0]

@dirkf
Even let me try to contribute on this as this is my first contribution to open source project.

[dirkf]

No-one has reported any further progress: have a go. Per #29724 (comment) and linked issue #29310, make sure to review the base class (extractor/common.py) and some other new extractors to understand the typical style and API usage.

[nolanwelch]

@Debojit-0 I previously made an attempt at implementing this extractor, please feel free to use my code here if you find it helpful.

[Debojit-0]

And were you able to resolve it ?

[nolanwelch]

Unfortunately no. But I think it's a good start.

[Debojit-0]

Ok fair enough its a good start for me as well as I am still in learning phase 🙂