Tags
I came across a Flash video that I was not able to save with any Video Downloader app, including the ones that actually sniff traffic on your network adapter, such as Replay Media Catcher and many others.
Turns out that this particular page was using the new Adobe HTTP Dynamic Streaming (HDS) technology. With HDS, the original MP4 or FLV file is split up into many F4F segments, which are then served to the media player on the page one after the other, so there is no single video file to download like with most other video streaming technologies.
You can easily check if HDS is being used by using Firefox to watch the video.
- Clear Firefox cache (Tools -> Options -> Network, Clear Cached Web Content, Clear User Data)
- Load the page with the video
- Open a new tab and browse to about:cache?storage=disk
- Search for a bunch of files that have the word ‘Frag’ in them. They’ll look something like this:
http://ams-vp11.9c9media.com/hds-vod/ae/2015-01-29/3FA6DB15557BA5F0/CTVNews-546418-29-WPG-WEBPARKOUR08-SOT-Adaptive_08.mp4Seg1-Frag39 http://ams-vp11.9c9media.com/hds-vod/ae/2015-01-29/3FA6DB15557BA5F0/CTVNews-546418-29-WPG-WEBPARKOUR08-SOT-Adaptive_08.mp4Seg1-Frag38 http://ams-vp11.9c9media.com/hds-vod/ae/2015-01-29/3FA6DB15557BA5F0/CTVNews-546418-29-WPG-WEBPARKOUR08-SOT-Adaptive_08.mp4Seg1-Frag37 http://ams-vp11.9c9media.com/hds-vod/ae/2015-01-29/3FA6DB15557BA5F0/CTVNews-546418-29-WPG-WEBPARKOUR08-SOT-Adaptive_08.mp4Seg1-Frag36
These are all the F4F fragments of the video. You could download them all and combine them together, but that’s not the best way to do this.
There is a script called AdobeHDS.php
This manifest file is easy to obtain, b/c it is delivered via a plain GET request that is issued before the video starts playing. To find the URL:
- Open Firefox Console (Ctrl+Shift+K) or Tools -> Web Developer -> Web Console
- Make sure that “Net” filter is selected
- Clear the Console
- Open the video page and let the video load
- In the Filter text box type “f4m” and you should now see a few F4M requests. You want the first one, which will probably be called “manifest.f4m“. Mine looked like this:
GET http://capi.9c9media.com/destinations/ctvnews_web/platforms/desktop/contents/540901/contentpackages/546418/stacks/1130329/manifest.f4m
Now just run the script with the manifest URL and you should get the re-combined flv file:
$ php AdobeHDS.php --delete --manifest "http://capi.9c9media.com/destinations/ctvnews_web/platforms/desktop/contents/540901/contentpackages/546418/stacks/1130329/manifest.f4m" KSV Adobe HDS Downloader Processing manifest info.... Quality Selection: Available: 2048 1856 1536 1280 896 640 480 299 Selected : 2048 Fragments Total: 55, First: 1, Start: 1, Parallel: 8 Downloading 55/55 fragments Found 55 fragments Finished
You should now have an FLV file waiting for you in the script directory.
For Mac Users
Posting some info from a comment by Eric L. Pheterson below:
To add a few more baby steps (for Mac users) :
- When you view the AdobeHDS.php file at Sourceforge, copy/paste it into a file, and name it AdobeHDS.php
- PHP should be installed alreadyon your mac
- A dependency of AdobeHDS is not installed, so in Terminal run :
brew install homebrew/php/php55-mcrypt
- After installing mcrypt, you must open a new terminal window or tab to use it
- If you don’t have brew installed, in Terminal run :
/usr/bin/ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”
- After installing brew, run
brew update
- This Firefox extension will perfectly present you with the php command you need : https://addons.mozilla.org/en-US/firefox/addon/hds-link-detector/
N1nja Hacks 
Hello Thanks for your post. I am a complete newb at this I do video editing only but over the next few months I have to be able to capture 72 videos that are using this new streaming code. When I look at your article it looks like exactly what I need but it also looks like a foreign language to me. I could really use some help if you are willing to do so. I don’t even know how to use a php script. I am trying to learn as I go but having someone that can help me out when I have questions would be a blessing. So if you are feeling generous and want to help me out please shoot me a email. I entered it to post this comment but if you need it please just let me know. I am really hoping you will get back to me until then thanks for this post. – Charlie
How do I install the script. everything I read says that you need to setup your computer as a server?
@Charlie: Although I have never tried running a PHP script on Windows, I am still pretty sure that you don’t need to install a web server. Just install php as explained here: http://php.net/manual/en/install.windows.manual.php, and then use php.exe executable to run AdobeHDS.php from the command line. I think that will work…
As for helping you, feel free to ask specific questions here. I’ll help if I can, although there might be a limit to how helpful I can be, b/c I haven’t used Windows in over a decade, and I assume that this is what you are using for your project.
it is saying Access denied what would be the issue?
same here, got the script working, ended up with:
“Processing manifest info….
Quality Selection:
Available: 651 494 205 41 40
Selected : 651
Fragments Total: 120, First: 1, Start: 1, Parallel: 8
Access Denied! Unable to download fragments.”
any ideas? thanks!
hi I did all right but I can not understand what you means with ”Now just run the script with the manifest URL and you should get the re-combined flv file:”
how I can run the script ?
can you explain better ?
thanks
Hi John,
Not sure what you are asking. The syntax for running the script is shown right under the sentence you asked about. Is there something you don’t understand about the command shown below?
Did you download AdobeHDS.php? Do you have php installed? Did you pull out the manifest URL with Firefox console, as explained in the paragraph above? If so, just run AdobeHDS.php the way shown in the article.
to run the script, open a terminal window, type “php”, drag the AdobeHDS.php file in, type “–delete –manifest” and then paste the manifest in (don´t forget the “”).
This helped me a ton, thank you!
To add a few more baby steps (for Mac users) :
– When you view the AdobeHDS.php file at Sourceforge, copy/paste it into a file, and name it AdobeHDS.php
– PHP should be installed alreadyon your mac
– A dependency of AdobeHDS is not installed, so in Terminal run : brew install homebrew/php/php55-mcrypt
– After installing mcrypt, you must open a new terminal window or tab to use it
– If you don’t have brew installed, in Terminal run : /usr/bin/ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”
– After installing brew, run : brew update
– This Firefox extension will perfectly present you with the php command you need : https://addons.mozilla.org/en-US/firefox/addon/hds-link-detector/
Unfortunately, the Mac instructions no longer work. Trying to install mcrypt fails:
==> Installing homebrew/php/php55-mcrypt dependency: libpng
==> Downloading https://downloads.sourceforge.net/project/libpng/libpng16/1.6.17
curl: (22) The requested URL returned error: 404 Not Found
Trying a mirror…
==> Downloading ftp://ftp.simplesystems.org/pub/libpng/png/src/libpng16/libpng-1
curl: (78) RETR response: 550
Error: Failed to download resource “libpng”
Download failed: ftp://ftp.simplesystems.org/pub/libpng/png/src/libpng16/libpng-1.6.17.tar.xz
And trying to brew update just pushes out Permission denied errors that even sudo can’t overcome.
Are you outside of US or something? I had no issues getting mcrypt installed after brew update. Try re-installing brew? Reboot?
mcrypt is no longer a quick install on Macs since OS X 11 El Capitan came out. I found a web site that has instructions, and they’re fairly complicated, including circumventing the System Integrity daemon: https://michaelgracie.com/2015/10/plugging-mcrypt-into-php-on-mac-os-x-el-capitan-10-11/
Even that web site’s instructions aren’t quite complete. It forgets to include the instruction to install the developer command line tools, and it should be updated so that you end up downloading the version of PHP that matches your OSX point release, and not an older one.
It took several hours, but I did manage to get it to work.
[English]
Hello, guys. As I’ve noticed that people are having some troubles doing the procedure, I will tell step-by-step what I’ve done that returns sucessful.
First of all, my pc qualifies and files that I’ve used to make things work:
* Windows 10;
* PHP 7.0 (7.0.4) available at http://windows.php.net/download#php-7.0 . As my CPU is x64, I’ve downloaded this one: http://windows.php.net/downloads/releases/php-7.0.4-Win32-VC14-x64.zip
* VC14 available at http://www.microsoft.com/en-us/download/details.aspx?id=48145
* AdobeHDS.php file available at https://github.com/K-S-V/Scripts
* Mozilla Firefox 45.0.1;
* Mozilla ADDon available at https://addons.mozilla.org/pt-br/firefox/addon/hds-link-detector/
Step 1: Download and unzip all the content of PHP zip file athe the root (withou subpasts) of the path “c:\php”
Step 2: Install VC14 file (x86 or x64 version);
Step 3: Copy AdobeHDS.php file to the root path of “c:\php”
Step 4: Install hds link detector addon at mozilla firefox;
Step 5: Make a copy of the file “c:\php\php.ini-production”. With the copy, rename it to php.ini and open the file. Look for “curl”. Uncomment the line containing “;extension=php_curl.dll” by removing the “;” and kepping only “extension=php_curl.dll” without the quotes (“”).
** The steps dont necessarily be followed in that order, but all are required.
Step 6: Open the mozilla firefox browser and play your video. Make sure that the HDS LINK DETECTOR is enable. There is a red icon on right corner that must be red. If itis gray, the app is off. Click on it to activate. When the video is starting, it will popup a box that say to click on it to clipboard a code. Click on it.
Step 7: Open the command prompt of Windows.
Step 8: In the prompt, go the php path. You must use a command such as: “cd\php” to go to the php folder that was created on step 1.
Step 9: Paste the command coppied on the step 6 on the prompt.
Step 9: Enjoy your video that must be in the php path.
[Portuguê-BR]
Olá, pessoal. Como tenho notado que as pessoas estão tendo dificuldades com o procedimento, irei descrever passo-a-passo o que eu fiz para dar certo.
Primeiramente, seguem as configurações do meu computador e os arquivos necessários para as coisas funcionarem:
* Windows 10;
* PHP 7.0 (7.0.4) disponível http://windows.php.net/download#php-7.0 . Como meu computador tem processador x64, eu fiz o download deste pacote: http://windows.php.net/downloads/releases/php-7.0.4-Win32-VC14-x64.zip
* VC14 disponível em http://www.microsoft.com/en-us/download/details.aspx?id=48145
* O arquivo AdobeHDS.php disponível em https://github.com/K-S-V/Scripts
* Mozilla Firefox 45.0.1;
* ADDon do Mozilla disponível em https://addons.mozilla.org/pt-br/firefox/addon/hds-link-detector/
Passo 1: Faça o download and descompacte todo o conteúdo do arquivo zipado do PHP zip na pasta principal do “c:\php” (com isso, quero dizer que o conteúdo do arquivo zip não pode ficar em subpastas, tal como c:\php\php\)
Passo 2: Instale o arquivo VC14 (versão x86 ou x64);
Passo 3: Copie o arquivo AdobeHDS.php para a pasta principal de “c:\php”
Passo 4: Instale o addon “hds link detector” no mozilla firefox;
Passo 5: Faça uma cópia do arquivo “c:\php\php.ini-production”. Com a cópia, renomeie-a para php.ini e abra o arquivo. Procure por “curl”. “Descomente” a linha contendo “;extension=php_curl.dll” removendo o “;” e mantendo apenas “extension=php_curl.dll” sem aspas (“”).
** Estes passos não precisam ser feitos necessariamente nessa mesma ordem, mas todos tão obrigatórios.
** Para quem ficou confuso com a expressão “descomentar”, é o seguinte: alguns programas contêm arquivos configuráveis, onde o usuário pode manualmente fazer modificações, geralmente no formato “.ini”. O php tem o php.ini. Nele, todos os comandos que o programa NÃO deve executar (ler) estão iniciados com “;” (“comentados”). Se você tirar o “;”, ou seja, “descomentar”, o programa irá executar aquele comando.
Passo 6: Abra o navegador do Mozilla Firefox e inicie o vídeo desejado. Antes, tenha certeza que o addon HDS LINK DETECTOR esteja ativado. Deve haver um ícone no canto superior direito que deve estar vermelho. Se estiver cinza, o aplicativo está desativado. Clique nele para ativar. Quando o vídeo começar, irá “pular” uma caixinha que diz para clicar nela para copiar um código. Clique na caixinha.
Passo 7: Abra o prompt de comando do windows.
Passo 8: No prompt, vá até a pasta do php, criada no passo 1. Para quem não sabe usar muito o prompt, dê uma olhada na internet, mas o comando para ir até a pasta deve ser algo do tipo: “cd\php” (sem aspas).
Passo 9: Cole o comando copiado no passo 6.
Passo 10: Aproveite o vídeo baixado, que deve estar na pasta do php.
Thanks very much. It worked perfectly.
don’t understand what to do when I get to the page https://github.com/K-S-V/Scripts/blob/master/AdobeHDS.php . What am I supposed to do with this code, copy & paste somewhere?
No, you are to save that file on your computer. Click on Raw -> Right-click -> Save as…
I attempted to use this approach with the flash video stream on this page without success:
http://www3.nhk.or.jp/nhkworld/en/vod/jtripplan/20160418.html
I installed the HDS Link Detector Add-On for FireFox, but I don’t get a notification with the php command line. What am I missing? Is another approach needed for this flash video stream?
Dave
I want to save the flash video stream on this page:
http://www3.nhk.or.jp/nhkworld/en/vod/jtripplan/20160418.html
I installed the HDS Link Detector FireFox Add-on, but I get no command line to execute. Is another approach required for this video stream? How can I save it?
Dave
Follow the steps listed above by Norton.
I tried it and it worked.
I downloaded the video successfully.
Pingback: Загрузка видео из Flash | 攻殻機動隊
doesnt work for me
Can someone help me to download some videos from this website: http://www.trainerpass.com/
I have an old mac (no updates anymore) and i’m using chrome. thanks in advance everyone
So i thought I found a video similar to the post. However, I think it is different. Any information on how I can extract the video off of http://allvod.sbs.co.kr/allvod/vodEndPage.do?mdaId=22000028670&btn=free . The last video, not the commercials (they are in plain mp4).
I am also downloading from this website. Does it work?
I would like very much to root my phone. I have looked at countless videos on You Tube, but somehow I just don’t know what to trust.
I have a ZTE ZMAX PRO 6.0.1 MARSHMALLOW (Android) phone.
Could someone please help me. I have never rooted a phone before, so I am nervous about it.
Thank you
Pingback: Saving Lecture Videos Embedded in Adobe HDS – Coding Bull Junky
Thanks Ninja! I’d been wanting to download a video from CTV News (Canada channel) that wouldn’t download with YouTube DL or RTMP Dump or any of the more common tools. I knew it wasn’t a geoblocking issue, as even though I’m in the USA, the video played in the browser. Didn’t know about this Adobe HDS script, but it worked wonderfully! Where there’s a will, there’s a way, eh? 🙂
Dear i want to download videos from http://www.channel4.com. i used the above method very carefully but no success. It is worth mentioning that i am outside UK and using browser vpn for viewwing contents online on channel4. Please help me for my life.