You are here

error '404' with hoerzu.de or sky.de

6 posts / 0 new
Last post
Bernard
Offline
Donator
Joined: 6 months
Last seen: 3 months
error '404' with hoerzu.de or sky.de

Hi!

I'm happy to say that finally I got Webgrab+plus up and running. Using a debian-VM.

During configuration I tried to get my requests a minimal amount of 'siteini'.
At first I used tvinfo which - for my first two channels - did the job. Changing this to hoerzu.de and sky.de I'm getting errors:

found: /home/epg/webgrabplus_data/.wg++/./siteini.pack/Germany/hoerzu.de.ini -- Revision 00
[ Info ] encrypted in 'new (V3)' mode
[ Debug ]
[ Info ] found: /home/epg/webgrabplus_data/.wg++/./siteini.pack/Germany/sky.de.ini -- Revision 06
[ Info ] encrypted in 'new (V3)' mode
[ Debug ] xmltv input file - /home/epg/webgrabplus_data/epg/halut-guide.xml - found
[ Debug ] 0 superfluous shows removed
[ Info ] Found existing channel (xmltv_id=Das Erste) in the config file
[ Info ] Found existing channel (xmltv_id=ZDF) in the config file
[ Info ] Found existing channel (xmltv_id=ARD alpha) in the config file
[ Info ] Found existing channel (xmltv_id=ZDFinfo) in the config file
[ Info ] Found existing channel (xmltv_id=ZDFneo) in the config file
[ Info ] Found existing channel (xmltv_id=WDR) in the config file
[ Info ] Found existing channel (xmltv_id=RTL) in the config file
[ Info ] Found existing channel (xmltv_id=RTL Zwei) in the config file
[ Info ] Found existing channel (xmltv_id=RTL Living) in the config file
[ Info ] Found existing channel (xmltv_id=RTL Passion) in the config file
[ Info ] Found existing channel (xmltv_id=RTL Crime) in the config file
[ Info ] Found existing channel (xmltv_id=NITRO) in the config file
[ Info ] Found existing channel (xmltv_id=Sat.1) in the config file
[ Info ] Found existing channel (xmltv_id=Sat.1 emotions) in the config file
[ Info ] Found existing channel (xmltv_id=Sat.1 Gold) in the config file
[ Info ] Found existing channel (xmltv_id=ProSieben) in the config file
[ Info ] Found existing channel (xmltv_id=ProSieben Maxx) in the config file
[ Info ] Found existing channel (xmltv_id=ProSieben Fun) in the config file
[ Info ] Found existing channel (xmltv_id=kabel eins) in the config file
[ Info ] Found existing channel (xmltv_id=Kabel 1 Doku) in the config file
[ Info ] Found existing channel (xmltv_id=Super RTL) in the config file
[ Info ] Found existing channel (xmltv_id=Arte) in the config file
[ Info ] Found existing channel (xmltv_id=VOX) in the config file
[ Info ] Found existing channel (xmltv_id=Sixx) in the config file
[ Info ] Found existing channel (xmltv_id=DMAX) in the config file
[ Info ] Found existing channel (xmltv_id=Disney Channel) in the config file
[ Info ] Found existing channel (xmltv_id=Discovery Channel) in the config file
[ Info ] Found existing channel (xmltv_id=National Geographic) in the config file
[ Info ] Found existing channel (xmltv_id=National Geographic Wild) in the config file
[ Info ] Found existing channel (xmltv_id=History) in the config file
[ Info ] Found existing channel (xmltv_id=Sky Nature HD) in the config file
[ Info ] Found existing channel (xmltv_id=Sky Documentaries HD) in the config file
[ Info ] Found existing channel (xmltv_id=Animal Planet) in the config file
[ Info ] Found existing channel (xmltv_id=Spiegel Geschichte) in the config file
[ Info ] Found existing channel (xmltv_id=Warner TV Film HD) in the config file
[ Info ] Found existing channel (xmltv_id=Warner TV Serie HD) in the config file
[ Info ] Found existing channel (xmltv_id=Warner TV Comedy HD) in the config file
[ Info ] Found existing channel (xmltv_id=Universal TV) in the config file
[ Info ] Found existing channel (xmltv_id=Comedy Central) in the config file
[ Info ] Found existing channel (xmltv_id=Welt der Wunder) in the config file
[ Info ] Found existing channel (xmltv_id=Geo Television) in the config file
[ Info ] Found existing channel (xmltv_id=MTV) in the config file
[ Info ] Found existing channel (xmltv_id=Deluxe Music) in the config file
[ Info ] Found existing channel (xmltv_id=Jukebox) in the config file
[ Info ]
[ Info ]
[ Info ] i=index .=same c=change g=gab r=replace n=new
[ Info ]
[ Info ]
[ Info ] Group (0) :
[ Info ] update requested for - 44 - out of - 44 - channels for 14 day(s)
[ Debug ]
[ Info ] ( 1/44 ) HOERZU.DE -- chan. (xmltv_id=Das Erste) -- mode Incremental
[Warning ] error downloading page: Response status code does not indicate success: 404 (Not Found).
[Error ] Unable to update channel Das Erste
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from Das Erste
[Error ] unable to update channel, try again later
[ Info ] No guide data to restore
[ Debug ]
[ Info ] ( 1/44 ) HOERZU.DE -- chan. (xmltv_id=ZDF) -- mode Incremental
[Warning ] error downloading page: Response status code does not indicate success: 404 (Not Found).
[Error ] Unable to update channel ZDF
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from ZDF
[Error ] unable to update channel, try again later
[ Info ] Existing guide data restored!

and so on and on.

Is there anything I did wrong on my side? Especially sky.de is something I would like to use...

Any hints?

Kind regards,

Blackbear199
Offline
Blackbear199's picture
WG++ Team memberDonator
Joined: 9 years
Last seen: 10 min

hoerzu.de is gone i think.
sky.de should work.

other good germany ones are web.magentatv,de,tvtoday.de,tvdigital.de,tvinfo.de to name a few.

you should always create a new channel list,update c usually works.see..
http://webgrabplus.com/documentation/configuration/sitechannelsxml

if you still cannot get things working upload your webgrab config file.
DO NOT forget to remove your license info.

a few common issues.
for the retry setting time-out="5" is usually too low,i recommend 30
make sure you have a user-agent
see..
http://webgrabplus.com/documentation/configuration/webgrabconfigxml

Bernard
Offline
Donator
Joined: 6 months
Last seen: 3 months

Hi Blackbear199!

I changed my provider to magentatv and it is working now.

But sky is still giving an error:

----------------------------------------------------------
[ Debug ]
[ Info ] ( 117/130 ) SKY.DE -- chan. (xmltv_id=Sky Sport Austria 7 HD) -- mode Incremental
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Warning ] error downloading page: Response status code does not indicate success: 403 (Forbidden).
[Error ] Unable to update channel Sky Sport Austria 7 HD
[Critical] Generic syntax exception:
[Critical] message:
[Error ] no index page data received from Sky Sport Austria 7 HD
[Error ] unable to update channel, try again later
[ Info ] Existing guide data restored!
[ Debug ]
-----------------------------------------------------------

Can this be fixed?

Kind regards,

Blackbear199
Offline
Blackbear199's picture
WG++ Team memberDonator
Joined: 9 years
Last seen: 10 min

it appears they are banning ip's if you grab too much data.
i got it to work once and then ran it again and got 403 forbidden also.
its probably a temporary ban and it will work again if i wait awhile.

edit the ini and change this line..
site {episodesystem=onscreen}
to
site {episodesystem=onscreen|retry=<retry channel-delay="1" index-delay="1" show-delay="1" time-out="30">4</retry>}

the delays will slow webgrab down.
these are in seconds so u may want to set them higher at first to make sure you dont get a ip ban and lower them a little at a time(1 sec interval).

the most important one is the show-delay,this is the one probably causing the ban.
i would go with a show-delay of 2 or 3 and if you dont get banned lower is 1 sec,dont go below 1.

next would be index-delay,i would leave it at 1.
finally channel-delay,i would leave it at 1 also.

Bernard
Offline
Donator
Joined: 6 months
Last seen: 3 months

I used another IP (from Germany) and it worked yesterday.

I used the following settings:
- retry channel-delay="1"
- index-delay="1"
- show-delay="2"

and the cron-job was again succesful this morning with the same (german) IP/locaction.

Changed it back to another european location and will have a look at it tomorrow.

Blackbear199
Offline
Blackbear199's picture
WG++ Team memberDonator
Joined: 9 years
Last seen: 10 min

thks,sent a msg to mat8861 to add delays to sky.de ini in siteini.pack.

Log in or register to post comments

Brought to you by Jan van Straaten

Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Supported by: servercare.nl