Hello
Firstly I am able to get EPG data from many supported sites so I know how to set up the application. TV Guide was/is working and I do see the warning about collecting EPG data from them as they do not like it when people do that. So I am trying other places to get this data. Zap2it does not seem to be supported, DirectTV does not work. I am trying to get a complete EPG for the US channels with some locals. What site should I use? None seem to have the channel selection that I need. Thoughts?
I am using SchedulesDirect for US and Canada channels, it has a lot of channels. But "oh my god" it is slow. I have around 30 channels, and it took 4 hours to fetch EPG for a single day. Way too long.
Can it be that you overlooked this in the instructions? :
A.
Follow the instructions at schedulesdirect.org
!! Important when chosing a lineup: Keep the number of channels low !! Only add channels that you are really going to use! Keep in mind that also for the channels you do not use to get an epg from, the data is downloaded. That takes time and slows down the process.
That is really very important !!
Jan
I hope you don't misunderstant this .. you must reduce the channels in the SD- linup to only what you want to grab with WG++. Only reducing the number of channels in the WG++ config has no effect!
Jan
Thanks or the information. I tried Schedules Direct and I only have 2 channels in the list to get data from. It is painfully slow. Not like the other ones at all.
Forgive me that I don't follow you.:
You have 30 channel in your config but only 2 in your line-up ?? That doesn't make sense.
You should have the same channels in your SD lineup (and nothing more) as in the WG++ config.
Jan
That was a different person who responded to my initial question. He is collecting data from 30 channels.
I am testing out Schedules Direct and I have only 2 channels in my config file. I get the EPG data just fine but it's really slow. I am just collecting one day at this point.
Thanks
Hi beach40,
Could you upload your setup?
Maybe remove your SD credentials first, because those are private to you.
Francis
What do you mean by my set-up? The config file or the line-up on Schedules Direct?
config, log and ini file that you use actualy : rules & guidelines
Here are 2 of them... The EPG collection is still happenning now so I cannot send you the log file just yet. I did check it and there are no errors.
One thing... When I entered my credentials in the .ini file I had to take the asteric out from in the front of that line so I would not get an authentication error.
I know about the lineup, and I did set it up correctly. 30 channels in the lineup and 30 in the config.
The only thing is that I had to choose two different providers at SD, one US and one from Canada. So I acually have two lineups, one for each provider.
But still takes around 4 hours... and the CPU is trough the roof.
Upgraded today with the latest version of webgrab+.
Ran it with only 2 channels from SD, it was done after 15 min. This cannot be normal.
Added the log.
Sorry, but it is normal.
SD has 2 API's. We currently only support the first one. And a couple of weeks/months ago, I think they have switched this API to a mutch slower server.
That's why it is taking so long.
And currently we are not able to support the second (hopefully faster API)
About the speed. I don't have to repeat that it is essential to not have more channels in your lineup than in the config!
I tested the speed again and here it came to slightly les than 1 second/show. That is slow but not dramatic to my opninion. A regular site comes at about .5 second/show.
Jan
I have around 150 channels in total in my entire WebGrab++.config.xml file, 40 of them are from SchedulesDirect, the rest is SkyUK and tvprofil.com.
My lineup at SC is also excatly 40 channels. This morning the grabber started at 06:00 CET, and finished at 12:40 CET! And my CPU is at 100% while this is running. This cannot be normal at all.
What is also anoying is that SC adds new channels to the lineups, so you have to visit the site often to remove the new channels.
Ok,
Previous weekend I dove into this issue. And I think I came up with a faster version.
The actual problem with SD, is you need to parse the whole file to find the current show info. And if that is not enough, you have to do it 3 times.
My setup had only about 50 channels and the file for 16 days, was already 9MB. And if you know it is 9MB of text, you know we are speaking about a lot of characters.
To make a long story short, I've tweaked the sources a little bit, added a new feature (global_temp_) and rewrote the siteini.
The setup I used, was done on a 16 day/50 channel index page. And the scrub was done for 1 day for 2 channels.
This results in the next numbers:
WG++ V55 ini rev. 1: 672 sec.
WG++ V56.6 ini rev. 1: 287 sec.
WG++ V56.6 ini rev. 2: 59 sec.
So if you update to the latest SD ini (rev. 2) and the latest beta (V56.6), it should go faster.
1 Remark, you can't run SD ini (rev. 2) with older version of WG++
Hope this help you a bit.
Please share your results/findings.
Nice! Good job :)
I ran a quick test just now, with only a few SC channels, and noticed straight away that it did run a lot faster, 5-6 sec per channel.
I'm going to let it run with the full file tomorrow morning and report back.
This deserves another donation :)
Thank you!
How do I know the patch applied correctly? I do a right click and select properties (Windows) and I does not show any changes???
When you start webgrab++ you should see in the command-window what version it's running. Or in the .log file.
Allright, here is my status regarding the last run. Had to run it in the afternoon as it failed this morning due to some minor errors in some of my ini files.
Job started at 29.10.2015 11:35:00
6498 shows in 146 channels
40 updated shows
2954 new shows added
Job finished at 29.10.2015 11:56:53 done in 1314 seconds
20 min in total! :) Amazing improvment. Good job Francis!
Hello I am running version 1.56.6 with the latest schedules direct .ini I am testing 3 channels for a 3 day time frame and it runs for 10 minutes.
Thanks
@beach40: I recomend that you check your lineup at SC. Every time they add new channels they add them to your lineup. So you should go there and clean up once in a while.