You are here

Using MDB to scrape Category/Genre

11 posts / 0 new
Last post
Mike_F
Offline
Has donated long time ago
Joined: 9 years
Last seen: 4 years
Using MDB to scrape Category/Genre

Hi,

Just a general question, when I grab my channels from tvguide.com, 90% or more of the programs come accross with the general category of "tvshow" or "movie". For post processing I am using tvdb.com, + imdb.com.imdb_series for series, and for movies I am using imdb.com.ask + imdb.com.imdb.

Is it possible for me to modify the ini files so that I can scrape the proper category during mdb post processsing rather than being stuck with the generic tvshow/movie tags I get when I grab the channel from tvguide.com? I have been looking at the ini file to try and figure this out on my own but I just don't see or understand exactly how to parse specific information from the websites.

Thanks guys,

WGMaker
Offline
WGMaker's picture
WG++ Team memberDonator
Joined: 12 years
Last seen: 7 hours
Is the support helpful?
support us

Hi, interesting question. Currently the MDB postprocessor doesn't support mdb_category as element. To add that requires two things:

1. The category data  must be present in IMDb or TheTvdb. I haven't checked on that. Did you? (what isn't there cannot be grabbed!)
2. The mdb target element mdb_category needs to be added to the list of supported elements of mdb and rex. This is not so difficult to do but only makes sense if the answer on 1. is 'yes' .

Can you have a look at this?
Jan

Mike_F
Offline
Has donated long time ago
Joined: 9 years
Last seen: 4 years

Hi Jan,

Thank you for the response. The answer to question 1 is yes. Both thetvdb.com and imdb.com list genre's which could entail multiple elements per show/movie.

To give an example, imdb.com lists the genre for "Saving Private Ryan" as Action, Drama and War while thetvdb.com lists the genre for "Breaking Bad" as Crime, Drama and Suspense.

 

WGMaker
Offline
WGMaker's picture
WG++ Team memberDonator
Joined: 12 years
Last seen: 7 hours
Is the support helpful?
support us

OK,

I will see what I can do.  I'll be back in a couple of days

Jan

 

WGMaker
Offline
WGMaker's picture
WG++ Team memberDonator
Joined: 12 years
Last seen: 7 hours
Is the support helpful?
support us

Hi Mike F

 

with the new upgrade 55, (including PostProcess V 1.9) you can grab mdb-category.

 

In mdb.config.xml you can add the mdb-category to the xmltv element <category> like this:
<category>'category'</category>
<category>'mdb-category'</category>​

This will list the 'existing' xmltv category first and will add the mdb-category (if found) after that.
If you don't want or need the existing xmltv category you can specify like this
<category></category>
<category>'mdb-category'</category>
​​
or simply this
<category>'mdb-category'</category>

 

You cannot add comments like <category>{mdbcategory: 'mdb-category'}​</category>

 

​The new mdbinis:
imdb.com.ask, imdb.com.bing, imdb.com.google and imdb.com.imdb can already be used to grab the mdb-category from IMDb.
I will update the others,  imdb.com.imdb_series, imdb.com.imdb_series.onscreen and ​tvdb.com as soon as I have the time for it.

 

Enjoy   Jan

WGMaker
Offline
WGMaker's picture
WG++ Team memberDonator
Joined: 12 years
Last seen: 7 hours
Is the support helpful?
support us

Hi,

The mdbini's for series are also ready to grab mdb-category

imdb.com.imdb_series, imdb.com.imdb_series.onscreen and tvdb.com

 

​Jan

Mike_F
Offline
Has donated long time ago
Joined: 9 years
Last seen: 4 years

Hi Jan,

Thanks a lot for implementing this for me this is exactly what I was looking for. There is only one issue. When I run this on Linux I keep getting the following error during the mdb post processing.
[  Error ] Unhandled exeption
[  Error ] Exception.Message: A null value was found where an object instance was required.

What is strange is when I run this on my Windows PC with a copy / paste of WebGrab++.config.xml and WebGrab++.config.xml as well as all of the associated ini files it works no problem.

I have experimented with rolling back the ini files to their previous versions and still get the error but when I run with the previous .exe file (even with the new ini files) I no longer get the above error.

Does any of this make sense to you? I can't imagine why it would work on my PC but not on my Linux machine.

Thanks for all the support, I really appreciate it!

francis
Offline
francis's picture
Has donated long time agoWG++ Team member
Joined: 12 years
Last seen: 8 months
Is the support helpful?
support us

Well,

The diff between the windows run an linux run, is that on windows, wg++ is run with .NET. On linux it is run with mono. And sadly those 2 are not 100% equal. So that will be the problem.

Could you run next debug .exe on both machines (especially on linux) and send the logs.

Attachments: 
Mike_F
Offline
Has donated long time ago
Joined: 9 years
Last seen: 4 years

Francis,

Attached are the logs you requested.

Thanks!

-Mike

francis
Offline
francis's picture
Has donated long time agoWG++ Team member
Joined: 12 years
Last seen: 8 months
Is the support helpful?
support us

Ok, we think we have found it.

Try this one

Attachments: 
Mike_F
Offline
Has donated long time ago
Joined: 9 years
Last seen: 4 years

Looks like that took care of it. Everything seems to be working fine now

 

Thanks!

Log in or register to post comments

Brought to you by Jan van Straaten

Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Supported by: servercare.nl