Osclass forums

Development => Plugins => Topic started by: code monkey on March 29, 2011, 12:33:41 pm

Title: Google XML Sitemaps
Post by: code monkey on March 29, 2011, 12:33:41 pm
Need something like the WordPress Plugin: http://wordpress.org/extend/plugins/google-sitemap-generator/

An example XML sitemap that was generated: http://bit.ly/fJN6xX

Must include:
- loc
- lastmod
- changefreq (always|hourly|daily|weekly|monthly|yearly)
- priority (1.0 to 0.0)

The actual XML looks like this:
Code: [Select]
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="sitemap.xsl"?>
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://domain.com/</loc>
<lastmod>2011-03-28T12:00:00+00:00</lastmod>
<changefreq>daily/changefreq>
<priority>1.0</priority>
</url>
</urlset>

URL's to include:
- home (always or hourly)
- items (list URL in each language; daily)
- categories (list URL in each language; if pagination in use - list each page; hourly)
- country (hourly)
- country/region (hourly)
- country/region/city (hourly)
- pages (yearly)

Possible improvement: create table to track search phrases.
When the phrase has been searched more than __ times, add the link to search results in sitemap.

Finally, ping search engines like Google, Bing, Yahoo, Ask.com to notify them of your updated sitemap.
Title: Re: Google XML Sitemaps
Post by: code monkey on April 02, 2011, 07:43:29 am
Having difficulty,I could use some help.

1. If I make this a plugin, how can I run the sitemap generate file - is there a plugin hook in hourly/daily cron?

2. How can I generate a list of items I can loop through, that will allow me to use the item helpers?

3. How to generate item URL for each locale?
Title: Re: Google XML Sitemaps
Post by: Juan Ramón on April 04, 2011, 01:36:00 pm
Having difficulty,I could use some help.

1. If I make this a plugin, how can I run the sitemap generate file - is there a plugin hook in hourly/daily cron?

2. How can I generate a list of items I can loop through, that will allow me to use the item helpers?

3. How to generate item URL for each locale?

Code: [Select]
View::newInstance()->_exportVariableToView('items', Item::newInstance()->listLatest( [i]num[/i] ) ) ;
However, in the core we should add the system to ping google, yahoo and the other search engines.
Title: Re: Google XML Sitemaps
Post by: _CONEJO on April 04, 2011, 06:48:56 pm
1. If I make this a plugin, how can I run the sitemap generate file - is there a plugin hook in hourly/daily cron?
I created hooks for cron, (cron, cron_hourly, cron_daily and cron_weekly) there're on develop branch

2. How can I generate a list of items I can loop through, that will allow me to use the item helpers?
As Juan Ramón said :
Code: [Select]
View::newInstance()->_exportVariableToView('items', Item::newInstance()->listLatest( $number_of_lastest_items_you_want ) ) ;We have another function to retrieve ALL items, but it could generate problem with memory (if you have a huge database with a lot of items, the function is : Item::newInstance()->listAllWithCategories();

3. How to generate item URL for each locale?
It's not possible now, we could do that, I will check the URL's and make that possible.


We could work this out together, I will work on the URL's, tell me if you need anything else.

Thanks!!
Title: Re: Google XML Sitemaps
Post by: _CONEJO on April 04, 2011, 08:19:52 pm
I did that http://issues.osclass.org/browse/OSCLASS-464

You could pass a locale as an argument of osc_item_url() and osc_static_page_url()

For example osc_item_url('es_ES'); or osc_static_page_url('en_US');
Title: Re: Google XML Sitemaps
Post by: _CONEJO on April 05, 2011, 08:42:29 pm
Hi

I've uploaded a previous version of the Sitemap Generator : https://github.com/osclass/osclass-plugins/tree/develop/sitemap_generator

I didn't tested it yet, but I think it's ok.

What do you think?
Title: Re: Google XML Sitemaps
Post by: code monkey on April 06, 2011, 04:47:04 am
Hey, looks great! See: http://bit.ly/h1YsGo

Couple changes:

1. In sitemap_add_url(), I had to add htmlentities() around $url so that & is converted into &amp; - or XML says there is an error.

2. Categories weren't generating. I added:     

osc_categories_search();
osc_goto_first_category() ;

...before...

if (osc_count_categories() > 0) {

...and now categories are being generated.

3. Add <priority></priority>. Can be 1.0 (highest) to 0.0.


Thanks  :D
Title: Re: Google XML Sitemaps
Post by: _CONEJO on April 06, 2011, 11:09:01 am
Great! I will update it.

I didn't added priority since I'm not sure which is more important. Maybe latests item? How many? 10? 100? 200?
Anything else will get a 0.5 which is the default value for priority (even if not specified)
Title: Re: Google XML Sitemaps
Post by: code monkey on April 06, 2011, 07:38:44 pm
I put...

home - 1.0
items - 0.9
categories - parent: 0.8, sub: 0.7
city: 0.6, region: 0.5, country: 0.4
pages: 0.3

...because I want users to land on the posts. I guess the first 100 or so seems right for 0.9. Really I don't know how it works, as long as items is higher than categories/regions/pages.
Title: Re: Google XML Sitemaps
Post by: kingsult on May 05, 2011, 07:17:50 am
My sitemap was refused by Google... I have uninstalled the plugin. I have seen that Chrome does not show if a feed is not correct.

Did we validate the feeds using the w3c validators ?

Thx
Title: Re: Google XML Sitemaps
Post by: _CONEJO on May 05, 2011, 11:18:37 am
Which was the reason to reject it?
Title: Re: Google XML Sitemaps
Post by: kingsult on May 06, 2011, 05:30:52 pm
The fields are not valid... here is an example of the cars feed:

URL: http://www.sxm-market.com/index.php?page=search&sCategory=2&sFeed=google_cars

Total File Size: 8.62 KB
Detected file format: XML
Recent uploads

May 5, 2011 7:07 pm PDT - 0 of 5 items inserted
May 4, 2011 7:07 pm PDT - 0 of 5 items inserted
May 3, 2011 7:07 pm PDT - 0 of 5 items inserted
May 3, 2011 6:26 pm PDT - 0 of 5 items inserted
Feed Level Messages

 Error: XML formatting error
Our system encountered an error when processing your data feed.
Learn more.
Examples:
Line Nr.   Column Nr.
76   152
Item Errors - 6 items with errors

3
 Invalid location value in attribute: location
Please include a street, city, state, postal code, and country, in that order, separated by commas. Also make sure you are able to validate your locations through Google Maps.
Examples:
Item Nr.   Line Nr.   Value
1   6   , Cay Hill, Sint Maarten, St Maarten - St-Martin
2   22   91 Billy Folly rd @Rancho cielo estates, Simpson Bay, Sint Maarten, St Maarten - St-Martin
3   45   , Cay Hill, Sint Maarten, St Maarten - St-Martin
1
 Missing required attribute: model
Please make sure you include all required attributes in your data feed.
Examples:
Item Nr.   Line Nr.
4   59
1
 Missing required attribute: year
Please make sure you include all required attributes in your data feed.
Examples:
Item Nr.   Line Nr.
4   59
1
 Missing required attribute: make
Please make sure you include all required attributes in your data feed.
Examples:
Item Nr.   Line Nr.
4   59
Item Warnings - 9 items with warnings

4
 Unrecognized attribute: publish date
Your feed contains an attribute name our system doesn't recognize. Please verify your attribute name as necessary, and please be aware that this attribute may no longer be included in our Feed Specification.
Learn more.
Examples:
Item Nr.   Line Nr.
1   14
2   37
3   51
4   66
1
 Missing recommended attribute: mileage
While items missing recommended attributes will process successfully, we recommend including relevant attributes if they are available.
1
 Missing recommended attribute: vehicle type
While items missing recommended attributes will process successfully, we recommend including relevant attributes if they are available.
1
 Missing recommended attribute: image link
While items missing recommended attributes will process successfully, we recommend including relevant attributes if they are available.
1
 Missing recommended attribute: vin
While items missing recommended attributes will process successfully, we recommend including relevant attributes if they are available.
1
 Missing recommended attribute: color
While items missing recommended attributes will process successfully, we recommend including relevant attributes if they are available.
Title: Re: Google XML Sitemaps
Post by: _CONEJO on May 06, 2011, 05:34:45 pm
Last question, Is out there a google feed validator or something?
Thanks
Title: Re: Google XML Sitemaps
Post by: kingsult on May 06, 2011, 09:02:08 pm
You need to submit and get their feedback. I just googled this one: http://www.celebird.com/ckbase.htm

I haven't tried it yet... :-(
Title: Re: Google XML Sitemaps
Post by: kingsult on May 10, 2011, 08:00:52 pm
Hi CONEJO,

Any news on the RSS side of things ? ;-)

Kingsult
Title: Re: Google XML Sitemaps
Post by: _CONEJO on May 16, 2011, 10:37:33 am
Yes,

The plugin it's updated, but not yet released (this week!). For the google thing, the RSS is valid, the problem could be that some required fields are not filled correctly by the end-user (I have to check more carefully).

Thanks for waiting.