Transcript Publishing on the GeoWeb Keith Golden April 03 2009 Copyright 2009 Google Inc.

Publishing on the GeoWeb
Keith Golden
April 03 2009
Copyright 2009 Google Inc.
The GeoWeb, 2 years ago
Copyright 2009 Google Inc.
The GeoWeb, today...
Copyright 2009 Google Inc.
The GeoWeb, today... unfiltered
Copyright 2009 Google Inc.
The GeoWeb
Hidden dangers in the Deep Web
DB-backed websites are great
AJAX is nifty
We love JS and Flash APIs
But how will people find your data?
Search engines don't index this
Copyright 2009 Google Inc.
How to get your geo data indexed
1. Commercial GIS server
• If possible, configure to export crawlable KML
o ArcGIS Server
o GeoServer
• If not, see #2
2. Your own custom database-backed website
• Use standards (KML, GeoRSS)
• Follow best practices
• Create geo sitemaps
3. In the cloud
• geocommons
• Google Maps Data API
• Other hosting options, see #2
Copyright 2009 Google Inc.
Use standards: KML or GeoRSS
Choose GeoRSS if you’re…
Already outputting RSS
Want to convey that “updates” or “most recent” items are
contained in the data
o Have relatively simple (e.g., point) data
o Example:
o
o
 http://gmapssamples.googlecode.com/svn/trunk/ggeoxml/geoRSSOverlay.html
Choose KML for all other cases.
KML is a standard that is optimized for transferring geospatial
information, but is flexible enough to transfer arbitrary, your-appspecific information
o Example:
o
 http://gmaps-samples.googlecode.com/svn/trunk/ggeoxml/kmlOverlay.html
 http://maps.google.com/maps?f=q&hl=en&q=http%3A%2F%2Fkml.lover.goo
glepages.com%2Fetna_ground_overlay.kml&ie=UTF8&z=10&om=1
Copyright 2009 Google Inc.
How-to: Use KML/GeoRSS in Your Mashup
Two general strategies.
• Directly
data
KML/
GeoRSS
mashup
sitemap
• As a Reference for indexing
?
mashup
KML/GeoRSS
sitemap
data
Copyright 2009 Google Inc.
Use KML/GeoRSS in Your Mashup Directly
• Pros:
o
Data already in KML/GeoRSS format, so one format serves
both mashup + sitemap.
• Cons:
o
o
KML/GeoRSS are heavyweight
JSON may be faster, work better with JS apps
data
KML/
GeoRSS
mashup
sitemap
Copyright 2009 Google Inc.
How-to: Use KML/GeoRSS Directly
In the Google Maps JS API
GGeoXML:
• Retieves KML or GeoRSS file (must be on a public server)
• Returns an Overlay
http://www.google.com/apis/maps/documentation/kmlOverlay.html
var map = new GMap2(document.getElementById("map"));
map.setCenter(new GLatLng(49.496675,-102.65625), 3);
var geoXml = new GGeoXml("http://nigel.tao.googlepages.com/blackbirds.kml");
map.addOverlay(geoXml);
Example: http://n.girasol.googlepages.com/kml.html
Using: http://n.girasol.googlepages.com/ginzasen.kml + 20 More KML files
Also: http://www.singeo.com.sg/singeo/maps/denguemap.html
Copyright 2009 Google Inc.
How-to: Use KML/GeoRSS Directly
(Custom parser)
These are XML markup languages, so you can easily:
• Use the API provided XML-wrapper functions (GDownloadUrl, GXml)
to get the XML elements of the KML or GeoRSS
• Parse the elements to find the ones you’re using (e.g. <Placemark />)
•
Display elements however you’d like
var xmlDoc = GXml.parse(doc)
var placemarks = xmlDoc.documentElement.getElementsByTagName("Placemark");
for (var i = 0; i < placemarks.length; i++) {
var name = GXml.value(placemarks[i].getElementsByTagName("name")[0]);
var desc = GXml.value(placemarks[i].getElementsByTagName("description")[0]);
var coords =
GXml.value(placemarks[i].getElementsByTagName("coordinates")[0]);
// Do whatever you’d like here
}
• Pro: Using your own parsing function is the most flexible solution
• Con: Takes time to write own
KML or GeoRSS parser.
Copyright 2009 Google Inc.
Use KML/GeoRSS as a Reference for SiteMaps
Pros:
• Can make independent decisions on best KML
for sitemap vs. optimal data format for mashup
intake.
Cons:
• Must maintain two data output formats - one for
your mashup and one for your sitemap.
?
mashup
KML/GeoRSS
sitemap
data
Copyright 2009 Google Inc.
How To: Use KML/GeoRSS as a Reference for SiteMaps
3 options, depending on your frequency of data
change:
• Set up a cron job at certain interval (e.g. weekly) that
produces KML file(s) at static URLs
• Point sitemap to script that outputs KML, so it’s generated
each time it’s crawled
• Produce the KML once, using whatever tools, and never
change
Copyright 2009 Google Inc.
Best Practices for KML/GeoRSS Creation
•
•
•
•
Give your document a meaningful name
Make sure the item description field is relevant
Use attribution (author and link)
Divide your data to topic/regionspecific files
• Put all your files in a sitemap
• Use robots.txt if needed
• More info:
http://code.google.com/apis/kml/documentation/kmlSearch.html
http://code.google.com/apis/maps/sitemap.html
Copyright 2009 Google Inc.
Publish Your Content in Geo Site Map
Allow webmasters to submit specific geo content
• Easier and faster discovery
• More reliable indexing
• Fresher data
Copyright 2009 Google Inc.
Sitemap Structure
We’ve added the <geo:geo> and <geo:format> elements.
You can refer to one file…
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns=http://www.sitemaps.org/schemas/sitemap/0.9
xmlns:geo="http://www.google.com/geo/schemas/sitemap/1.0">
<url>
<loc>http://www.example.com/example1.kml</loc>
<geo:geo>
<geo:format>kml</geo:format>
</geo:geo>
</url>
</urlset>
Copyright 2009 Google Inc.
Sitemap Structure
Or multiple files
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.example.com/example1.kml</loc>
<geo:geo>
<geo:format>kml</geo:format>
</geo:geo>
</url>
<url>
<loc>http://www.example.com/example2.rss</loc>
<geo:geo>
<geo:format>kml</geo:format>
</geo:geo>
</url>
<url>
<loc>http://www.example.com/example3.kmz</loc>
<geo:geo>
<geo:format>kmz</geo:format>
</geo:geo>
</url>
</urlset>
Copyright 2009 Google Inc.
Submitting a Sitemap to Google
• Submit the sitemap to Google Webmaster
Central http://www.google.com/webmasters/
• More info:
http://www.google.com/support/webmasters/bin/answ
er.py?answer=94554
• Full geo search documentation:
http://code.google.com/apis/kml/documentation/km
lSearch.html
Copyright 2009 Google Inc.
More KML Creation Tips
Create KML files which make sense in isolation.
• Put searchable keywords in <description> elements.
• Use document and folder <description> for keywords which apply
to sets of features.
Copyright 2009 Google Inc.
More KML Creation Tips
Keep the number of features per file reasonable.
• Hundreds are OK.
• Tens of thousands probably not.
• Use region-based network links for large datasets.
http://code.google.com/apis/kml/documentation/regions.html
Copyright 2009 Google Inc.
More Creation Tips - Use Attribution in KML
KML 2.2 supports new elements for including attribution
atom:author element - parent element for atom:name
atom:name element - the name of the author
atom:link element - contains the href attribute, the URL of
the web page containing the KML/KMZ file
<kml xmlns="http://earth.google.com/kml/2.2"
xmlns:atom="http://www.w3.org/2005/Atom">
<Document>
<atom:author>
<atom:name>J. K. Rowling</atom:name>
</atom:author>
<atom:link href="http://www.harrypotter.com" />
…
</kml>
Copyright 2009 Google Inc.
How-to: Use Attribution in GeoRSS
<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:georss="http://www.georss.org/georss"
xmlns="http://www.w3.org/2005/Atom">
<title>Best Breakfasts in NYC Feed</title>
<link href="http://foodCritic.com/feeds/notes_for" rel="self"/>
<id>http://foodCritic.com/notes_for</id>
<updated>2007-03-09T09:14:02Z</updated>
<generator uri="http://foodCritic.com">TheFoodCritic</generator>
<entry>
<title>the original Big Stacks, best pancakes this side of the
Mississippi</title>
<author>
<name>arnold</name>
</author>
<georss:point>40.776371 -73.978439</georss:point>
<published>2007-03-08T14:19:47Z</published>
<content type="html"> . . . </content>
</entry>
</feed>
Copyright 2009 Google Inc.
Publishing your data in the cloud
Data hosting & publishing options
• geocommons
• Google Maps Data API
Other data hosting options
• Geo-specific or generic
• Watch out for walled gardens
• Make your data crawlable if you want people to
find it
Copyright 2009 Google Inc.
geocommons
1. Finder
o Download or display a large selection of geo data
o Upload your own data (Creative Commons license)
o Convert between Shapefile, CSV, and KML
o Tag and describe data sets
o Search uploaded data
2. Maker
o Create maps to visualize uploaded data sets
o Display multiple layers to show correlations
o Slick, easy-to-use UI
 Plot individual points and lines
 Create thematic maps by color or size.
3. Enterprise s/w & services available
Copyright 2009 Google Inc.
geocommons
Copyright 2009 Google Inc.
Google Maps Data API
Google Data API = AtomPub
• XML/HTTP protocol for reading and writing to data feeds.
• http://code.google.com/apis/gdata/
Google Maps Data API offers
• Storage of geospatial data
• Point, polyline and polygon geometry, with more to come
• Create and edit My Maps
• Content optionally indexed and searchable in Google Maps
• Unlimited user-defined attributes
• Batch operations
• Incremental upload and download of changes
• Client libraries in a variety of languages
Copyright 2009 Google Inc.
Great, where do I sign up?
Currently in use by partners & early testers
Available to you today
• Contact me
• [email protected]
Will open to the general public soon
Copyright 2009 Google Inc.
Google Maps Data API
Copyright 2009 Google Inc.
Maps feed
http://maps.google.com/maps/feeds/maps/default/full
o GET: list my maps
o POST: add a new map
o PUT: edit the metadata of a map
o DELETE: remove a map
•
•
•
•
Set title and summary
Set public/unlisted (draft) status
Associate custom properties with the map
Access the feed of features in the map
Copyright 2009 Google Inc.
Accessing maps
Requests require authentication token in HTTP headers
• AuthSub, OAuth
• ClientLogin: service name is "local"
My maps:
http://maps.google.com/maps/feeds/maps/default/full
Someone else's (public) maps:
http://maps.google.com/maps/feeds/maps/[email protected]/full
Copyright 2009 Google Inc.
Features feed
http://maps.google.com/maps/feeds/features/...
o GET: list the contents of the map
o POST: add a new feature to the map
o PUT: edit a feature
o DELETE: remove a feature from a map
• Set the title
• Store a kml:Placemark
o Point, line or polygon geometry
o Embedded <Style> to control appearance
o HTML description
o Use atom:link and atom:author for custom attribution
• Associate custom properties with the feature
Copyright 2009 Google Inc.
Adding a feature to a map
HTTP POST an entry to the edit link of the feature feed
<atom:entry xmlns:atom='http://www.w3.org/2005/Atom'
xmlns:gact="http://schemas.google.com/maps/2008">
<atom:title>Random place of the day</atom:title>
<atom:content type='application/vnd.google-earth.kml+xml'>
<Placemark xmlns='http://www.opengis.net/kml/2.2'>
<atom:author>
<atom:name>John Q. Random</atom:name>
</atom:author>
<description> Some place </description>
<Style> ... </Style>
<Point>
<coordinates>
-90.869489,48.254501,0.0
</coordinates>
</Point>
</Placemark>
</atom:content>
</atom:entry>
Copyright 2009 Google Inc.
Controlling which content is indexed
Not all your content need be indexed.
<entry>
<category term="http://schemas.google.com/maps/2008#map"
scheme="http://schemas.google.com/g/2005#kind"/>
<app:control xmlns:app="http://www.w3.org/2007/app">
<app:draft>yes</app:draft>
</app:control>
<title>MyMapsPoly</title>
...
</entry>
You decide.
Copyright 2009 Google Inc.
Controlling attribution
By default
• Content is attributed to the account it was created under
• Links from search results display the content in Google
Maps
If you want different behavior, use KML attribution
• atom:author to specify the true author
• atom:link to link back to your site
Copyright 2009 Google Inc.
Adding your own custom properties
<gd:customProperty name='land cover'>
coniferous forest
</gd:customProperty>
<gd:customProperty unit='m' name='elevation'>
81
</gd:customProperty>
Copyright 2009 Google Inc.
Batch requests
<atom:feed xmlns:atom='http://www.w3.org/2005/Atom'
xmlns:batch='http://schemas.google.com/gdata/batch'>
<atom:entry>
<atom:title type='text'>batch insert 0-0</atom:title>
<atom:content type='application/vnd.google-earth.kml+xml'>
<Placemark xmlns='http://earth.google.com/kml/2.2'>
...
</Placemark>
</atom:content>
<batch:id>0-0</batch:id>
<batch:operation type='insert' />
</atom:entry>
<atom:entry> ... </atom:entry>
...
</atom:feed>
Copyright 2009 Google Inc.
Client libraries
No need to talk HTTP directly
Client libraries in Java, Javascript, C#.
Works with Zend PHP libraries
Use libkml for KML processing
• http://code.google.com/p/libkml/
Copyright 2009 Google Inc.
Java example
URL queryUrl =
new URL("http://maps.google.com/maps/feeds/features/"
+ MAPID + "/full");
GoogleService myService = new MapsService("exampleCo-exampleApp-1");
myService.setUserCredentials(USERNAME, PASSWORD);
FeatureFeed feed = myService.getFeed(queryUrl, FeatureFeed.class);
for (FeatureEntry entry : feed.getEntries()) {
entry.addCustomProperty(
new CustomProperty("annotation", null, null,
entry.getTitle()));
myService.update(new URL(entry.getEditLink().getHref()),
entry);
}
Copyright 2009 Google Inc.
Caveats
• This is a beta service
• Invitation only for now
o But you're all invited
o Talk to me, and I'll set you up
• There's more to come
o Search via the API
 by keyword
 by attribute
 by location
o Richer content
o Better Maps API integration
o More structure
o New feeds
Other geo data hosting options
If you just want data hosting for your own applications
• MapQuest Data Manager
• Microsoft MapPoint Web Service Customer Data Service
But also publish your data in KML/GeoRSS, and use a geo
sitemap to make sure it gets indexed
Both offer
• Bulk upload of data in CSV format
• Bulk geocoding of uploaded locations
• Custom fields for metadata filtering
• Web based GUI for managing data
• Location-based search
• Enterprise SLAs
Copyright 2009 Google Inc.
Conclusions
• If you want users to find your site, help them find your data
• Beware Deep Webs and Walled Gardens
• Use standards
o KML
o GeoRSS
• Use attribution
• Publish a Geo Sitemap
• Perhaps publish your data by storing it in the cloud
Contact me if you want to try out the Google Maps Data API:
[email protected]
Copyright 2009 Google Inc.
Questions?
Copyright 2009 Google Inc.