Transcript Blogging, Googling, Syndication: What’s the Web coming to?

blogging, googling, syndication: what’s the web coming to?
ian graham
Enterprise IT Strategy & BMO Connect
T: 416.513.5656
E: [email protected]
W: http://www.utoronto.ca/ian/talks/
“The race for integration”
 With thanks to Sherri Bondy for some example weblog sites.
2005 Professional Excellence Symposium
what this talk is about
• The culture of web applications
– What are the new applications about?
– How are they ‘designed’ and put together
– Where are they taking the ‘public’ Web (and us!)
• Examples (Lots)
– …that help to illustrate the above
• Some conclusions
What the talk is not about
– Banking/financial services, software development,
Web Services (tech details), lobsters, rodents.
2005 Professional Excellence Symposium
web culture
Concepts
Tools / Technologies
• Location
• Blogging
• Connectivity
• Syndication  RSS
• Discovery
• Google
• Communication &
Presence
• Jabber, ICQ
• Community
• Ebay, Yahoo, Slashdot,
Amazon, Blogspots, …

The right side lists just a few example tools &
technologies & platforms there are many others
…
2005 Professional Excellence Symposium
a) having a home: blogging
• Short for ‘Weblogging’
– A ‘log’ authored on and accessible over the Web
• Simple Web-site generation
– Personal (or group) publishing & communications:
• Users often publish regularly (daily, or weekly), and link
to friends / colleagues / coworkers
• Links establish virtual ‘communities’ with shared interests
• Categorization for topics, etc.
– Software is often free to download (and play with)
• Free (or near) hosting available (see list at end)
– Software often supports user-level scripting to building
custom functions, behaviors
2005 Professional Excellence Symposium
some example blogs
• http://www.sifry.com/alerts/archives/000298.html
– Blog site about blog sites (news, information)
•
•
•
•
•
•
[1]
http://dear_raed.blogspot.com/2003_02_01_dear_raed_archive.html [2]
– Reports from inside Iraq before and during 2003 war
http://blogs.msdn.com/dmassy/
– Internet Explorer development team member
http://www.nerdgirl.com
– A ‘normal’ person (purely social site)
http://weblogs.mozillazine.org/roadmap/
– Inventor of JavaScript – roadmap for development team
http://www.joelonsoftware.com/
– Articles on software development (professional)
http://www.richardcleaver.com/
– Not what you were
expecting !!
2005 Professional Excellence Symposium
(later)
uses of weblogs
• Social networking
– Post musings and share them with friends
• (Semi-) professional information sharing
– Writing for richer communication (e.g. news, current
affairs, professional commentary)
• Work-related networking
– Write status reports, decisions and analyses, etc, for
sharing / cross-linking with others.
• Popularity?
– There are circa 10 million blogsites
– (of course, most are started, and then abandoned!)
2005 Professional Excellence Symposium
“blogspace”
• The collection of blog sites all across the Internet
• Unconnected except
for
hand-created
2005
Professional
Excellence Symposiumlinks in the pages
b) two blogspace issues:
1.
Letting people know what you have
–
–
2.
Publishing information ‘about’ your blog postings
• An advertisement, or summary, about your daily postings.
• RSS -- An XML language for publishing summaries
• RSS stands for Resource Site Summary, or RDF Site Summary,
or Really Simple Syndication, or …) [some examples later ]
RSS is also used by regular web sites, news sites, etc.
Finding interesting/relevant stuff
–
–
Aggregation sites and desktop clients
• Aggregate RSS info ‘about’ published blogs / other sites
• Headliner clients, topic-specific Web sites
Categorization and rating systems
• Categorize blog items, Web page/sites
• separate wheat from chaff
2005 Professional Excellence Symposium
b.1) rss feeds from a blogsite





Time-stamped
‘summary’ about
a blog item
Metadata (for
cataloging,
indexing)
Headline text
A link back to
the original
… An ‘ad’ for the
original content.
<item>
<dc:creator>Dave Massy [MSFT] </dc:creator>
<title>Security, Security, Security</title>
<link>
http://blogs.msdn.com/dmassy/archive/
2004/06/20/160649.aspx
</link>
<pubDate>Sun, 20 Jun 2004 12:56:00 GMT
</pubDate>
<description>
<P>This topic seems to be coming up
<P> &nbsp;</P><img
src ="http://blogs.msdn.com/
dmassy/aggbug/160649.aspx"
width = "1" height = "1" /> …more text …
</description>
</item>
2005 Professional Excellence Symposium
E.g.: http://blogs.msdn.com/dmassy/
 RSS link [3]
b.2) aggregators & syndication
RSS
RSS
RSS
or
HTML
RSS
RSS
Retrieves and categorizes RSS from blog (and other) sites
And provides a categorized feed to users, or a Web listing:



http://www.oreillynet.com/meerkat/
an RSS feed aggregator
http://www.newzcrawler.com
an RSS desktop client
2005 Professional Excellence Symposium
http://aggregator.weblogs.co.uk/
a blog-RSS feed aggregator
[4]
[5]
rss: not just for bloggers
• RSS has been adopted by major media sites as a
tool for “syndicating” headlines
– http://www.cnn.com/services/rss/
– http://news.bbc.co.uk/
• Blogging has moved beyond ‘cool’
– http://news.bbc.co.uk/2/hi/programmes/click_online/439
8243.stm (political and cultural impacts)
2005 Professional Excellence Symposium
api’s for accessing aggregators….
• Programming interfaces to aggregation sites
– Simple, easy to use, and URL-based
– E.g. Meerkat API -• http://www.oreillynet.com/meerkat/
[6]
• http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html
(documentation)
– API based on arguments in the URL (above explains how
this works). For example:
• http://www.oreillynet.com/meerkat/?p=1065&_fl=rss10
• http://www.oreillynet.com/meerkat/?p=1065&_fl=rss
RSS feed of items in profile
category 1065
2005 Professional Excellence Symposium
[6.1]
Same, but use different
RSS version for feed
and embeddable in web pages:
( . . . demo.html)
[6.2]
<html>
<head>
<title> Example of RSS feed inclusion using
JavaScript </title>
</head>
<body>
<h1> Heading For the Example </h1>
<p> Here is some regular HTML -- ok,
i didn't have time to make it pretty </p>
<p> And here is the JavaScript example code:
<script language="JavaScript"
src="http://meerkat.oreillynet.com/?_fl=js">
</script>
Includes script sent by Meerkat; script
</body> </html>
generates HTML summaries and
2005 Professional Excellence Symposium
writes them out as HTML document
summary
• Blogging is for individuals or groups.
• RSS (and syndication) help build connectivity, and
communities
• Aggregators support simple APIs so it’s easy to build functions
that integrate these tools with others (blogsites, portals, other..)
– Richer communities via richer functionality
– No conscious high-level architecture: people just try new
ideas, and adopt what works (and drop what doesn’t)
– A heterogeneous collection of apps, connected using
simple protocols, APIs.
• Each application and protocol is simple (limited scope, easy to
use) and open (encourages reuse, extension)
Key idea: URL as the reference for everything
2005 Professional Excellence Symposium
c. googling
• Google -- A web search engine
– http://www.google.com
[7]
– Other stuff:
• blogger.com; google groups; news, special
search services; maps; gmail (email) …
• Some numbers:
– Estimated space
> 3,000,000 GB
– Number of servers
> 100,000 (Linux)
• Access to the index:
– Via web pages, and several APIs -- some simple,
some complex
2005 Professional Excellence Symposium
google as a service
• Via APIs that let you reuse the google engine as part
of your own application:
– A URL-based interface (like Meerkat)
• URL says where to go
• Stuff in URL says what you want back
– A full-blown Web services interface
• (yes, I know I promised not to mention web
services!)
– Both let you, essentially, use Google as the search
engine / database within your own applications
(subject to restrictions)
2005 Professional Excellence Symposium
Google’s URL-based API
( . . . demo-google.html) [7.1]
<form id="search" method="get"
action="http://www.google.com/custom">
<input type="hidden" name="cof"
value="LW:600;LH:58;L:http://www.bmo.com/images/
...banner.gif;GIMP:#cc0000;T:black;
...;AWFID:9262c37cefe23a86;">
<input type="hidden" name="domains"
value=“bmo.com">
<input type="hidden"
name="sitesearch" value=“bmo.com">
<input type="text" id="q" name="q"
accesskey="s" size="30">
<input type="submit" id="submit" value="go">
</form>
----- Info – How it Works ----------------------http://www.google.com/faq_freewebsearch.html
http://cyberzoide.developpez.com/html/google/
2005 Professional Excellence Symposium
google’s web services api
• Experimental interface, but widely used
– Requires registration for access key (simple, and
free)
– Provides library and tools for building simple
applications, using SOAP-based web services.
• Java and .NET examples provided
• API covers
– Ways to query the google database
– Response formats / error responses
2005 Professional Excellence Symposium
public class GoogleAPIDemo {
public static void main(String[] args) {
…
// Create a Google Search object, set our authorization key
GoogleSearch s = new GoogleSearch(); s.setKey(clientKey);
// do search, then print out result
try {
if (directive.equalsIgnoreCase("search")) {
s.setQueryString(directiveArg);
GoogleSearchResult r = s.doSearch();
System.out.println("Google Search Results:");
System.out.println(r.toString());
} else if (directive.equalsIgnoreCase("cached")) {
….
} else if (directive.equalsIgnoreCase("spell")) {
…
} else {
printUsageAndExit(); }
} catch (GoogleSearchFault f) {
System.out.println("The call to the Google Web APIs failed:");
System.out.println(f.toString()); } }
private static void printUsageAndExit() { … } }}
2005 Professional Excellence Symposium
Google Example
[7.2]
Google Summary
• Simple API focused on single function
– Querying the google database.
• Loose coupling
– Google doesn’t care what you’re doing.
• Coarse-grained
– Here’s a query, and there’s your answer. No
details beyond those needed for business
function.
• These two characteristics are core to almost all ‘Web’
development models.
2005 Professional Excellence Symposium
d. jabber: messaging & presence
Jabber clients
• Presence
• User directory
• Proxys to Yahoo, ICQ
• Other services
Jabber
server
Jabber
server
Jabber servers
2005 Professional Excellence Symposium
Jabber Example
Jabber client
[8.1]
Jabber
client
Jabber
client
Jabber
server
Jabber
server
• Requests and responses all
sent in XML
• Generic XML protocol for
• Connect
exchanging messages, plus
register presence
some services.
• Lookup user
• Can be extended to non-text
contact database
messaging applications
• Send text message
• Now used for virtual presence:
contact database
lets you know who else is at a
blog or other web site
2005 Professional Excellence Symposium
Virtual Presence
(Jabbber extension: http://www.lluna.de/ [8.2])
1.
2.
3.
1
WS
VP
server
4.
3
2
4
VP
client
5
5.
Web Server registers with
virtual presence (VP)
server, using a URL
VP client gets name of VP
server (URL) from Web
server.
VP client registers with VP
server.
VP server tells VP client
who else is there.
VP client provides chat
service to other users of the
site.
2005 Professional Excellence Symposium
Key features
• Simple, single-purpose
• URL-based referencing scheme
– Underlying protocol uses XML
• Loose coupling between components
– servers, clients – failure is expected
• Less-simple protocol than RSS …
– May explain why jabber (and family) isn’t
as popular as other technologies.
2005 Professional Excellence Symposium
And many other examples
• Bittorrent - file sharing / transfers
– http://www.bittorrent.com
[9]
• 30% of all Internet traffic is generated by BitTorrent
• Wikis -- collaborative web authoring
– http://en.wikipedia.org/wiki/Main_Page -- a
collaboratively edited ‘encyclopedia’
• Over 500,000 entries (English)
– http://en.wikipedia.org/wiki/Wiki
2005 Professional Excellence Symposium
[10]
An example application
• Web page (from web server) that includes data from
an aggregator,
– uses google to provide searching
– publishes new content info using RSS
– Uses jabber to provide presence functionality to
visitors
– Provides bittorrent links to downloadable data
• All integrated using URLs, and simple web protocols.
2005 Professional Excellence Symposium
Conclusions
• Social  focus on community, networking,
connection, sharing
• Technical  Software design focused on
– Simplicity of interfaces, and applications
– URL as the key reference for an application,
function, method
– Loose coupling between applications
• Construction of Web applications by integrating
these different ‘application services’ together.
• Nice essay on this at:
– http://www.smallpieces.com/ -- “Small pieces
loosely Joined”
2005 Professional Excellence Symposium
Conclusion II
“A large part of how we think about music is
influenced by the methods by with which it has
conventionally been distributed. We think of pop songs
as being three or four minutes long because 40 years
ago that was all that could fit on one side of a vinyl
single.”
Moby
 We
often think of Internet-based computing is the
same way -- in terms of what we know -- not what it is
becoming. It is different, and challenges our thinking!
 Couldn’t
be a more fun time to be doing software!
2005 Professional Excellence Symposium
--- The End --blogging, googling, syndication: what’s the web coming to?
ian graham
Enterprise IT Strategy & BMO Connect
T: 416.513.5656
E: [email protected]
W: http://www.utoronto.ca/ian/talks/
“The race for integration”
2005 Professional Excellence Symposium
Some Tools and Sites
•
•
•
•
Blogging software
–
http://www.moveabletype.com
–
http://www.livejournal.org/
–
http://manila.userland.com/
–
http://www.slashcode.com/
Free blogging sites
–
http://www.blogger.com  http://newrandom.blogspot.com (Ian’s site)
–
http://www.livejournal.com
RSS / Atom Aggregation sites
–
http://aggregator.weblogs.co.uk/
–
http://www.bloglines.com/
–
http://www.syndic8.com/
–
http://meerkat.oreilly.com (Open Wire Service)
• API Documentation: http://www.oreillynet.com/pub/a/rss/2000/05/09/meerkat_api.html
RSS / Atom Desktop clients
–
http://www.newzcrawler.com
–
http://www.headlineviewer.com
2005 Professional Excellence Symposium
Some Tools and Sites
•
•
•
•
•
Google tools and APIs
–
Google Hacks, Calishain and Dornfest, O’Reilly and Associates, 2003
• http://www.oreilly.com/catalog/googlehks/index.html
–
http://searchenginewatch.com/searchday/article.php/2161301
–
http://www.google.com/faq_freewebsearch.html (Free websearch)
–
http://www.google.com/services/custom.html (custom search)
–
http://cyberzoide.developpez.com/html/google/
–
http://www.google.ca/apis/ (Google Web service APIs)
Jabber
–
http://www.jabber.org
–
http://www.lluna.de (virtual presence extension)
Bittorrent
–
http://bitconjurer.org/BitTorrent/
Wikis
–
http://en.wikipedia.org/wiki/Wiki (all about Wikis)
Java Development Kit
–
http://java.sun.com/j2se/1.4.2/download.html
2005 Professional Excellence Symposium