Transcript Incorporating Web Services in Mobile Applications April 3, 2009 – San Francisco Slides available at www.aduci.com/web2expo2009sf.ppt.

Incorporating
Web Services in
Mobile
Applications
April 3, 2009 – San Francisco
Slides available at
www.aduci.com/web2expo2009sf.ppt
Disclaimer
Web services and APIs of third parties referenced in this presentation
are owned by the third parties and governed by their agreements.
Before using any of the APIs or code referenced in this presentation
review the terms of use by the third party providing the API.
These slides are available under the Creative Commons AttributionNoncommercial-Share Alike 3.0 License. You can blog, photograph,
and share this information with others.
Trademarks and brands are the property of their respective
owners.
Introduction - Convergence
‣ Available APIs are growing daily
- Currently 1229 (and growing) APIs registered at Programmable Web
‣ Shifts in Consumer Expectations
- Consumers now expect mobile devices to be connected to Internet
Sites and Services
‣ Platform Development becoming easier and more standardized
- iPhone SDK – Greater than 50,000 registered developers
- Android – 5% of smart phone web usage in just three months1
‣ Seeing extension of enterprise applications to mobile devices
- Salesforce.com
1 Techcrunch, March 24, 2009, Erick Schonfeld
Introduction
‣ Many Web Services and APIs were originally developed with server
to server or server to browser in mind, not mobile applications
‣ Mobile platforms have their own set of challenges given:
- Bandwidth
- Memory and CPU Availability
- Storage Capacity
- Connectivity Options and Issues
- Security
- User Interaction and Display
Web Service
Mobile
Integration
Challenges
API Developer Programs
‣ Is mobile access allowed?
“5.1.5. Unless we have provided you our express written
consent in advance, you are not permitted to use Amazon
Associates Web Service in connection with any handheld,
mobile or mobile phone application.”
From the Amazon Web ServicesTM Customer Agreement
http://aws.amazon.com/agreement/
‣ Other considerations:
‣ Call Limitations (# per second, total per hour/day, pricing above)
‣ Caching and Storage of Data
‣ Persistence of Data – Length of time stored
‣ Freshness of Data – Length of time before refreshing
Caching and Storage Limits
‣ Persistence Limitations
-You may otherwise store Amazon Properties generally (other than
pricing or availability information) for caching purposes for up to 24
hours. However, you may store the following Amazon Properties for
caching purposes for up to 1 month:…
From the Amazon Web Services User Agreement
‣ Refreshing Requirements
You must refresh and re-display any Amazon Properties (other than
pricing or availability information) at least once every 24 hours or
once every month, as applicable, by making a call to AWS and
refreshing your Application’s contents immediately after the call.
From the Amazon Web Services User Agreement
Application Verification
‣ Certification of your Mobile Application
“If any of your Facebook Platform Applications are client-resident
(including on a mobile device), you agree to furnish a copy of such
Facebook Platform Applications and any supporting documentation
upon request for the purpose of verifying your compliance with this
Agreement; and …”
From the Facebook Developer Terms of Service
‣ Some API sets require certification on top of distributor certification
‣ Determine Costs / Timeframe / Effort given a mobile app
‣ Prepare for testing – typically looking for error handling, API abuse
‣ Certification is a positive!
‣ Credibility, validation, marketing
‣ Keeps the neighborhood safe
Mobile
Techniques
Authentication vs. Authorization
‣ The Difference
Authentication from the API provider – API Key
Authorization from the user – authToken
‣ Session Key
- By providing combination of API Key and authToken can then
receive the session key
- What is the shelf life of the session key
‣ Authorization will commonly
affect user’s experience on
mobile
- eBay authentication and
authorization screens
Last.fm - Sample Flow
‣ http://www.lastfm.com/api/mobileauth
‣ Get an API key
‣ Request auth – auth.getMobileSession
- Inputs:
- username
- authToken (md5(username + md5(password)))
- api_key
- api_sig
‣ Returns session key
‣ Make API call with sk (session key)
‣ Note: Try to keep credentials in one area to facilitate later changes
Last.fm auth.getmobilesession
‣ auth.getmobilesession
http://ws.audioscrobbler.com/2.0/?method=auth.getmobilesession&api_key=be8c2
…&username=chudsonfm&authToken=4eb71…&api_sig=e93fb…
‣ Result with new session key
<lfm status="ok”>
<session>
<name>chudsonfm</name>
<key>142c0085138a4b72d1782215b1da770a</key
>
<subscriber>0</subscriber>
</session>
</lfm>
username (Required) : The last.fm username.
authToken (Required) : A 32-byte ASCII
hexadecimal MD5 hash of the user's Last.fm
credentials.
api_key (Required) : A Last.fm API key.
api_sig (Required) : A Last.fm method signature.
Last.fm – Call Execution
‣ Now we can make the call with our sk (session key) – user.getinfo
http://ws.audioscrobbler.com/2.0/?method=user.getinfo&api_key=be8c2…&sk=142c0…&api_sig=ccbd1
…cc9dd
‣ Response
<lfm status="ok”>
<user>
<id>17317105</id>
<name>chudsonfm</name>
<realname/>
<url>http://www.last.fm/user/chudsonfm</url
>
<image/>
<lang>en</lang>
<country/>
<age/>
<gender>n</gender>
<subscriber>0</subscriber>
<playcount>0</playcount>
<playlists>0</playlists>
</user>
</lfm>
sk (Required) : The session key returned by auth.getMobileSession
service.
api_key (Required) : Your 32-character API key.
api_sig (Required) : Your API method signature
Speed Traps
‣ Could be on wifi, 3G or 1G based signal and bandwidth
‣ Segment functionality / calls to prevent issues
- Standard Apple test for approval is bandwidth usage when on cell
network
‣ Polling
- Create subset of calls and functionality when on slower bandwidth
- Use notifications if possible
- Modify polling interval based on need
‣ Recovery from intermittent or lost connections
‣ Test the use of APIs via public vendor API test tools on various
bandwidths
Large Data Set Handling
‣ Decreasing returns for XML with large data sets and repetitive XML
sections
‣ Even more the case with mobile devices given:
- Communication challenges
- Memory and processing limitations
‣ Minimize large data sets:
- Use Just-in-Time (JIT) methodology
- Ask for only those elements that you require
- Ask for only those items that you require
- Store what you can locally instead of requesting the same data
Response Processing
‣ Leverage Stream Parsing for mobile environments
- Provides low memory overhead
- But, requires more heavy lifting
‣ iPhone SDK based on stream parsing – NSXMLParser class
‣ Android has Xstream as the streaming parser
Response
• Minimize to just
what is needed
Parse Stream
• Stream Element
Based
• Sub parsing for
specific items
Display / Store
• Cache what
will be reused
(file, db,
memory)
iPhone Parsing
‣ NSXMLParser delegate methods
- 5 key methods
Method
Purpose
parser:didStartElement:
Start of element tag occurred with attributes
parser:foundCharacters:
Element data located which may be all or some
parser:didEndElement:
End of element tag located
parser:didEndDocument:
End of XML document occurred
parser:parseErrorOccurred:
Unrecoverable parsing error occurred
- Parser:didStartElement:
- Parser:foundCharacters:
- Parser:didEndElement:
- Parser:didEndDocument:
- Parser:parseErrorOccurred:
‣ Typical didStartElement, foundCharacters, and didEndElement
Parsing
‣ In long parsing streams provide feedback
when available
‣ Example: Control4 My House
- Home control from your iTouch/iPhone
- During initial project load describes the
sections being processed
- Stores those items that will not change
including house devices
- But when providing current data,
requests from server current state
- Blends stored information with real time
status information to provide display
Filtering
- Types of Request Filtering
- Selection criteria for narrowing data set returned
- In general determine how efficient are the calls – reduce data waste
- How much extra data, streaming, parsing is happening to access
the data you use? How much data is being dropped on the floor?
<?xml version="1.0" encoding="utf-8"?>
<FindItemsAdvancedRequest xmlns="urn:ebay:apis:eBLBaseComponents">
<!-- Standard Input Fields -->
<MessageID> string </MessageID>
<!-- Call-specific Input Fields -->
<BidCountMax> int </BidCountMax>
<BidCountMin> int </BidCountMin>
<CategoryHistogramMaxChildren> int </CategoryHistogramMaxChildren>
<CategoryHistogramMaxParents> int </CategoryHistogramMaxParents>
<CategoryID> string </CategoryID>
<CharityID> int </CharityID>
<Condition> ItemConditionCodeType </Condition>
<Currency> CurrencyCodeType </Currency>
…
<ShippingLocation> CountryCodeType </ShippingLocation>
<ShippingPostalCode> string </ShippingPostalCode>
<SortOrder> SortOrderCodeType </SortOrder>
<StoreName> string </StoreName>
<StoreSearch> StoreSearchCodeType </StoreSearch>
</FindItemsAdvancedRequest>
Paging
‣ Provide paging functionality to users
‣ Not all calls have paging available
‣ Inform user of current location in set, easy access to next and
previous if applicable
‣ Cache key information depending on
- The user flow (detail page from results page)
- The aging of the information (when does it expire)
Page Requests
Specify the page that you are requesting
<PageNumber>
‣ How many items are on a page
- Set the number of entries per page returned
<MaxEntries>
‣ http://developer.ebay.com/products/overview/
<?xml version="1.0" encoding="utf-8"?>
<FindItemsAdvancedRequest
xmlns="urn:ebay:apis:eBLBaseComponents">
<PageNumber>2</PageNumber>
<QueryKeywords>iphone</QueryKeywords>
<MaxEntries>20</MaxEntries>
</FindItemsAdvancedRequest>
List Paging Example
‣ Must cache of course the
current page
‣ Remember though the
total pages can change
dynamically so check total
pages after subsequent
requests
<TotalItems> int </TotalItems>
<TotalPages> int </TotalPages>
<?xml version="1.0" encoding="utf-8"?>
<FindItemsAdvancedResponse
xmlns="urn:ebay:apis:eBLBaseComponents">
<Timestamp>2007-0706T00:51:05.250Z</Timestamp>
<Ack>Success</Ack>
<Build>e525_core_APILW_4863605_R1</Build>
<SearchResult>
<ItemArray>
<Item>
<ItemID>123000268897</ItemID>
<EndTime>2009-0723T09:26:21.000Z</EndTime>
…
</Item>
<Item>
<ItemID>123000268898</ItemID>
<EndTime>2009-0724T09:26:21.000Z</EndTime>
…
</Item>
….
</ItemArray>
</SearchResult>
<PageNumber>2</PageNumber>
<TotalPages>6</TotalPages>
<TotalItems>103</TotalItems>
…
</FindItemsAdvancedResponse>
Display
‣ Caching of information for future retrieval and navigation
- Be careful of data aging depending on what is being shown
‣ Execute web service parsing and display in background when
possible to prevent UI blocking
- example – eBay item description and images
‣ When parsing large data blocks, progress indications can be provided
from element name (sections of the XML)
‣ Synchronous versus Asynchronous calls
‣ Use JIT and Information on Demand to maximize small layouts and
minimize web service calls
Caching Options
‣ To minimize API round trips leverage caching and storage
- Cache in memory
- Cache in onboard light weight database or file
- Off device persistence
Memory Cache
Onboard
• Memory Intensive
• Non-persistent
• Eg. Data key for
last viewed item
• Space Limitations
• Persistence
across sessions
• Eg. User
Information
Off Device
• Requires
Connection
• Reuse across
users
• Eg. Statistics,
look up tables
Expanding Your
Options
Blending Multiple Results
‣ What if you have to:
- Blend results from multiple calls
in the same API
Mobile View
Mobile App
API Service
Trigger
Request 1
- Blend results from multiple API
sources
Fulfill
‣ This could take the form of:
Result 1
- Extra item information on top of
item details call
Request 2
Fulfill
Result 2
‣ Note this is simplified so does not
show authentication and
authorization checking, encoding,
or the like.
Blend
Outlet
Blending Multiple Services
‣ Multiple Services
- Dependent on multiple
services
Mobile View
Mobile App
Trigger
API Service
1
API Service
2
Request 1
‣ Example:
Fulfill
- Weather request from
additional WS based
on zip code returned
by latitude and
longitude from a
geoposition service
Result 1
Request 2
Fulfill
Result 2
Blend
Outlet
Proxy Server Solution
‣ Provides layer of abstraction to consolidate or manage web services
‣ Proxy Server retrieves data on demand or on scheduled basis
‣ Allows for single consolidated call from the mobile device
‣ Processes data and returns subset or stores data for retrieval later
•
•
•
•
•
Benefits
Challenges
Reduces round trips
Custom Filtering
Custom Business Logic
Off device storage
Cache duplicate requests
• Another layer to maintain
• Data aging
• Data synchronization
‣ Can also reduce API calls by caching similar results across user
‣ Allows for extending business logic layer
Proxy Flow
‣ Provides for single
round trip for mobile
application
Mobile View
Mobile App
Trigger
Proxy
Server
API
Services
Request
‣ Provides easy
maintenance of web
services business logic
layer
Request
Fulfill
‣ Easily modify
Result
Request
- Business logic
Fulfill
- API credentials
Result
Result
Outlet
Proxy Landscape
Web Service
Web Service
Proxy Server
Designing your
WS/API Destiny
Creating Web Services
‣ Try to serve lowest common device
‣ Balance flexibility with overhead in calls:
- Filtering of criteria and paging of data functionality
- Split out high traffic calls versus critical requests
- Easy but secure authentication and authorization for both the
consumer and application
‣ Benefit for mobile consumers, application programmers and web
service providers
- Decreased round trips
- Increased efficiency of calls and applications
- Better use of call volume restrictions
- User experience improves
Wrap Up
Provide user preference options that translate into better tuned API calls
- number of items to show on a page as an example
This will translate to data efficiency, fewer callers and greater use of
data retrieved
Minimize the number of trips, combine in single calls when possible
Leverage caching and data storage when allowed
Balance data granularity with length of returns for parsing
Final thought to ponder
The mobile device that is typically serving as a web service client
could in the now or future be a web service provider…
Thank you
Slides available at:
www.aduci.com/web2expo2009sf.ppt
Chuck Hudson
[email protected]
aduci
Appendix A - Resources
‣ API / Web Service Online Catalogues
- Programmable Web
- www.programmableweb.com/api
- Web Service List
- www.webservicelist.com
Appendix B – Code Resources
-
-
iPhone URL Connections
http://developer.apple.com/documentation/Cocoa/Conceptual/URLLoadingSystem
/Tasks/UsingNSURLConnection.html
iPhone Rest Wrapper
ihttp://github.com/akosma/iphonerestwrapper/tree/master
iPhone XML Parsing – Cocoa Stream Parser
http://developer.apple.com/documentation/Cocoa/Conceptual/XMLParsing/XMLPa
rsing.html
SOAP with iPhone examples
http://icodeblog.com/2008/11/03/iphone-programming-tutorial-intro-to-soap-webservices/
http://developer.omniture.com/node/321
Façade Server landscape – O’REILLY – Windows platform
http://www.ondotnet.com/pub/a/dotnet/2004/02/23/mobilewebserviceapps.html
Connecting to Web Services – Sean Sullivan
http://www.slideshare.net/sullis/connecting-to-web-services-on-android
W3C – Mobile Web Application Best Practices
http://www.w3.org/TR/2008/WD-mwabp-20081222/
Appendix C - Efficient XML Interchange (EXI)
‣ W3C Working Draft 19 September 2008 http://www.w3.org/TR/2008/WD-exi-20080919/
‣ “EXI is a very compact representation for the Extensible Markup
Language (XML) Information Set that is intended to simultaneously
optimize performance and the utilization of computational resources.”
‣ Components of XML Complexity
1. size
2. total number of elements
3. number of unique elements
4. tree-height (nesting)
5. number of unique external references
6. total number of attributes
7. number of unique attributes
8. regularity, or "self-similarity"
Appendix D - Other Auth Examples
‣ Facebook
- Notice the session key expiration
-
http://wiki.developers.facebook.com/index.php/Authorizing_Applications
<?xml version="1.0" encoding="UTF-8"?>
<auth_getSession_response xmlns="http://api.facebook.com/1.0/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://api.facebook.com/1.0/
http://api.facebook.com/1.0/facebook.xsd">
<session_key>5f34e11bfb97c762e439e6a5-8055</session_key>
<uid>8055</uid>
<expires>1243814400</expires>
</auth_getSession_response>
‣ Oauth
‣ MPOAuthMobile – compiled in project library - Karl Adam
- http://code.google.com/p/mpoauthconnection/wiki/MPOAuthMobile