With the development of technology, more and more people are using smart phones nowadays. According to International Data Corporation, total shipments for smart phones in 2010 were 302.6 million units up 74.4 percent from 2009. This makes smart phones 21.8 percent of all handsets shipped (International Data Corporation, 2011)[1]. The capability of smart phones has also been greatly expanded with better hardware and advanced mobile operating system.
There are currently two most prevalent mobile operating systems in the market: iOS from Apple Inc. and Android from Google Inc.. With iOS leading in terms of both market share and the number of applications available, Android is getting closer and closer right after iOS because of its open source nature. The reason why these two mobile operating systems are so popular is mainly because there are tens of thousands of applications developed for them which enable users to do all kinds of things.
One particularly kind of applications attracts our attention. These applications enable smart phones to decode the information contained in Quick Response (QR) codes by scanning these codes using phone cameras. Quick Response code is a kind of barcode, except that it encodes information in two dimensions and it is becoming more and more popular. With the decoded information, these applications can do online searches for relevant information. Some available applications are Barcode Scanner and Google Goggles. However, these applications are only for general searching purpose, which means they simply search the internet and return everything they find. Due to the huge amount of information out there on the internet, it could be really hard for end-users to determine which information is reliable and credible. Therefore, it would be great to set up a system in which content-providers can provide reliable content and end-users can fetch credible information, thus enable information sharing between content-providers and end-users. Assume that a visitor is at a museum for an art exhibition. Under each artwork, there would be a short description of the artwork. What if the visitor wants to know more about the artwork? Of course he/she can do an online search if he/she has a smart phone in hand. However, there would be tons of information returned by a search engine. It would be very hard to determine which one is the best source. This problem can be solved by using the application developed in this project. Using our application, the visitor can scan a QR code under an artwork. The application will decode and fetch the corresponding information from a web server; it could be a video introduction of the artwork for example. The video will be downloaded into the visitor's smart phone and he/she will be able to play the video again at his/her leisure. Besides that, the visitor can also rate and comment on the video so that other users of our application will have better understandings. Also, if the visitor would like to be notified if a new video has been uploaded by content-providers in the future, he/she can subscribe to content-providers.
The project is to develop an application for the Android mobile operating system and the corresponding web application which in combine will enable information sharing between end-users and content-providers. The supported information/content type is mostly multimedia content, like video, audio, picture, text and URL link. By using the Android SDK provided by Google Inc., the Android application is developed using Eclipse Integrated Development Environment (IDE) and tested both on Android virtual machine and on an Android smart phone, HTC Hero. Even though HTC Hero is using Android version 2.1, the developed application should be able to work for all versions of Android. The backend server and the web application are also developed using Eclipse IDE. They are hosted on the Google App Engine which basically provides an infrastructure to host online applications and servers. In order to use the application, end-users need to have access to internet, whether it is over Wi-Fi or over cellular network.
End-users can use the Android application to retrieve reliable information by scanning quick response codes. The application will retrieve the corresponding multimedia information/content using information contained in quick response codes and show this multimedia information/content on end-users' Android smart phone. The application is also capable of manage the content that end-users download to their Android smart phones, like allowing end-users to browse content at later time and share content with their friends.
The main structure of the system on the end-users' side is demonstrated in Figure 1.
Figure 1 System structure on end-users' side
Content-providers can use the web application to upload, manage multimedia content and create the corresponding quick response code. The main structure of the system on content-providers' side is demonstrated in Figure 2.
Figure 2 System structure on content-providers' side
Since all content is provided by dedicated content-providers, a reliable and credible information sharing system can be set up.
This project is a group project which is done by Gao Qi and Sim Pei Ying. While Pei Ying is responsible for implementing the QR code decoding and content management functions in the end-user side system, the content-provider side system and the information/content fetching function in the end-user side system are designed and implemented by Gao Qi.
The paper is divided in five chapters. In detail I will proceed as follows:
Chapter 2:
Chapter 3:
Chapter 4:
CHAPTER 2
BACKGROUND
Android
Android is a software stack for mobile devices. It includes an operating system, middleware and applications. Android offers an open development environment built on a open source Linux kernel. Although Apple's iOS and Windows Mobile now provide a simplified and richer development environment for mobile applications, they still restrict communication between third parties applications and native applications. The native applications are given higher priority in run time. Android on the other hand, treats all applications equally. All applications have hardware access through a series of API libraries. Native and third party applications are written using the same API and are executed on the same run time. Android phone users are able to remove native application and replace it with third party applications. For example, the dialer and home screen of the phone can be replaced.
Google describes Android as:
"The first truly open and comprehensive platform for mobile devices, all of the software to run a mobile phone but without the proprietary obstacles that have hindered mobile innovation." [The Official Google blog, 2007]
Android is made up of several parts. Firstly is a hardware reference design. This design describes the capabilities and specifications needed for a mobile device to support the software stack. By having this design, Android can run on many different mobile devices. For the low level interface with the hardware, memory management and process control, Android uses a modified version of the Linux operating system kernel. There are some open source libraries for application development, which includes SQLite, WebKit, OpenGL and a media manager. Another part of the Android is a run time system to execute and host Android applications. This system includes the Dalvik virtual machine and the core libraries for Android written in Java. The executable files in Dalvik run with minimal memory footprint thus making the run time system efficient for use on mobile devices. Besides that, there is also an application framework that enables developers to access the hardware devices and other native applications in the mobile device. The services and systems include a set of Views, Content Providers, Resource Manager, Notification Manager and Activity Manager. The Android also contains preinstalled applications as part of the stack. Lastly, Android has a rich development environment including the tools and plug-ins used to create applications.
All native applications are written in Java using the Android SDK. The date stored and used by these applications such as contact details are also available to third party applications. Third party applications can also handle events like SMS message and incoming calls.
Introducing the Development Framework
Later in this the development framework will be explained in detail starting with the technical explanation of the Android software stack, what is included in the Android SDK and an introduction to the Android libraries.
Applications in Android run in separate processes in its own Dalvik instance. The run time system manages the memory used and processes running. It will stop and kill processes as it sees necessary to manage the limited resources in the device. Dalvik relies on the Linux kernel to handle low-level hardware interaction including drivers and memory management. A set of APU provides developers with access to all the services, features and hardware in Android.
Android SDK
Android software development kit (SDK) includes:
The Android API
The API provides the developers access to the Android stack. Google uses the same API for developing the native applications.
Development Tools
The tools create a development environment for developers to create applications. This lets you compile and debug your applications.
Android Emulator
Using the emulator, developers can run their applications on it. The developers can see how the applications will behave on an actual Android device.
Full Documentation
The SDK provides reference information on each package and class and how to use them. The documentation also explain how to get started and the fundamentals behind Android development.
Sample Code
The SDK also includes a selection of sample applications that demonstrates how to use some of the main API.
Understanding the Android Software Stack
The figure below shows the components in the stack.
Figure 3: System architecture
Linux Kernel
Core system services such as security, memory management, process management, network stack and driver model are handled by Linux version 2.6. The kernel also provides an abstraction layer between the hardware and the remainder of the stack.
Libraries
Android includes various C/C++ core libraries such as:
System C library
Media libraries that supports playback and recording of audio and video media, as well as static image files. Formats supported include MPEG4, H.264, MP3, AAC, AMR, JPG and PNG
Surface manager that provides display management
Graphics libraries that include SGL and OpenGL ES 1.0 APIs for 2D and 3D graphics
SQLite for lightweight native database engine for all applications
SSL and WebKit for integrated web browser and Internet security
Android Run Time
What makes an Android phone an Android phone rather than a mobile Linux implementation is the Android run time. Android includes the core libraries for most of the functionality in the core libraries of the Java programming language. As mentioned earlier, every Android application runs in its own process with an instance of the Dalvik virtual machine. The device can run multiple virtual machines. The executes files are run in Dalvik Executable (.dev) format which uses minimal memory footprint.
Dalvik Virtual Machine
Dalvik is a register-based virtual machine that's been optimized to ensure that a device can run multiple instances efficiently. It relies on the Linux kernel for threading and low-level memory management.
Application Framework
The application framework provides the classes used to create Android applications. It also provides a generic abstraction for hardware access and manages the user interface and application resources.
Application Layer
All applications, both native and third party, are built on the application layer using the same API libraries. The application layer runs within the Android run time using the classes and services made available from the application framework.
Android Application Architecture
Android's architecture encourages the concept of component reuse, allowing you to publish and share activities, services, and data with other applications with access managed by the security restrictions you put in place.
The same mechanism that lets you produce a replacement contact manager or phone dialer can let you expose your application components to let other developers create new UI front ends and functionality extensions, or otherwise build on them.
The following application services are the architectural cornerstones of all Android applications, providing the framework you'll be using for your own software:
Activity Manager
The building blocks of the user interface. It controls the life cycle of your activities, including management of the activity stack.
Views
These are used to construct the applications including list, text box and buttons.
Notification Manager
This enables an application to display the custom messages to the users.
Content Providers
Lets your applications share data between applications such as contact details.
Resource Manager
Provide the applications to access non-code resources such as string, layout and colours.
Android Libraries
Android offers a number of APIs for developing applications. The following list of core APIs that was used for this project:
android.util The core utility package that provides common utility methods such as date and time manipulation, containers, string formatters, and XML parsing utilities.
android.graphics The graphics API that supports canvases, colors, and drawing primitives. You can draw canvases using this package.
android.text The text processing tools for displaying and parsing text.
android.content The content API is used to manage data access and publishing by providing services for dealing with resources, content providers, and packages.
android.view This package is to manage the screen layout and interaction with the users.
android.widget Contains all the UI elements used such list, buttons.
android.app A high-level package that provides access to the application model. The application package includes the Activity and Service APIs that form the basis for all Android applications.
Advanced Android Libraries
The core libraries provide all the functionality needed to start creating applications for Android, but it won't be long before you're ready to delve into the advanced APIs that offer the really exciting functionality.
Android hopes to target a wide range of mobile hardware, so be aware that the suitability and implementation of the following APIs will vary depending on the device upon which they are implemented.
android.media The media APIs provide support for playback and recording of audio and video media files, including streamed media.
android.hardware Where available, the hardware API exposes sensor hardware including the camera, accelerometer, and compass sensors
Access to Hardware including
Camera
Android includes API libraries to simplify development involving the device hardware. These ensure that you don't need to create specific implementations of your software for different devices, so you can create Android applications that work as expected on any device that supports the Android software stack.
Shared Data and Interapplication Communication
Android includes three techniques for transmitting information from your applications for use elsewhere:
Notifications are the standard ways in which a mobile device traditionally alerts users. Using the API, you can trigger audible alerts, cause vibration, and fl ash the device's LED, as well as control status bar notification icons.
Intents provide a mechanism for message passing within and between applications. Using Intents, you can broadcast a desired action (such as dialing the phone or editing a contact) system-wide for other applications to handle.
Content Providers are a way to give managed access to your application's private database. The data stores for native applications, such as the Contact Manager, are exposed as Content Providers so you can create your own applications that read or modify these data stores.
Extensive Media Support
Android also offers comprehensive libraries for handling still images, video, and audio fi les including the MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and GIF formats.
QR code
Quick Response (QR) code is a two dimensional code readable by dedicated QR code readers. The code consists of black modules arranged in a square pattern on a white background. The information encoded are text, usually URL.
Figure 4: QR code
QR code was created by Toyota subsidiary Denso-Wave in 1994. It is very common in Japan and South Korea. Probably every phone in Japan and South Korea can read the QR code. It is called the Quick Response code as the creator intended the code to be decoded at high speed. [Denso Wave]
Figure 5: QR code in detail. QR code contains the positioning of the code, so a QR code can be read even if it is upside down.
QR codes were initially used for tracking parts in vehicle manufacturing when it was created by Denso Wave. But now, the codes are used in both commercial tracking application and convenience- oriented applications. It is known commonly as mobile tagging. QR code are used to display text to users, to add vCard contact to the user's device, to open a URI or to compose an email or text message. Using free QR code generating sites, users can also generate their own QR code and send it to others. QR codes appear in magazines, busses, on sign, business card or any other places where users might need information. The act of linking from a physical object (QR code) is known as hardlink or physical world hyperlink.
QR code is gradually gaining popularity due to the growing market of smartphones. Google Android powered mobile supports QR codes by natively including the Zxing Barcode Scanner on some models. Nokia's Symbian operating system is also providing a barcode scanner [Nokia, 2007] while mbarcode [Maemo] is a QR code reader for Maemo operating system. In Apple iOs, a QR code reader is not natively included but users can download free Apps that can read QR codes. HP/Palm's webOS users can also get a app named QR deCODEr to read these QR codes [Palm].
Storage
QR code data capacity [Denso Wave]
Numeric code only Max 7,089 characters
Alphanumeric Max 4,296 characters
Binary (8 bits) Max 2,953 bytes
Kanji/Kana Max 1,817 characters
CHAPTER THREE
LITERATURE REVIEW
Zxing Barcode Scanner
Zxing, pronounced as "zebra crossing", is an open source, multi-format 1D/2D barcode image processing library implemented in Java. This Android app uses the built-in camera on a smartphone to photograch and decode barcodes on the device, without communicating with a server. The barcodes supported by the app are UPC-A and UPC-E , EAN-8 and EAN-13, Code 39, Code 128, QR Code, Data Matrix ('alpha' quality), PDF 417 ('alpha' quality) and ITF.
From the Zxing Barcode Scanner website [Zxing], the library is divided into several components; some are actively supported:
core: The core image decoding library, and test code
javase: J2SE-specific client code
android: Android client, called Barcode Scanner
androidtest: Android test app
android-integration: Supports integration with our Barcode Scanner app via Intent
zxingorg: The source behind zxing.org/w
zxing.appspot.com: The source behind our web-based barcode generator
Some modules are contributed and/or intermittently maintained:
javame: JavaME client
csharp: Partial C# port
cpp: Partial C++ port
rim: RIM/Blackberry-specific client build
iphone: iPhone client + port to Objective C / C++ (QR code only)
bug: Client for BugLabs's BUG
Barcode Contents
A 2D barcode encodes text. The text can represent many things. Usually, the 2D barcodes encode a URL, like "http://google.com". This string is recognized as a URL by the reader and therefore the reader will open the URL in a browser. For the readers to understand the text encoded, Zxing developed some standards. Some of these standards are in fact already in use.
URL
To encode a URL, the protocol http:// must be included for the reader to recognise it as a URL. Example of the text to be encoded: "http://google.com/m".
The reader will then display the URL and ask the user whether to proceed with opening the URL in the device's web browser or not. This way, the user will be able to see the URL before the web browser accesses it.
The URL can also have a prefix of "URLTO:" to indicate that it is a URL.
E-mail address
To encode an e-mail address, the prefix mailto: must be included for the reader to recognise it as a e-mail address. Example of the text to be encoded: "mailto:[email protected]".
The readers should then open a blank e-mail message to the given address.
Telephone numbers
To encode a telephone number, the prefix "tel:" must be included for the reader to recognise it as a telephone number. Example of the text to be encoded: " tel:+6012345678".
The readers should then invoke the device's dialer and fill the number but will not initiate the call.
Contact information
To encode contact information, the vCard format is used. This format proves a bit verbose for use in 2D barcodes, whose information capacity is limited
The readers should then open a new address book entry with the entire fields filled and prompt the user to add a new contact.
AU format
Note that KDDI AU also proposes a slightly different format for this information.
BIZCARD
To encode BIZCARD, the prefix "BIZCARD:" should be included. For example BIZCARD:N:Alice;X:Tan;T:Software Engineer;C:NUS;A:25 Kent Ridge Road, Singapore;B:+6012345678;E:[email protected];;
vCard
vCard format has been used as well to encode contact information, though it is more verbose.
SMS
To encode sms, the prefix "sms:" must be included for the reader to recognise it as a sms. Example of the text to be encoded: "sms:hello".
Other prefix for sms are "sms:number:subject", "smsto:".
The readers should then open a new SMS message for the user to compose and send it.
MMS
Just like SMS, to encode mms, the prefix "mms:" must be included for the reader to recognise it as a sms. Example of the text to be encoded: "mms:hello".
Other prefix for mms are "mms:number:subject", "mmsto:".
The readers should then open a new MMS message for the user to compose and send it.
Geographic information
To encode geographic information, the prefix "geo:" must be included for the reader to recognise it as a sms. Example of the text to be encoded: "geo:40.71872,-73.98905,100" for 40.71872 deg N latitude, 73.98905 deg W longitude, at a point 100 meters above the office.
A reader might open a local mapping application like Google Maps to this location and zoom accordingly, or could open a link to this location on a mapping web site like Google Maps in the device's web browser.
Platform-specific
Android Market
To encode a hyperlink to an application in Android Market, the encoded text should be like:
market://search?q=pub:"Author" or
market://search?q=pub%3A%22Author
Likewise one can encode a link to a particular application by referencing its package,
market://search?q=pname:com.fyp.application
Wifi Network config (Android)
Scanning such a code would, after prompting the user, configure the device's wi-fi accordingly. The only client that implements this at the moment is for Android.
Example: WIFI:T:WPA;S:mynetwork;P:mypass;;
Parameter
Example
Description
T
WPA
Authentication type; can be WEP or WPA
S
mynetwork
Network SSID
P
mypass
Password
Unconfirmed, Unreleased, Possibilities
The Zxing team have suggested the following for their future works.
YouTube URI
To work on the iPhone at least, triggering the native YouTube player: "youtube://ID"
iCal
Used to encode calendar events. Readers could add events to the user's calendar in response.
They suggest using an abbreviated form that omits the VCALENDAR element:
BEGIN:VEVENT
SUMMARY:TestMeeting
DTSTART:20080811T190000Z
DTEND:20080811T200000Z
END:VEVENT
Chat status and presence
Another suggestion is a new URI syntax to communicate chat status updates, and to follow, for instance, a user in a service like Twitter:
pa:status:Enjoying a cup of coffee
pa:add:alice
pa:add:sip:[email protected]
Google Goggles
Google Goggles is an Augmented Reality (AR) app the uses the Android smartphone camera as input and search for relevant information related to the image captured. Google Goggles can recognize landmarks, books, contact information, artworks, wine and logos. All you have to do is snap a picture of it and the app will return information about the object, often, however, with very mixed results. Goggles can recognize text and bar codes. It can even translate the text. However, the search results are not always accurate.
The app can read QR or 2D bar codes and properly register them according to the content type it is. One particularly useful feature is the Goggles can scan business cards and directly save it to your contacts with the entire proper field filled. If you snap a picture of a book, you will arrive at a web page where Amazon offers the book for purchase. Goggles can also recognize many celebrities and famous people. Animals and ordinary objects generally do not match up well with the search results.
In summary, this app is just a search engine that takes images as its input. It might or might not return you the desired results you want. We want to deliver information closely related to an object so that users will not need to go through a big pool of information return by a search engine to find what he/she wants.
CHAPTER 4
TECHNICAL SECTION
Environment
Eclipse Integrated Development Environment (IDE) and Google Android SDK are downloaded and installed. The Android SDK provides the entire essential library needed and simulator for Android powered phone. Then Android Development Tools (ADT) for Eclipse IDE is added to prepare an integrated environment to build Android applications. Eclipse needs to know the location of the Google Android SDK, so set up the location on the SDK in Eclipse.
QR code scanner
Camera
The scanner uses the phone camera to take a preview image which will be used for decoding the QR code. The class android.hardware.Camera is used to manage the actual camera hardware. The table below shows the methods present in the Camera class with their corresponding descriptions.
Method Name
Description
Init
Initialises the camera.
openDriver
Opens the camera driver and initializes the hardware parameters.
closeDriver
Closes the camera driver.
startPreview
Asks the camera hardware to begin drawing preview frames to the screen.
stopPreview
Asks the camera hardware to stop drawing preview frames.
requestPreviewFrame
A single preview frame will be returned to the handler supplied.
requestAutoFocus
Asks the camera hardware to perform an autofocus.
Figure 6: Camera Manager class methods
Decoder
The Zxing Barcode Scanner library is used for the decoder. After getting a preview image of the QR code, the QR code will be located and save as a Bitmap, a 2D array of bits. A black module in the QR code represents "true" while the white module represents "false". The Bitmap will then be decoded by the decoder class in Zxing Barcode Scanner library and return the content of the code in the form of a string. The content types in Zxing Barcode Scanner library includes URL, e-mail address, telephone number, contact information, bizcard, sms, mms and geographical information. These content types have different prefixes. For example, a telephone number would be encoded as "tel:12345678", "tel:" being its prefix. The information provided by the content provider are indentified by a identification key. Thus, a new content type with the prefix "id:" was created for our application. Once the code is decoded, it will be sent to the server to retrieve the information corresponding to the code.
Share
The users can then have the option to share the QR code or the information retrieved via email or social network such at Facebook and Twitter. This is done with Intent.ACTION_SEND from android.content.Intent class. An intent is an abstract description of an operation to be performed. Below is a code snippet of the share function.
Intent intent = new Intent(Intent.ACTION_SEND);
intent.setType("text/plain");
intent.putExtra(Intent.EXTRA_TEXT, mContent);
startActivity(Intent.createChooser(intent,getText(R.string.button_share)));
"mContent" is the object that will be shared.
What the users will see is a pop-up dialog where they can choose how to send the object. Anything configured to support the ACTION_SEND Intent in their activity will be shown on the pop-up dialog. So, most users will get Mail, Gmail and SMS. But if the users installed Twidroid and Facebook on their phone, Twitter and Facebook will show up as well because both these applications support the ACTION_SEND protocol.
File manager
Files retrieved from the server will be downloaded to the path specified by the users. The files in Android have a specific set of metadata, which are file name, file size, permission to read or write and last modified time. To annotate more information to the file, a hidden text file is created for every file that is downloaded. The text file would contain the file name, author of the file, date file was uploaded, file type, file description, file size and the name, address, country and postal code of the place where the QR code for the corresponding file was placed. A file manager is therefore created for the end users to browse through the files downloaded, open the files and view the additional details of the files. The users can also create a new folder, delete files and rename the files with the file manager.
The main API used for the file manager is java.io.File. A "File" object is identified by a pathname. The pathname may be absolute, which is relative to the root directory of the file system, or relative to the current directory in which the program is running. The file referenced by a "File" may or may not exist. A folder might also be referred to as a "File". The class includes limited functionality for getting and setting the file permissions, file type and last modified name.
The table below shows the methods present in the File Manager class with their corresponding descriptions.
Method Name
Description
getDir
Gets a list of files starting from a path given.
viewDetails
Displays the file details on a dialog box.
openFile
Opens the file such as video, audio, text files.
createFile
Creates a file or folder.
renameFile
Renames a file or folder.
deleteFile
Deletes a file or folder.
refreshList
Refreshes the list of files.
Figure 7: File Manager class methods
