|
|
|
|
|
|
|
|
|
|
|
|
|
|
This paper describes the architecture of the Badge, its distributed computing environment, and presents initial results of application development trials conducted by a class of Telecommunications students at KTH.
Keywords: badge, location dependent, location aware, sensor fusion, location tracking, mobile computing, prototype, user trials.
We provide environment awareness using an artefact we call a Badge. Badges have evolved culturally to make access control and authentication easier. We believe that computer and sensor augmented badges can also act as user interaction devices providing a convenient human-centred way for people to interact with the ever expanding computing and communications environment they find themselves immersed in. Most importantly a Badge provides the last link in the computing and communications environment allowing the "system" to know where the user is and what the environment is like at that location rather than just the network addresses of the user's computing and communication peripherals. The Badge matches the user, the device and the application together, and in a sense creates an interface dynamically that is centered about the human's identity, capabilities and environment. The Badge makes devices instantly useful, even ones the user has never encountered before.
There have been a number of steps towards environment awareness. However, to date most systems have only been location aware, for example: Active BadgeTM work at the Olivetti Oracle Cambridge Research Lab, Ubiquitous Computing Project at XEROX PARC, Adaptive Operating System Services and Applications in Mobile Computing Environments at Japan Advanced Institute of Science and Technology, Walkstation II project at KTH Teleinformatics, especially [Liu96a, Liu96b, Liu96c], Dataman project at Rutgers (particularly Location-dependent Information Browsing), InfoPad at U.C. Berkeley, and Things That Blink, MIT Media Lab.
The first generation mobile systems tried to overcome the effects of mobility leading to developments in areas such as low-power sleep modes and file systems for disconnected operation. The second-generation mobile systems tried to exploit some knowledge of the user's location resulting in mobile map and navigation systems and location enhances 911 emergency services. Environment awareness is an enabling factor for the next generation of mobile computing systems. Environment awareness takes mobile computing beyond simple location awareness, by provides the system not only with details of its current location, but also environmental factors such as temperature, humidity, ambient light level, and orientation. This information can be used to help the computer system interact with its environment and to help the environment better cater to the people using it. In short, environmental awareness helps systems move towards human centered behaviours.
To provide environmental awareness we need:
To be viable a badge must be small, light and convenient to wear. A computationally and environmentally augmented badge has some additional features and requirements that contribute to its design:
Power consumption must be low so the Badge can be worn for long periods without the inconvenience of changing the batteries. The desire for low power consumption also dictates that Badge transmissions should be relatively fast as transmission is the main use of power in the system.
Alternate technologies are available including ultrasound, near-field and radio frequency transceivers and RFID tags. However, none of these systems offers the combination of transmission range, speed and volume production available from infrared technology.
We have constructed a Dumb Badge from a PIC microcontroller and IrDA compliant infrared circuits. The badge uses IrLAP format message framing to be compatible with the rest of the emerging IrDA infrastructure. Our Dumb Badge transmits its fixed 32-bit identity number and an 8-bit status byte every 2.3 seconds. The status byte contains a low-battery indication. A frame consists of 22 bytes (including IrLAP framing overhead) giving the device a duty-cycle of 0.3%, a measured operational range of 4m and a 15 degree half angle. The Badge data frame encapsulated in the IrLAP frame is designed to allow easy extensibility simplifying the construction of Badge Servers to interpret and act upon the received data. Dumb Badge data is not encrypted.
The I/O port allows the Smart Badge to control its environment. Typically, the port would be used to interface to an attached computing device, telephone, or GPS system. It can also be used to control light, doors, air conditioning, heating, ventilation, etc.
Figures 1 shows Photographs of the prototype Smart Badge (the prototype Badge measures 54x103mm with version 2 measuring 54x86mm, the same size as a standard corporate identity card). The front and rear plastic panels have been removed to show the components. The white cylinders in the rear view (Figure 1) are four tilt switches in quadrature providing the orientation sensor of the badge. The Humidity and temperature sensors are to the bottom left of the Badge. The microphone is on a fly-lead as we are currently experimenting with the best location for it. An outward-facing microphone is useful to determine ambient noise and recognise when the wearer is speaking. An inward-facing microphone could be used as a heart rate and respiration sensor. The front of the badge (Figure 1) contains the IrDA transceiver, piezoelectric transducer and light sensor (bottom right corner). A cosmetic faceplate attaches to the front of the badge to provide a photo of the wearer, a name, identity number and bar code can also be provided for backward compatibility.
The Smart Badge prototype supports limited reprogramability. It allows the direction of the I/O pins to be changed to suit a particular application. It also allows the Smart Badge identity numbers to be changed and the badge encryption keys to be changed. While it is possible to provide full reprogramabilty securing such a system is currently problematic and has been left for future study.
Smart Badge data is encrypted. The encryption is needed to:
As an example, first consider the case of a single persona. When powered on a Smart Badge has a default identity and encryption key. The Smart Badge configuration software transmits a Smart Badge command to the new badge telling it to change its identity to a new identity specified in the command. The command is encrypted with the default key and the operation is assumed to take place in a trusted environment. Once the Smart Badge identity has been changed the Smart Badge configuration software sends another command telling the badge to change its encryption key to the randomly chosen key contained in the command. The identity and key are recorded in the Badge Server database and associated with the Smart Badge wearer. The badge wearer now has secure, private, communication to the Badge Server without needing to enter, remember, or remembering to change encryption keys.
Security is maintained by changing the Badge identity and encryption key randomly as the badge moves through its environment. Potentially each time the badge communicates with a device the Badge key is changed, preventing anybody with an IrDA snifter from capturing and re-transmitting the Badge communication to gain unauthorised access.
To be able to effectively decrypt the badge messages at the Badge Server the current Badge identity must be sent unencrypted. It is used as an index to the database to find the encryption key (and its predecessor). As badge commands are unacknowledged the previous Badge identity and Badge key must be kept in the database until a message is received from the badge using the new identity or key. Until this happens the Badge Server continues to send identity or key change commands to the Badge in the hope that one of them will get through and effect the required change.
While changing the Badge identity and Badge key randomly will defeat many forms of attack it is still possible to learn the identity and habits of a Badge wearer by monitoring Badge packets on the Internet connecting the Badge transceivers to the Badge Server. A simple example is that a Badge message is seen going from the IP address of a particular door transceiver to the Badge Server at about the same time each day and that a particular wearer comes back from coffee at that time each day. Simple correlation allows you to deduce the wearers current Badge identity. Changes to that identity can then be tracked through the Internet using snifters to follow the wearer. To overcome this problem we propose to apply a second layer of symmetric encryption between the Badge Server and the Badge transceivers. Thus while a snifter may be able capture the Badge message from our returning coffee drinker, it still has to decrypt it before it can determine the Badge identity. If Badge transceivers use different keys and change them using the same random rolling technique that is used to change Badge keys we believe the resulting system has sufficient strength to protect the privacy of individual Badge wearers. The general cryptographic hierarchy of Badge transmissions is illustrated in Figure 2.
Our prototype Smart Badge currently implements encryption using the Tiny Encryption Algorithm (TEA)
[Wheeler94]
with a claimed strength similar to 128 bit DES. TEA was chosen as it
trades space off against time and hence will fit in the Smart Badge's limited
RAM (currently only 196 bytes). We don't believe the lengthened encryption
time will impact on Badge performance as badges send messages very infrequently
(for power reasons if nothing else).
The Intelligent Badge aims to be a platform to present a device's user interface to the user in a customisable human-centred way. A simple example is a photocopier, when the user approaches the copier the copier's agent notices the co-location of the wearer and the copier, it then arranges for the copier's user interface to be presented on the wearers Intelligent Badge. The actual format of the interface is determined by the wearer, thus allowing a consistent interface (language, colour, modalities, etc.) to be constructed to all devices. The photocopier's interface could equally well be presented as desktop icons, animated characters, or speech prompts - depending on the badge user's preferences. An Intelligent Badge acting as an I/O platform allows the wearer to chose the modality and frees the device designer of the need to chose how to present the interface to the user. All the designer needs to do is present the in a structured way to the network using techniques for telecommunications and distributed processing work (e.g., CORBA).
Four of the PIC's eight analogue inputs are utilised to sample sensor data. The sensors are operated to reduce the power consumption of the Badge. Sensors derive their power from output pins on the PIC. Sensors are turned on, given some time to stabilise, their value is then sampled and the sensor is turned off to reduce power consumption. The Tilt Sensors are simple switches and are connected directly to digital inputs on the PIC with weak internal pull-ups. All the sensors are samples once per cycle of the Badge program with the exception of the audio sensor.
The audio sensor is turned on and sampled 256 times over 0.5 seconds. The 8-bit sound samples are averaged over the 0.5-second period. This allows the Badge to report the average sound level. A streaming communication between the badge and the data network is needed before continuously sampled audio is viable.
The Smart Badge also has provision for a daughter card providing an RS232 interface (DECT and Ethernet can also be supported). This daughter card allows the Dumb Badge to become an IrDA gateway to the (fixed) infrastructure. This provides a simple interface to local area networks, desktop PCs and mobile IrDA equipment (laptops, notebooks, PDAs, scanners, printers, etc.). Note that for many laptops, notebooks, PDAs, etc. the IrDA interface ports tend to face away from the user. The Dumb Badge also operates with existing IrDA to LAN gateway devices like the HP NetbeamIR. Interoperation with the existing data networking infrastructure allows badge communication to easily become ubiquitous, as no new cabling is needed.
Smart Badge activity is sequenced. First the sensors are polled (0.5 seconds), then a frame is transmitted (0.008 seconds) then the badge waits for received frames (1 second) and finally the Badge enters low power sleep mode (2.3 seconds). The Badge cannot receive and transmit at the same time (IrDA is inherently half-duplex) and ignores all external stimuli during sleep mode. The receive phase is currently set to 1 second. This duration was determined empirically be experiments with a directly connected Badge Server. Because of the limited badge receive time window it is important that frames destined for a Badge are buffered at the Badge Transceiver and transmitted as soon as a frame has been received from the badge. If the frame was sent directly from the Badge Server across the Internet to the Badge through the Badge Transceiver is it unlikely that frames would ever be received because of Internet latency and variance in that latency.
We are experimenting with multicast techniques to collect and distribute Badge data to and from the Badge servers. Thus depending on the multicast address which a given Badge uses and where the servers are located, it is possible to provide use across organisational boundaries (i.e., via firewall traversal) as servers can be behind or in front of firewalls, but you have to have the correct rights to access the data (as per the above comments on security). Note that access to data can be restricted based not only on who you are, but also on when you ask, who else is with you, and the conditions of your environment at the present time. It might even be possible, if you have physiological monitoring to tell if the requestor is under duress, thus prevent extorting information from individuals.
Multicast also allows messages to be sent to moving Badges by the Badge Server. Instead of just sending a message to the last Badge Transceiver that saw the Badge of interest, the Badge Server multicasts the message to a group of geographically adjacent Badge Transceivers. Thus, the Badge will receive the message as soon as it announces itself to any of the Badge Transceivers in its neighbourhood.
The current prototype implements the Badge Transceiver server on a PC connected to a Badge acting as a transceiver. The Badge Server is a WWW server plus CGI scripts. We expect to replace the PC Transceiver software with NetbeamIR devices providing direct IrDA to Ethernet interconnection. To effectively use the commercial NetbeamIR devices and communicate directly with IrDA compatible desktop and mobile computers the Badge must first implement a more extensive subset of IrLAP and the IrLMP link management protocol so the Badge can respond to IrDA poll and sniff messages. Currently the Badge frame is encapsulated in an IrLAP unit-data broadcast frame. These frames are interpreted as commands by the NetbeamIR and not passed across the gateway to the Ethernet.
This type of application has already been shown with Active BadgesTM and has proved to be useful. A further feature which Active BadgesTM development showed necessary is that the door agent must react quickly to the presence of the person near the door, as users did not like to wait long periods of time before the door opened. In the case of the Active Badge, they added another sensor, which detected a RF field, and they used the detection of a change in the field to trigger an immediate emission of the Badge ID. Thus different fields were used to encode spatial zones; transition from one zone to another zone caused an immediate report form the Badge. We believe that our Badges represent a significant generalisation of this idea, as shown in the examples that follow.
Note that this correlation of information can be used to compensate for variations in the individual sensors. Thus the sensors need not be calibrated beforehand, but can have the correction data derived over time. Of course, if we want to have absolute measurements then we need to have some calibrated sensors that we then calibrate the new sensors with.
If the room agent analyses the Badge server data for spatio-temporal correlations it can identify and hence predict the normal patterns of a person (and hence persons) and anticipate their arrival, departure, and routine movements. (see earlier work on predicting user movement, such as: [Liu96a, Liu96b, Liu96c] and Mobile Service Network Mobility Models [Banh95]). If this technology is applied to all buildings, then both the building management agents and the Power Company's agent should be able to predict load fluctuations and micromanage their power demands, generators and power grids.
If an Intelligent Badge is used, a person can also interact with the building system and control lights, doors, temperature etc. from a Badge based interface. This provides an improvement over the simple motion detectors used in many offices to turn lights off - if there is no activity in the office. In such environments it is often difficult for the user to get the behaviour that they want, for example to have the lights off even if there are people in the room or to keep them on when the people in the room are stationary. It also could greatly simplify the wiring necessary, since if each power outlet is network controlled, then one could eliminate all the light switches - since the command to turn on or off a device could be provided by agents interacting with Badge servers.
See for example: the U.S. National Science Foundation workshop that examined the effect of architecture, building operations, and communications. A participant in this workshop noted that half of the cost of HVAC systems was the wiring to all the sensors and actuators. Using the Badge system could potentially eliminate the need to run wires to the sensors or in many cases even the need for particular sensors.
Note that one of the by-products of this approach is that the user has their own interface to appliances. Thus the user interface can be consistent for this user, not because each appliance supports this interface, but because the user's interface provides the consistency.
A simple example of this would be an automatic teller machine, which no longer needs to have a human centered interface, but now receives all its commands via the network as EDI requests. While the user's device provides whatever interface is necessary for this user to enable them to get cash. A Badge might replace smart cards in electronic commerce transactions by provide higher levels of security, as it could utilise a biomedical signature of the person currently using the Badge, ensuring that only a specific user can use it.
While this type of device might seem like a prop on Star TrekTM, all the components are available as products today. Voice and data communication, along with a graphics screen with WWW browser are features of the Nokia 9000 GSM phone. Handheld GPS units are currently sufficiently cheap and provide sufficient accuracy that the U.S. FCC has mandating that location information be available from mobile 911 (emergency) calls by 1999 [E911]. The U.S. DARPA Microelectromechanical Systems (MEMS)[MEMS] program has encouraged the production of highly integrated sensors. Thus, all that is required to realise a universal authentication, communications, and interaction peripheral is to integrate the available components and provide the conceptual framework of an environment aware computer system to allow applications to be developed.
In the course the students are using the Badges to provide a Location and Environment aware mobile infrastructure and then build novel mobile computing application using the infrastructure. Five example applications were constructed with the fifth application left to the imagination of the student groups. The four example applications are typical of the application of location aware mobile computers.
A Smart Badge and a network of fixed Badge Transceivers make it possible to track a persons movements around a building or larger environment. With the ability to track comes the ability to log the location data and analyse it. A Smart Room can be constructed by analysing location data for a room and extracting useful spatial and temporal correlations from it. The correlations can then be used by a Room Agent to control the workstations, door, lighting and environmental controls. The Room Agent could:
Turn on the overall room lights and adjusting the room temperature just before the first occupant arrives for the day.
Turn off the lighting, turning down the temperature and shutting down and turning off all computers just after the last occupant has gone for the day.
Turn task lighting off and on as occupants move during the day enabling screen-lock and monitor power-down when a computer workstation user moves away from the workstation for a prolonged period (e.g. lunch).
Adjusting the room environment so that all the occupants are as close to 20 degrees and 70% humidity as possible.
Report room energy trends to a Building Agent, which can then optimise the building energy usage and report building patterns to the Power Company to help them optimise their production.
Two extension to the Smart Room and Smart Door scenarios were developed by a student groups, Date IT is a location augmented dating service, Smart Hospital is a Badge augmented medical environment.
Date IT [BJE97] is a computer matchmaking system with location aware augmentation. The service itself provides mutual matching of members wearing badges, using a shared database and a transceiver infrastructure. A meeting between matching customers results in a signal to both badges, making them beep. The service is designed to be deployed under license in clubs and bars that also provide an information station to fill in the personal profile needed by the service. The information station (and WWW based database) can point the customer to bars and clubs with a high number of potential matches and can also be used to point out the location of potential matches within a room.
Smart Hospital [LAR97] (Figure 5) aims to improve information flows in hospitals. The system is based on displayed on a pen sensitive screen that works as an access point between the doctor, the patient and a central computer. The central computer stores hospital records and provides authentication and access control services. Patients are equipped with badges that report their current location to the system, have a panic button to summon help, and monitor biometric information like temperature, respiration and heart rate. The badges also provide access to smart doors to prevent patients (and staff) from entering restricted areas. The scheme can also be used to track the spread of infection or contamination be recording where people went, who they met and who else was near bye. An example of the systems use would be for an accident patient needing an X-Ray diagnosis. When the patient is admitted they are equipped with a badge. As the patient moves through the hospital each person that sees them and each procedure that is performed generates data in the central computer. The system allows the patients identity to be recorded along with the procedure results. When a doctor visits the patient in a ward, the patient's records are immediately available on the doctor's hand-held screen. The system not only improves patient care but also improves hospital record keeping and streamlines the billing and disbursement process.
A simple extension discussed in the City Guide system below is to provide the badge with navigational capability so staff and patients can find their way in a large hospital.
Record all the places the Smart Badge wearer went during the day, how long was spent at the location and who else was at the same location the same time (perhaps this indicates a meeting).
Assuming that a Smart Badge wearer accesses computer files on a single network file system from wherever they happen to be at any particular time during the day, the diary can be augmented with details of the files that were accessed and modified by the wearer during the day.
If the Smart Badge wearer has an online diary file containing their appointments the Diary Agent should correlate the appointment information for badge wearers with the actual meetings that took place and highlight the missed meetings that need to be rescheduled.
If a meeting (co-location of two wearers) takes place within range of a Badge equipped computer workstation that has a sound (video) input capability then the Diary Agent should be able to record the meeting and add the recording to the diary as a meeting record.
If the Diary Agent has access to PABX call records then they can be included in the diary along with the A party and B party numbers. The diary then contains a record of telephone contacts along with details of where the wearer was when the call occurred. If both parties were wearers then both locations could be recorded.
The Intelligent Fitness Centre, a variation on the diary agent, was devised by one of the student groups. Fitness centre patrons are equipped with a Smart Badge in a wristwatch form factor. These badges identify the patrons to the fitness centre and to its equipment that has Badge receivers attached and is connected through a local area network. When a patron uses a piece of exercise equipment their exercise profile is loaded into the machine allowing a fully tailored exercise program to be developed for each patron. The system also automates record keeping so performance measurements from the exercise machine can be logged to the patrons database for later analysis. The Badge is also used as an authentication device to bill patrons for their time at the fitness centre and even to control the type music played over the P.A. to match the tastes of the centre's current occupants. A simple extension to the system would integrate biometric sensors on the badge with the Fitness Centres monitoring equipment to provide records of pulse rate and respiration rates that would allow exercise physiologists to better tailor exercise programs to customers performance level and to ensure clients did not over-exert themselves.
After constructing a Super FLUMP the system was extended by students to create Sm@rt Guide and City Guide.
City Guide is meant to be used by tourist visiting a city or by residents as a replacement for a street map. It can also potentially be used as an electronic entry ticket. The City Guide uses an Intelligent Badge like devices incorporating a direction display and a GPS receiver. The system is coupled to Info Stations spread about the city. A customer selects a destination of interest (or a series of them) from an Info Station and the City Guide then navigates the customer through the city to the selected destinations using GPS information. All information and prompts can be displayed in the customer's own language. Figure 6 shows mock-ups of the City guide and Info Station displays.
The Sm@rt Guide simplifies the system for use in Exhibition Spaces. In Sm@rt Guide a Dumb Badge is used and the direction display is placed in the environment by using a network of Electronic Direction Signs (EDS) like smart street signs. Sm@rt Guide allows customers to be routed around temporary congestion in the Exhibition Space because the path is not pre-loaded into the badge at the Info Station.
The two systems can be easily combined to provide intelligent traffic routing in the City Guide scenario by placing EDS' in the environment (for example at already networked sites like traffic lights and public telephone booths) and then using GPS to navigate between EDS. Each EDS then has the opportunity to upload a new route to the City Guide badge as it passes to avoid short-term congestion. It should be noted that the system can be used equally well by pedestrians and motorists.
Our collection of publications and design documents (including code and schematics) are available from our WWW site as: http://www.elec.uow.edu.au/people/staff/beadle/badge/location_aware.html.