iSonic: Interactive Data Sonification for Blind Users
Figure 1: Sounds create the effect of a virtual map
Figure 2: Customized 3x3 map partition for exploration with a keyboard
Figure3: Highly coordinated table view and map view of Maryland counties
Figure 4: a user study participant
Project description:
Interactive data visualization tools are helpful to gain insight about data, find patterns and exceptions, but are usually non accessible to users with vision impairments. In the case of geo-referenced data where users need to combine demographic, economic or other data in a geographic context for decision-making, we designed iSonic, an interactive sonification tool that allows users to explore in highly coordinated table and choropleth map views of the data. Sounds of various timbres and pitches are tied to map regions and other interface widgets to create a virtual auditory data display. The integrated use of musical sounds and speech allows users to grasp the overall data trends and to explore for more details. We use the MIDI sound technique to provide high availability, and also use virtual 3-D sound technique, when available, to enhance user experiences.
Users use a standard computer keyboard, or a smooth surface touchpad when available, to interact with data. Examples of already implemented interactions include: (1) Automatically sweep to scan the map or the table to hear the data patterns; (2) Recursively partition the map into 3 by 3 ranges and use the keyboard number pad to explore each range, or use arrow keys to move among individual regions; (3) Glide a finger or press individual spots on a smooth surface touchpad to examine individual regions; The touchpad can be remapped to a partial map through zooming; (4) Dynamically adjust the auditory feedback information detail level. Our goal is to explore the design space by conducting user studies to identify effective sonification of choropleth maps and geo-referenced data, and examine the effectiveness of our tool in helping vision-impaired users. We also want to investigate the sonification of maps as a complement to visual maps for sighted users (e.g. to make "visible" the District of Columbia).
in collaboration with:
Dmitry Zotkin and Ramani Duraiswami (Perceptual Interfaces and Reality lab)
Ben Smith and Kent Norman (Dept. Psychology).
Franco Delogu and Marta Olivetti Belardinelli(University of Rome, Italy)
Jonathan Lazar (Towson University)
Also used in book chapter, in Lazar, J. (Ed.) Universal Usability, John Wiley & Sons Ltd., Hoboken, NJ (2008) 141-174
Zhao, H., Shneiderman, B., Plaisant, C., Listening to Choropleth Maps: Interactive Sonification of Geo-referenced Data for Users with Vision Impairment, in Lazar, J. (Ed.) Universal Usability, John Wiley & Sons Ltd., Hoboken, NJ (2008) 141-174
Zhao, H., Plaisant, C., Shneiderman, B., Delogu, F., Palmiero, M., Pasqualotto, E., Olivetti, M., iSonic: Interactive Sonification for Geo-referenced Data Exploration for the Vision Impaired,
(Demonstration) in Proc. Interact 2005
Zhao, H., Shneiderman, B., Plaisant, C., Zotkin, D., Duraiswami, R.
Improving Accessibility and Usability of Geo-referenced Statistical Data.
(Demonstration) in Proc. of the Digital Government Research Conference (2003) 315-316.
You can run the demo directly from the Web using the Java Web Start technology or download the package and run locally.
Option 1: Run from the Web (last updated on 7/17/2005)
Step 1: You need to have Java Web Start on your browser.
Step 2: Download and install Microsoft Speech SDK 5.1 (68 MB) if you are a first time user. Note: You need Microsoft Speech SDK 5.1 because iSonic currently produces synthesized speech through a speech server written based on the Microsoft Win32 Speech API. We may change to other Java based speech techniques in the future.
Step 3: Start iSonic speech server. When prompted "Open" or "Save", choose "open". A MS-DOS window will open and wait for speech commands from iSonic. Ctrl-C will close the window and the server.
Step 4: Launch iSonic through Java Web Start. Three maps (US state map, Maryland county map, and Idaho county map) are currently available with "2003 disability data". More maps and data will come soon. Please choose the following launching options.
Option 2: Download and run locally (US state map and data only) (last updated on 7/17/2005)
Step 1: Download and install Microsoft Speech SDK 5.1 (68 MB) if you are a first time user. Note: You need Microsoft Speech SDK 5.1 because iSonic currently produces synthesized speech through a speech server written based on the Microsoft Win32 Speech API. We plan to change to other Java based speech techniques soon.
Step 2: Download and install Java 2 Platform Standard Edition Running Environment. iSonic has been tested on J2SE v1.4.2 JRE although other versions may also work.
